diff options
author | Samuel Thibault <samuel.thibault@ens-lyon.org> | 2013-09-28 16:22:08 +0200 |
---|---|---|
committer | Samuel Thibault <samuel.thibault@ens-lyon.org> | 2013-09-28 16:22:08 +0200 |
commit | ca39ad0592e9b99dac9d99c68bb36ef1d27f72df (patch) | |
tree | 5ad12783d506039cd440ccfacbac264085137075 /open_issues | |
parent | be2307c1bf9aef3e22984dd298827d8e1ca18b2c (diff) | |
parent | 264b066cd313b23f6748711c6f9b4d3336e03136 (diff) |
Merge branch 'master' of braunbox:~hurd-web/hurd-web
Diffstat (limited to 'open_issues')
64 files changed, 7471 insertions, 278 deletions
diff --git a/open_issues/64-bit_port.mdwn b/open_issues/64-bit_port.mdwn index 66da44b9..b0c95612 100644 --- a/open_issues/64-bit_port.mdwn +++ b/open_issues/64-bit_port.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2012, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -137,3 +138,20 @@ See also [[microkernel/mach/gnumach/memory_management]]. <braunr> hm actually no, it would require mcmodel=large <braunr> hum, that's stupid, we can make the kernel run at -2g, and use 3g up to the sign extension hole for the kernel map + + +# IRC, freenode, #hurd, 2013-07-02 + +In context of [[mondriaan_memory_protection]]. + + <xscript> BTW, it's not like I have an infinite amount of time for this, + but having 64-bit support would be valuable for me, so I might contribute + that back if it's not a too monumental task + <xscript> I saw some discussions about 32bit apps on top of 64bit mach, but + I'd like a full 64bit system + <xscript> any clues? + <xscript> I suppose the compiler support is all there already + <xscript> is MIG (and mach) the only piece missing? + <braunr> the problem is the interfaces themselves + <braunr> type widths + <braunr> as passed between userspace and kernel diff --git a/open_issues/anatomy_of_a_hurd_system.mdwn b/open_issues/anatomy_of_a_hurd_system.mdwn index 677e4625..ba72b00f 100644 --- a/open_issues/anatomy_of_a_hurd_system.mdwn +++ b/open_issues/anatomy_of_a_hurd_system.mdwn @@ -323,7 +323,9 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l <braunr> etc.. -# IRC, freenode, #hurd, 2012-12-06 +# Service Directory + +## IRC, freenode, #hurd, 2012-12-06 <spiderweb> what is the #1 feature that distinguished hurd from other operating systems. the concept of translators. (will read more when I get @@ -333,6 +335,132 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l <braunr> and the VFS permissions to control access to those services +## IRC, freenode, #hurd, 2013-05-23 + + <gnu_srs> Hi, is there any efficient way to control which backed + translators are called via RPC with a user space program? + <gnu_srs> Take for example io_stat: S_io_stat is defined in boot/boot.c, + pfinet/io-ops.c and pflocal/io.c + <gnu_srs> And the we have libdiskfs/io-stat.c:diskfs_S_io_stat, + libnetfs/io-stat.c:netfs_S_io_stat, libtreefs/s-io.c:treefs_S_io_stat, + libtrivfs/io-stat.c:trivfs_S_io_stat + <gnu_srs> How are they related? + <braunr> gnu_srs: it depends on the server (translator) managing the files + (nodes) you're accessing + <braunr> so use fsysopts to know the server, and see what this server uses + <gnu_srs> fsysopts /hurd/pfinet and fsysopts /hurd/pflocal gives the same + answer: ext2fs --writable --no-inherit-dir-group --store-type=typed + device:hd0s1 + <braunr> of course + <braunr> the binaries are regular files + <braunr> see /servers/socket/1 and /servers/socket/2 instead + <braunr> which are the nodes representing the *service* + <braunr> again, the hurd uses the file system as a service directory + <braunr> this usage of the file system is at the core of the hurd design + <braunr> files are not mere files, they're service names + <braunr> it happens that, for most files, the service behind them is the + same as for regular files + <braunr> gnu_srs: this *must* be obvious for you to do any tricky work on + the hurd + + <gnu_srs> Anyway, if I create a test program calling io_stat I assume + S_io_stat in pflocal is called. + <gnu_srs> How to make the program call S_io_stat in pfinet instead? + <braunr> create a socket managed by pfinet + <braunr> i.e. an inet or inet6 socket + <braunr> you can't assume io_stat is serviced by pflocal + <braunr> only stats on unix sockets of pipes will be + <braunr> or* + <gnu_srs> thanks, what about the *_S_io_stat functions? + <braunr> what about them ? + <gnu_srs> How they fit into the picture, e.g. diskfs_io_stat? + <gnu_srs> *diskfs_S_io_stat + <braunr> gnu_srs: if you open a file managed by a server using libdiskfs, + e.g. ext2fs, that one will be called + <gnu_srs> Using the same user space call: io_stat, right? + <braunr> it's all userspace + <braunr> say rather, client-side + <braunr> the client calls the posix stat() function, which is implemented + by glibc, which converts it into a call to io_stat, and sends it to the + server managing the open file + <braunr> the io_stat can change depending on the server + <braunr> the remote io_stat implementation, i mean + <braunr> identify the server, and you will identify the actual + implementation + + +## IRC, freenode, #hurd, 2013-06-30 + + <hacklu> hi, what is the replacer of netname_check_in? + + <hacklu> I want to ask another question. in my opinion, the rpc is the + mach's way, and the translator is the hurd's way. so somebody want to + lookup a service, it should not need to ask the mach kernel know about + this query. the hurd will take the control. + <hacklu> am I right? + <braunr> no + <braunr> that's nonsense + <braunr> service lookups has never been in mach + <braunr> first mach based systems used a service directory, whereas the + hurd uses the file system for that + <braunr> you still need mach to communicate with either of those + <hacklu> how to understand the term of service directory here? + <braunr> a server everyone knows + <braunr> which gives references to other servers + <braunr> usually, depending on the name + <braunr> e.g. name_lookup("net") -> port right to network server + <hacklu> is that people use netname_check_in to register service in the + past? now used libtrivfs? + <braunr> i don't know about netname_check_in + <braunr> old mach (not gnumach) documentation might mention this service + directory + <braunr> libtrivfs doesn't have much to do with that + <braunr> on the hurd, the equivalent is the file system + <hacklu> maybe that is outdate, I just found that exist old doc, and old + code which can't be build. + <braunr> every process knows / + <braunr> the file system is the service directory + <braunr> nodes refer to services + <hacklu> so the file system is the nameserver, any new service should + register in it before other can use + <braunr> and the file system is distributed, so looking up a service may + require several queries + <braunr> setting a translator is exactly that, registering a program to + service requests on a node + <braunr> the file system isn't one server though + <braunr> programs all know about /, but then, lookups are recursive + <braunr> e.g. if you have / and /home, and are looking for + /home/hacklu/.profile, you ask / which tells you about /home, and /home + will give you a right to /home/hacklu/.profile + <hacklu> even in the past, the mach don't provide name register service, + there must be an other server to provide this service? + <braunr> yes + <braunr> what's nonsense in your sentence is comparing RPCs and translators + <braunr> translators are merely servers attached to the file system, using + RPCs to communicate with the rest of the system + <hacklu> I know yet, the two just one thing. + <braunr> no + <braunr> two things :p + <braunr> completely different and unrelated except for one using the other + <hacklu> ah, just one used aonther one. + <hacklu> is exist anyway to anounce service except settrans with file node? + <braunr> more or less + <braunr> tasks can have special ports + <braunr> that's how one task knows about / for example + <braunr> at task creation, a right to / is inserted in the new task + <hacklu> I think this is also a file node way. + <braunr> no + <braunr> if i'm right, auth is referenced the same way + <braunr> and there is no node for auth + <hacklu> how the user get the port of auth with node? + <braunr> it's given when a task is created + <hacklu> pre-set in the creation of one task? + <braunr> i'm unconfortable with "pre-set" + <braunr> inserted at creation time + <braunr> auth is started very early + <braunr> then tasks are given a reference to it + + # IRC, freenode, #hurd, 2012-12-10 <spiderweb> I want to work on hurd, but I think I'm going to start with @@ -380,3 +508,298 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l <braunr> if you're looking for how to do it for a non-translator application, the answer is probably somewhere in glibc <braunr> _hurd_startup i'd guess + + +# IRC, freenode, #hurd, 2013-06-15 + + <damo22> ive been reading a little about exokernels or unikernels, and i + was wondering if it might be relevant to the GNU/hurd design. I'm not + too familiar with hurd terminology so forgive me. what if every + privileged service was compiled as its own mini "kernel" that handled (a) + any hardware related to that service (b) any device nodes exposed by that + service etc... + <braunr> yes but not really that way + <damo22> under the current hurd model of the operating system, how would + you talk to hardware that required specific timings like sound hardware? + <braunr> through mapped memory + <damo22> is there such a thing as an interrupt request in hurd? + <braunr> obviously + <damo22> ok + <damo22> is there any documentation i can read that involves a driver that + uses irqs for hurd? + <braunr> you can read the netdde code + <braunr> dde being another project, there may be documentation about it + <braunr> somewhere else + <braunr> i don't know where + <damo22> thanks + <damo22> i read a little about dde, apparently it reuses existing code from + linux or bsd by reimplementing parts of the old kernel like an api or + something + <braunr> yes + <damo22> it must translate these system calls into ipc or something + <damo22> then mach handles it? + <braunr> exactly + <braunr> that's why i say it's not the exokernel way of doing things + <damo22> ok + <damo22> so does every low level hardware access go through mach?' + <braunr> yes + <braunr> well no + <braunr> interrupts do + <braunr> ports (on x86) + <braunr> everything else should be doable through mapped memory + <damo22> seems surprising that the code for it is so small + <braunr> 1/ why surprising ? and 2/ "so small" ? + <damo22> its like the core of the OS, and yet its tiny compared to say the + linux kernel + <braunr> it's a microkenrel + <braunr> well, rather an hybrid + <braunr> the size of the equivalent code in linux is about the same + <damo22> ok + <damo22> with the model that privileged instructions get moved to + userspace, how does one draw the line between what is OS and what is user + code + <braunr> privileged instructions remain in the kernel + <braunr> that's one of the few responsibilities of the kernel + <damo22> i see, so it is an illusion that the user has privilege in a sense + <braunr> hum no + <braunr> or, define "illusion" + <damo22> well the user can suddenly do things never imaginable in linux + <damo22> that would have required sudo + <braunr> yes + <braunr> well, they're not unimaginable on linux + <braunr> it's just not how it's meant to work + <damo22> :) + <braunr> and why things like fuse are so slow + <braunr> i still don't get "i see, so it is an illusion that the user has + privilege in a sense" + <damo22> because the user doesnt actually have the elevated privilege its + the server thing (translator)? + <braunr> it does + <braunr> not at the hardware level, but at the system level + <braunr> not being able to do it directly doesn't mean you can't do it + <damo22> right + <braunr> it means you need indirections + <braunr> that's what the kernel provides + <damo22> so the user cant do stuff like outb 0x13, 0x1 + <braunr> he can + <braunr> he also can on linux + <damo22> oh + <braunr> that's an x86 specifity though + <damo22> but the user would need hardware privilege to do that + <braunr> no + <damo22> or some kind of privilege + <braunr> there is a permission bitmap in the TSS that allows userspace to + directly access some ports + <braunr> but that's really x86 specific, again + <damo22> i was using it as an example + <damo22> i mean you wouldnt want userspace to directly access everything + <braunr> yes + <braunr> the only problem with that is dma reall + <braunr> y + <braunr> because dma usually access physical memory directly + <damo22> are you saying its good to let userspace access everything minus + dma? + <braunr> otherwise you can just centralize permissions in one place (the + kernel or an I/O server for example) + <braunr> no + <braunr> you don't let userspace access everything + <damo22> ah + <damo22> yes + <braunr> userspace asks for permission to access one specific part (a + memory range through mapping) + <braunr> and can't access the rest (except through dma) + <damo22> except through dma?? doesnt that pose a large security threat? + <braunr> no + <braunr> you don't give away dma access to anyone + <braunr> only drivers + <damo22> ahh + <braunr> and drivers are normally privileged applications anyway + <damo22> so a driver runs in userspace? + <braunr> so the only effect is that bugs can affect other address spaces + indirectly + <braunr> netdde does + <damo22> interesting + <braunr> and they all should but that's not the case for historical reasons + <damo22> i want to port ALSA to hurd userspace :D + <braunr> that's not so simple unfortunately + <braunr> one of the reasons it's hard is that pci access needs arbitration + <braunr> and we don't have that yet + <damo22> i imagine that would be difficult + <braunr> yes + <braunr> also we're not sure we want alsa + <braunr> alsa drivers, maybe, but probably not the interface itself + <damo22> its tangled spaghetti + <damo22> but the guy who wrote JACK for audio hates OSS, and believes it is + rubbish due to the fact it tries to read and write to a pcm device node + like a filesystem with no care for timing + <braunr> i don't know audio well enough to tell you anything about that + <braunr> was that about oss3 or oss4 ? + <braunr> also, the hurd isn't a real time system + <braunr> so we don't really care about timings + <braunr> but with "good enough" latencies, it shouldn't be a problem + <damo22> but if the audio doesnt reach the sound card in time, you will get + a crackle or a pop or a pause in the signal + <braunr> yep + <braunr> it happens on linux too when the system gets some load + <damo22> some users find this unnacceptable + <braunr> some users want real time systems + <braunr> using soft real time is usually plenty enough to "solve" this kind + of problems + <damo22> will hurd ever be a real time system? + <braunr> no idea + <youpi> if somebody works on it why not + <youpi> it's the same as linux + <braunr> it should certainly be simpler than on linux though + <damo22> hmm + <braunr> microkernels are well suited for real time because of the well + defined interfaces they provide and the small amount of code running in + kernel + <damo22> that sounds promising + <braunr> you usually need to add priority inheritance and take care of just + a few corner cases and that's all + <braunr> but as youpi said, it still requires work + <braunr> and nobody's working on it + <braunr> you may want to check l4 fiasco.oc though + + +# System Personality + +## IRC, freenode, #hurd, 2013-07-29 + + <teythoon> over the past few days I gained a new understanding of the Hurd + <braunr> teythoon: really ? :) + <tschwinge> teythoon: That it's a complex and distributed system? ;-) + <tschwinge> And at the same time a really simple one? + <tschwinge> ;-D + <teythoon> it's just a bunch of mach programs and some do communicate and + behave in a way a posix system would, but that is more a convention than + anything else + <teythoon> tschwinge: yes, kind of simple and complex :) + <braunr> the right terminology is "system personality" + <braunr> 11:03 < teythoon> over the past few days I gained a new + understanding of the Hurd + <braunr> teythoon: still no answer on that :) + <teythoon> braunr: ah, I spent lot's of time with the core servers and + early bootstrapping and now I gained the feeling that I've seen the Hurd + for what it really is for the first time + + +# RPC Interfaces + +## IRC, freenode, #hurd, 2013-09-03 + + <rekado> I'm a little confused by the hurd and incubator git repos. + <rekado> DDE is only found in the dde branch in incubator, but not in the + hurd repo. + <rekado> Does this mean that DDE is not ready for master yet? + <braunr> yes + <rekado> If DDE is not yet used in the hurd (except in the dde branch in + the incubator repo), does pfinet use some custom glue code to use the + Linux drivers? + <braunr> this has nothing to do with pfinet + <braunr> pfinet is the networking stack, netdde are the networking drivers + <braunr> the interface between them doesn't change, whether drivers are in + kernel or not + <rekado> I see + + +# IRC, freenode, #hurd, 2013-09-20 + + <giuscri> HI there, I have no previous knowledge about OS's. I'm trying to + undestand the structure of the Hurd and the comparison between, say, + Linux way of managing stuff ... + <giuscri> for instance, I read: "Unlike other popular kernel software, the + Hurd has an object-oriented structure that allows it to evolve without + compromising its design." + <giuscri> that means that while for adding feature to the Linux-kernel you + have to add some stuff `inside` a procedure, whilst in the Hurd kernel + you can just, in principle at least, add an object and making the kernel + using it?... + <giuscri> Am I making stuff too simple? + <giuscri> Thanks + <braunr> not exactly + <braunr> unix historically has a "file-oriented" structure + <braunr> the hurd allows servers to implement whatever type they want, + through the ability to create custom interfaces + <braunr> custom interfaces means custom calls, custom semantics, custom + methods on objects + <braunr> you're not restricted to the set of file interfaces (open, seek, + read, write, select, close, etc..) that unix normally provides + <giuscri> braunr: uhm ...some example? + <braunr> see processes for example + <braunr> see + http://darnassus.sceen.net/gitweb/savannah_mirror/hurd.git/tree/HEAD:/hurd + <braunr> this is the collection of interfaces the hurd provides + <braunr> most of them map to unix calls, because gnu aims at posix + compatibility too + <braunr> some are internal, like processes + <braunr> or authentication + <braunr> but most importantly, you're not restricted to that, you can add + your own interfaces + <braunr> on a unix, you'd need new system calls + <braunr> or worse, extending through the catch-all ioctl call + <giuscri> braunr: mhn ...sorry, not getting that. + <braunr> what part ? + <kilobug> ioctl has become such a mess :s + <giuscri> braunr: when you say that Unix is `file-oriented` you're + referring to the fact that sending/receiving data to/from the kernel is + designed like sending/receiving data to/from a file ...? + <braunr> not merely sending/receiving + <braunr> note how formatted your way of thinking is + <braunr> you directly think in terms of sending/receiving (i.e. read and + write) + <giuscri> braunr: (yes) + <braunr> that's why unix is file oriented, access to objects is done that + way + <braunr> on the hurd, the file interface is one interface + <braunr> there is nothing preventing you from implementing services with a + different interface + <braunr> as a real world example, people interested in low latency + profesionnal audio usually dislike send/recv + <braunr> see + http://lac.linuxaudio.org/2003/zkm/slides/paul_davis-jack/unix.html for + example + <kilobug> braunr: how big and messy ioctl has become is a good proof that + the Unix way, while powerful, does have its limits + <braunr> giuscri: keep in mind the main goal of the hurd is extensibility + without special privileges + <giuscri> braunr: privileges? + <braunr> root + <giuscri> braunr: what's wrong with privileges? + <braunr> they allow malicious/buggy stuff to happne + <braunr> and have dramatic effects + <giuscri> braunr: you're obviously *not* referring to the fact that once + one have the root privileges could change some critical-data + <giuscri> ? + <braunr> i'm referring to why privilege separation exists in the first + place + <braunr> if you have unprivileged users, that's because you don't want them + to mess things up + <braunr> on unix, extending the system requires privileges, giving those + who do it the ability to destroy everything + <giuscri> braunr: yes, I think the same + <braunr> the hurd is designed to allow unprivileged users to extend their + part of the system, and to some extent share that with other users + <braunr> although work still remains to completely achieve that + <giuscri> braunr: mhn ...that's the `server`-layer between the + single-application and kernel ? + <braunr> the multi-server based approach not only allows that, but + mitigates damage even when privileged servers misbehave + <braunr> one aspect of it yes + <braunr> but as i was just saying, even root servers can't mess things too + much + <braunr> for example, our old (sometimes buggy) networking stack can be + restarted when it behaves wrong + <braunr> the only side effect being some applications (ssh and exim come to + mind) which need to be restarted too because they don't expect the + network stack to be restarted + <giuscri> braunr: ...instead? + <braunr> ? + <kilobug> giuscri: on Linux, if the network stack crash/freezes, you don't + have any other option than rebooting the system - usually with a nice + "kernel pani" + <kilobug> giuscri: and you may even get filesystem corruption "for free" in + the bundle + <braunr> and hoping it didn't corrupt something important like file system + caches before being flushed + <giuscri> kilobug, braunr : mhn, ook diff --git a/open_issues/arm_port.mdwn b/open_issues/arm_port.mdwn index 8a2a037f..ebbad1a4 100644 --- a/open_issues/arm_port.mdwn +++ b/open_issues/arm_port.mdwn @@ -274,3 +274,55 @@ architecture. duplicate efforts. <braunr> little addition: it may have started, but we don't know about it + +# IRC, freenode, #hurd, 2013-09-18 + + <Hooligan0> as i understand ; on startup, vm_resident.c functions configure + the whole available memory ; but at this point the system does not split + space for kernel and space for future apps + <Hooligan0> when pages are tagged to be used by userspace ? + <braunr> Hooligan0: at page fault time + <braunr> the split is completely virtual, vm_resident deals with physical + memory only + <Hooligan0> braunr: do you think it's possible to change (at least) + pmap_steal_memory to mark somes pages as kernel-reserved ? + <braunr> why do you want to reserve memory ? + <braunr> and which memory ? + <Hooligan0> braunr: first because on my mmu i have two entry points ; so i + want to set kernel pages into a dedicated space that never change on + context switch (for best cache performance) + <Hooligan0> braunr: and second, because i want to use larger pages into + kernel (1MB) to reduce mmu work + <braunr> vm_resident isn't well suited for large pages :( + <braunr> i don't see the effect of context switch on kernel pages + <Hooligan0> at many times, context switch flush caches + <braunr> ah you want something like global pages on x86 ? + <Hooligan0> yes, something like + <braunr> how is it done on arm ? + <Hooligan0> virtual memory is split into two parts depending on msb bits + <Hooligan0> for example 3G/1G + <Hooligan0> MMU will use two pages tables depending on vaddr (hi-side or + low-side) + <braunr> hi is kernel, low is user ? + <Hooligan0> so, for the moment i've put mach at 0xC0000000 -> 0xFFFFFFFF ; + and want to use 0x00000000 -> 0xBFFFFFFF for userspace + <Hooligan0> yes + <braunr> ok, that's what is done for x86 too + <Hooligan0> 1MB pages for kernel ; and 4kB (or 64kB) pages for apps + <braunr> i suggest you give up the large page stuff + <braunr> well, you can use them for the direct physical mapping, but for + kernel objects, it's a waste + <braunr> or you can rewrite vm_resident to use something like a buddy + allocator but it's additional work + <Hooligan0> for the moment it's waste ; but with some littles changes this + allow only one level of allocation mapping ; -i think- it's better for + performances + <braunr> Hooligan0: it is, but not worth it + <Hooligan0> will you allow changes into vm_resident if i update i386 too ? + <braunr> Hooligan0: sure, as long as these are relevant and don't introduce + regressions + <Hooligan0> ok + <braunr> Hooligan0: i suggest you look at x15, since you may want to use it + as a template for your own changes + <braunr> as it was done for the slab allocator for example + <braunr> e.g. x15 already uses a buddy allocator for physical memory diff --git a/open_issues/binutils.mdwn b/open_issues/binutils.mdwn index 641083a7..b19a81a4 100644 --- a/open_issues/binutils.mdwn +++ b/open_issues/binutils.mdwn @@ -39,8 +39,8 @@ git diff --patience --stat=$COLUMNS,$COLUMNS --patch --src-prefix=./ --dst-prefi --> -Last reviewed up to the [[Git mirror's e7e89a91affc920f0d422c56ca04867e49ac5ac1 -(2013-05-27) sources|source_repositories/binutils]]. +Last reviewed up to the [[Git mirror's d2a61dc33b01c56f5153c1ddea7b1fb8f304f20d +(2013-06-18) sources|source_repositories/binutils]]. * Globally @@ -128,11 +128,11 @@ Last reviewed up to the [[Git mirror's e7e89a91affc920f0d422c56ca04867e49ac5ac1 Here's a log of a binutils build run; this is from our [[Git repository|source_repositories/binutils]]'s `tschwinge/Paul_Desmond` branch, -commit e7e89a91affc920f0d422c56ca04867e49ac5ac1 (2013-05-27), run on +commit d2a61dc33b01c56f5153c1ddea7b1fb8f304f20d (2013-06-18), run on kepler.SCHWINGE and coulomb.SCHWINGE. $ export LC_ALL=C - $ ../Paul_Desmond/configure --prefix="$PWD".install --enable-gold --with-sysroot=/ SHELL=/bin/dash CC=gcc-4.7 CXX=g++-4.7 2>&1 | tee log_build + $ ../Paul_Desmond/configure --prefix="$PWD".install --enable-gold --with-sysroot=/ SHELL=/bin/dash CC=gcc-4.8 CXX=g++-4.8 2>&1 | tee log_build [...] $ make 2>&1 | tee log_build_ [...] @@ -142,8 +142,8 @@ harmonized. Debian GCC (which is used in binutils' testsuite) likes to pass `--sysroot=/` to `ld`, so we need to configure binutils with support for sysroots. -This takes up around 950 MiB, and needs roughly 13 min on kepler.SCHWINGE and -45 min on coulomb.SCHWINGE. +This takes up around 1100 MiB, and needs roughly 13 min on kepler.SCHWINGE and +53 min on coulomb.SCHWINGE. <!-- @@ -160,20 +160,13 @@ formats, and more emulation vectors. $ toolchain/logs/process binutils build - * gold GNU/Linux vs. GNU/Hurd - - -checking for glibc ifunc support... both - +checking for glibc ifunc support... dyn - - Missing [[IFUNC]] support on GNU/Hurd. - # Install $ make install 2>&1 | tee log_install [...] -This takes up around 160 MiB, and needs roughly 1 min on kepler.SCHWINGE and 3 +This takes up around 170 MiB, and needs roughly 1 min on kepler.SCHWINGE and 3 min on coulomb.SCHWINGE. @@ -189,7 +182,7 @@ min on coulomb.SCHWINGE. $ make -k check 2>&1 | tee log_test [...] -This needs roughly 6 min on kepler.SCHWINGE and 47 min on coulomb.SCHWINGE. +This needs roughly 5 min on kepler.SCHWINGE and 37 min on coulomb.SCHWINGE. ## Analysis @@ -220,7 +213,7 @@ This needs roughly 6 min on kepler.SCHWINGE and 47 min on coulomb.SCHWINGE. symbol handling in glibc, needed for our external [[/libpthread]]. TODO: document properly. - * `FAIL: gas/i386/rept` + * `FAIL: gas/i386/rept` (intermittently) Added in commit 06f1247c54126b9f1e6acb8ff8c7be35aec6f44c (2012-06-07) as part of the fix for [[!sourceware_PR 14201]], renamed in commit @@ -251,14 +244,3 @@ This needs roughly 6 min on kepler.SCHWINGE and 47 min on coulomb.SCHWINGE. -PASS: tls_phdrs_script_test +FAIL: tls_phdrs_script_test - - -PASS: ifuncmain1static - -PASS: ifuncmain1picstatic - -PASS: ifuncmain2static - -PASS: ifuncmain2picstatic - -PASS: ifuncmain4static - -PASS: ifuncmain4picstatic - -PASS: ifuncmain5static - -PASS: ifuncmain5picstatic - -PASS: ifuncmain7static - -PASS: ifuncmain7picstatic diff --git a/open_issues/boehm_gc.mdwn b/open_issues/boehm_gc.mdwn index 7f860bba..623dcb83 100644 --- a/open_issues/boehm_gc.mdwn +++ b/open_issues/boehm_gc.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2010, 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2012, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -433,3 +434,92 @@ restults of GNU/Linux and GNU/Hurd look very similar. <civodul> pinotree: is it a Debian-specific change, or included upstream? <pinotree> libgc using SIGUSR1/2? upstream <civodul> ok + + +### IRC, freenode, #hurd, 2013-09-03 + + <congzhang> braunr: when will libc malloc say memory corruption? + <braunr> congzhang: usually on free + <braunr> sometimes on alloc + <congzhang> and after one thread be created + <congzhang> I want to know why and how to find the source + <congzhang> does libgc work well on hurd? + <braunr> i don't think it does + <congzhang> so , why it can't? + <braunr> congzhang: what ? + <congzhang> libgc was not work on hurd + <pinotree> why? + <congzhang> I try porting dotgnu + <braunr> ah + <braunr> nested signal handling + <congzhang> one program always receive Abort signal + <pinotree> and why it should be a problem in libgc? + <congzhang> for malloc memory corruption + <braunr> libgc relies on this + <congzhang> yes + <congzhang> so, is there a workaround to make it work? + <braunr> show the error please + <congzhang> http://paste.debian.net/34416/ + <pinotree> where's libgc? + <congzhang> i compile dotgnu with enable-gc + <pinotree> so? + <congzhang> I am not sure about it + <pinotree> so why did you say earlier that libgc doesn't work? + <congzhang> because after I see one thread was created notice by gdb, it + memory corruption + <pinotree> so what? + <congzhang> maybe gabage collection happen, and gc thread start + <pinotree> that's speculation + <pinotree> you cannot debug things speculating on code you don't know + <pinotree> less speculation and more in-deep debugging, please + * congzhang I try again, to check weather thread list changing + <congzhang> sorry for this + <braunr> it simply looks like a real memory corruption (an overflow) + <congzhang> maybe PATH related problem + <pinotree> PATH? + <congzhang> yes + <braunr> PATH_MAX + <braunr> but unlikely + <congzhang> csant do path traverse + <congzhang> I fond the macro + <congzhang> found + <congzhang> #if defined(__sun__) || defined(__BEOS__) + <congzhang> #define BROKEN_DIRENT 1 + <congzhang> #endif + <congzhang> and so for hurd? + <pinotree> BROKEN_DIRENT doesn't say much about what it does + <WhiteKIBA> nope + <WhiteKIBA> whoops + <congzhang> it seems other port meet the trouble too + <pinotree> which trouble? + <congzhang> http://comments.gmane.org/gmane.comp.gnu.dotgnu.developer/3642 + <congzhang> (gdb) ptype struct dirent + <congzhang> type = struct dirent { + <congzhang> __ino_t d_ino; + <congzhang> unsigned short d_reclen; + <congzhang> unsigned char d_type; + <congzhang> unsigned char d_namlen; + <congzhang> char d_name[1]; + <congzhang> } + <congzhang> + <congzhang> d_name should be char[PATH_MAX]? + <congzhang> and + http://libjit-linear-scan-register-allocator.googlecode.com/svn/trunk/pnet/support/dir.c + <pinotree> no + <braunr> stop pasting that much + <_d3f> uhm PATH_MAX on the hurd? + <braunr> and stop saying nonsense + <congzhang> sorry, i think four line was not worth to pastbin + <pinotree> they are 8 + <congzhang> never again + <braunr> just try by defining BROKEN_DIRENT to 1 in all cases and see how + it goes + * congzhang read dir.c again + <congzhang> braunr: it does not crash this time, I do more test + + +#### IRC, freenode, #hurd, 2013-09-04 + + <congzhang> hi, I am dotgnu work on hurd, and even winforms app + <congzhang> s/am/make + <congzhang> and maybe c# hello world translate another day :) diff --git a/open_issues/clock_gettime.mdwn b/open_issues/clock_gettime.mdwn index 98454d45..65ab52df 100644 --- a/open_issues/clock_gettime.mdwn +++ b/open_issues/clock_gettime.mdwn @@ -197,4 +197,14 @@ In context of [[select]]. "atomic" update of the struct with time :) +# IRC, freenode, #hurd, 2013-09-04 + + <teythoon> do we have CLOCK_MONOTONIC ? + <braunr> teythoon: i think we do but it's actually a simple offset from + CLOCK_REALTIME .. :) + <teythoon> ah never mind, I do hate this posix time interface anyways + <braunr> really ? + <braunr> i think librt is decent + + # Candidate for [[vDSO]] code? diff --git a/open_issues/code_analysis.mdwn b/open_issues/code_analysis.mdwn index bdd2ae18..67798c6a 100644 --- a/open_issues/code_analysis.mdwn +++ b/open_issues/code_analysis.mdwn @@ -193,3 +193,17 @@ There is a [[!FF_project 276]][[!tag bounty]] on some of these tasks. * [Trinity: A Linux kernel fuzz tester (and then some)](http://www.socallinuxexpo.org/scale11x/presentations/trinity-linux-kernel-fuzz-tester-and-then-some), Dave Jones, The Eleventh Annual Southern California Linux Expo, 2013. + + * Mayhem, *an automatic bug finding system* + + IRC, freenode, #hurd, 2013-06-29: + + <teythoon> started reading the mayhem paper referenced here + http://lists.debian.org/debian-devel/2013/06/msg00720.html + <teythoon> that's nice work, they are doing symbolic execution of x86 + binary code, that's effectively model checking with some specialized + formulas + <teythoon> (too bad the mayhem code isn't available, damn those + academic people keeping the good stuff to themselvs...) + <teythoon> (and I really think that's bad practice, how should anyone + reproduce their results? that's not how science works imho...) diff --git a/open_issues/crash_server.mdwn b/open_issues/crash_server.mdwn index 7ed4afbf..5182df6f 100644 --- a/open_issues/crash_server.mdwn +++ b/open_issues/crash_server.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2009, 2010, 2011 Free Software Foundation, +[[!meta copyright="Copyright © 2009, 2010, 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -189,6 +189,65 @@ one... mach_msg_trap /home/tschwinge/tmp/gnumach/gnumach-1-branch-Xen-branch.build/../gnumach-1-branch-Xen-branch/ipc/mach_msg.c:1367 + +# IRC, freenode, #hurd, 2013-09-07 + + <rekado> I'm trying to investigate a crash in pfinet, so it will actually + die. I just want to know why it dies and what the value of a few + variables has been when it died. + <teythoon> have you tried to make it dump core? + <rekado> oh, good idea. + <rekado> I'll try that. + <teythoon> do you know how? + <rekado> I don't, but I think I can figure it out. + <teythoon> look into /servers + <rekado> do I just have to set CRASHSERVER=/servers/crash-dump-core and run + pfinet in that environment? + <teythoon> possibly, I've never heard of CRASHSERVER, but it's certainly + plausible ;) + <teythoon> I just link crash to crash-dump-core, that way it is permanent + and for all processes + <rekado> found it in the website contents + <rekado> gotta try that. + <rekado> hmm, I can't get pfinet to dump core; linked /servers/crash to + /servers/crash-dump-core and compiled pfinet to raise(6) at one point. + <rekado> But no core file is created. + <teythoon> :/ + <teythoon> rekado: try cd /tmp ; cat & kill -SIGILL %% to see if that dumps + core + <rekado> yes, this works. + <rekado> I replaced the original pfinet with my crashing version. + <rekado> Should it dump core to /hurd then? + <teythoon> I'm not sure about it's wd + <teythoon> hm, ok, I just did settrans -ca foo /hurd/pfinet and then killed + that pfient with SIGILL and it dumped core + <teythoon> to the directory I issued the settrans from + <rekado> So I must run it myself. I can't just replace the original binary + and have it dump core somewhere. + <teythoon> it seems that you have to use settrans -ca to start an active + translator + <teythoon> do fsysopts /servers/socket/2 to find out the cmdline of your + pfinet + <rekado> that's very helpful. + <rekado> thanks + <teythoon> then use this to restart it, e.g.: + <teythoon> settrans -afg /servers/socket/2 $(fsysopts /servers/socket/2) + <teythoon> if it dies it should dump core to you cwd + <rekado> great. Thank you very much. I had been wondering how to get the + full cmdline of pfinet. + * rekado makes a note of fsysopts + <rekado> yup, there's the core file. Nice. + <teythoon> cool 8D + <teythoon> btw, in case using gdb doesn't work out for your problem, if you + start pfinet (or any translator) this way (with -a == active), you can + write stuff to stderr + <rekado> yeah, I noticed that. The assert() call wrote to stderr. Useful. + <braunr> rekado: core dumps are another not-working-well feature of the + hurd :/ + <braunr> i recommend attaching + <tschwinge> rekado: In case that's still helpful: + <http://www.gnu.org/software/hurd/hurd/debugging/translator.html>. + --- If someone is working in this area, they may want to have a look at diff --git a/open_issues/dbus.mdwn b/open_issues/dbus.mdwn index 2f02579e..a41515a1 100644 --- a/open_issues/dbus.mdwn +++ b/open_issues/dbus.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2012, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -14,15 +15,17 @@ The dbus problems are due to missing scm credentials [[sendmsg_scm_creds]] and s [[pflocal_socket_credentials_for_local_sockets]]. There was also a problem with short timeout in [[select]], but that has been solved in Debian by setting a minimum timeout of 1ms. ---- +[[!toc]] -IRC, freenode, #hurd, 2011-11-26: + +# IRC, freenode, #hurd, 2011-11-26 <antrik> BTW, how much effort is necessary to fix dbus? <pinotree> basically, have pflocal know who's the sender (pid/uid/gid/groups) in the socket send op -IRC, freenode, #hurd, 2011-12-16: + +# IRC, freenode, #hurd, 2011-12-16 <braunr> pinotree: what's the problem with dbus ? <pinotree> braunr: select() returning 0 changed fd's with very short (eg < @@ -53,7 +56,8 @@ IRC, freenode, #hurd, 2011-12-16: <braunr> hm i agree with neal, i don't understand why the timeout is given to the kernel as part of the mach_msg call -IRC, freenode, #hurd, 2011-12-20: + +# IRC, freenode, #hurd, 2011-12-20 <braunr> hm, i don't see any occurrence of SCM_CREDENTIALS in dbus <braunr> only SCM_RIGHTS @@ -88,3 +92,164 @@ IRC, freenode, #hurd, 2011-12-20: <pinotree> iirc roland didn't like one or more parts of it (but i could be wrong) <braunr> ok + + +# IRC, freenode, #hurd, 2013-07-17 + + <teythoon> btw pinotree, what happened to your efforts to make dbus work? + <pinotree> not much, my initial patch was just a crude hack, a better + solution requires more thinkering and work + <teythoon> yes, ive seen that + <teythoon> but that was only a tiny patch against the libc, surely there + must be more to that? + <pinotree> not really + <teythoon> and the proper fix is to patch pflocal to query the auth server + and add the credentials? + <pinotree> possibly + <teythoon> that doesn't sound to bad, did you give it a try? + <pinotree> not really, got caught in other stuff + + +# IRC, freenode, #hurd, 2013-09-02 + + <gnu_srs1> something is wrong with libc0.3 since the switch to 2.17. dbus + does not run any longer when rebuilt + <gnu_srs1> the latest build of dbus was with 2.13: libc0.3-dev: already + installed (2.13-38) + <pinotree> debug it + <gnu_srs1> Yes, I will. Maybe somebody could rebuild it and verify my + findings? + <pinotree> gnu_srs1: your finding is "doesn't work", which is generic and + does not help without investigation + <gnu_srs1> just rebuild it and: e.g. ./build-debug/bus/dbus-daemon --system + (--nofork) + <pinotree> gnu_srs1: please, debug it + <gnu_srs1> I have partially already. But maybe the problems only shows on + my box. I'll rebuild on another box before continuing debugging. + <pinotree> gnu_srs1: are you, by chance, running a libc or something else + with your scm_creds work? + <gnu_srs1> I did, but I've backed to 2.17-92 right now. + <gnu_srs1> sane problem with dbus on another box, something's fishy:-( + <gnu_srs1> braunr: any good way to find out if the dbus problems are with + libpthread? Setting a breakpoint with libc0.3-dbg installed. + <braunr> gnu_srs1: i don't know + +See [[glibc]], *Missing interfaces, amongst many more*, *`SOCK_CLOEXEC`*. + + +# IRC, freenode, #hurd, 2013-09-04 + + <gnu_srs> Hi, looks like dbus requires abstract socket namespace: #undef + HAVE_ABSTRACT_SOCKETS What's missing? + <pinotree> uh? + <pinotree> abstract unix sockets are a Linux feature, and surely it is not + mandatory for dbus + <gnu_srs> Looks like dbus exits if they are not supported: + <gnu_srs> dbus_set_error (error, DBUS_ERROR_NOT_SUPPORTED, "Operating + system does not support abstract socket namespace\n"); _dbus_close + (listen_fd, NULL); 1061 return -1; + <pinotree> that is enclosed in a if (abstract) + <pinotree> and that parameter is set to true in other places (eg + dbus/dbus-server-unix.c) only when HAVE_ABSTRACT_SOCKETS is defined + <pinotree> so no, abstract sockets are not mandatory + <gnu_srs> Well this code is executed e.g. when running emacs remotely in + X. Have to dig deeper then to see why. + <pinotree> maybe it could have to do the fact that your dbus server is + running in linux and runs by default using such sockets type + <pinotree> but yes, you need to dig better + <gnu_srs> pinotree: You are right. when running natively the problem is: + <pinotree> *drums* + <gnu_srs> Manually: Process /usr/lib/at-spi2-core/at-spi-bus-launcher + exited with status 1 + <pinotree> eh? + <gnu_srs> Error retrieving accessibility bus address: + org.freedesktop.DBus.Error.Spawn.ChildExited: ^ + <pinotree> most probably that service does not start due to the lack of + socket credentials which affects dbus + <pinotree> uninstall or disable those additional services, they are not + your problem + <gnu_srs> credentials is enabled. which services to remove? + <pinotree> dunno + + +# IRC, freenode, #hurd, 2013-09-11 + + <gnu_srs> Hi, looks like frebsd had (2008) the same problem as hurd when + sending credentials over PF_INET: + <gnu_srs> + http://lists.freebsd.org/pipermail/freebsd-hackers/2008-May/024577.html + <gnu_srs> Since the dbus code is about the same now (2013), maybe they + added support? + <gnu_srs> The next message in the thread confirms that the dbus code is + invalid, does anybody have pointers? + <pinotree> from what i've seen so far, socket credentials are done only for + local sockets (ie PF_UNIX) + <pinotree> i don't see how things like uid/gid/pid of the socket endpoint + can have anything to do with AF_INET + <pinotree> and socket credentials in dbus are used only in the [local] + socket transport, so there's no issue + + +# IRC, freenode, #hurd, 2013-09-12 + + <gnu_srs> pinotree: Yes, there is an issue with dbus and AF_INET, see + test/corrupt.c: tests /corrupt/tcp and /corrupt/byte-order/tcp:-/ + <pinotree> gnu_srs: what's wrong with those? they are just testing the + connection over a tcp socket + <pinotree> as said above, socket credentials shouldn't be used in such + cases + <gnu_srs> They are, see also test/relay.c: /relay and /limit tests:-( + <pinotree> how are they? + <pinotree> please be more specifc... + <gnu_srs> Just run the tests yourself with DBUS_VERBOSE=1 + <pinotree> you are claiming there is a problem, so please specify what is + the actual issue + <gnu_srs> DBUS_VERBOSE=1 build-debug/test/test-relay + <pinotree> you are claiming there is a problem, so please specify what is + the actual issue + <gnu_srs> same with test-corrupt + <gnu_srs> look at the verbose output: Failed to write credentials: Failed + to write credentials byte: Invalid argument + <gnu_srs> coming from pfinet since PF_INET is used. + <pinotree> check what it does on linux then + <pinotree> put an abort() at the start of the read/write socket credential + functions in dbus-sysdeps-unix.c and see whether it is triggered also on + linux + <gnu_srs> SO_PEERCRED is used for linux and LOCAL_CREDS is used for + kfreebsd, so we are on our own here:-/ + <pinotree> and linux' SO_PEERCRED works also on AF_INET sockets? i'd doubt + it + <gnu_srs> + http://stackoverflow.com/questions/10037086/so-peercred-vs-scm-credentials-why-there-are-both-of-them + <pinotree> yes, i know the difference, but please read what i asked again + <gnu_srs> I'll check to be sure... + <braunr> gnu_srs: user credentials are not supposed to be passed through an + AF_INET socket + <braunr> how hard is that to understand ? + <gnu_srs> OK, linux use send since CMSGCREDS is not defined to write + credentials. Working on how they are received. + <gnu_srs> braunr: I do understand, but the dbus code tries to do that for + Hurd:-( + <pinotree> then it should do that on linux too + <pinotree> (since the local socket credentials code is isolated in own + functions, and they are called only for the unix transport) + <gnu_srs> Happiness:-D, almost all dbus tests pass! + <gnu_srs> 17(17) dbus tests pass:) + <braunr> gnu_srs: hopefully your patch does things right + <gnu_srs> which patch + <braunr> adding credentials through unix socket + <braunr> isn't that what you're doing ? + <gnu_srs> the mail to MLs is from the stock installed packages. + <braunr> ? + <gnu_srs> the test reports are with the SCM_CREDS patches, but I stumbled + on the SCM_RIGHTS issues reported to MLs + <gnu_srs> no patches applied, just test the attached file yourself. + <braunr> so what's your work about ? + <gnu_srs> I'm working on SCM_CREDS, yes, and created patches for dbus, + glib2.0 and libc. + <gnu_srs> the mail was about some bug in the call to io_restrict_auth in + sendmsg.c: without any of my patches applied (another image) + <teythoon> gnu_srs: you have to give us more context, how are we supposed + to know how to find this sendmsg.c file? + <pinotree> (it's in glibc, but otherwise the remark is valid) + <pinotree> s/otherwise/anyway/ diff --git a/open_issues/dde.mdwn b/open_issues/dde.mdwn index 65d84886..fe9fd8aa 100644 --- a/open_issues/dde.mdwn +++ b/open_issues/dde.mdwn @@ -512,6 +512,18 @@ After the microkernel devroom at [[community/meetings/FOSDEM_2013]]. <antrik> hm... good point +## IRC, freenode, #hurd, 2013-09-20 + + <braunr> i should take some time to integrate my pcap changes into the + libpcap debian package at least + <pinotree> braunr: if upstream is active, i'd say to go there directly + <braunr> the problem with that approach is that netdde is still not part of + our upstream code + <pinotree> don't understand the relation + <braunr> i don't want to send the pcap guys code for an interface that is + still not considered upstream ... + + # IRC, freenode, #hurd, 2012-08-14 <braunr> it's amazing how much code just gets reimplemented needlessly ... @@ -590,11 +602,4 @@ In context of [[libpthread]]. partitions/media... -# virtio - - -## IRC, freenode, #hurd, 2012-07-01 - - <braunr> hm, i haven't looked but, does someone know if virtio is included - in netdde ? - <youpi> braunr: nope, there's an underlying virtio layer needed before +# [[virtio]] diff --git a/open_issues/device_drivers_and_io_systems.mdwn b/open_issues/device_drivers_and_io_systems.mdwn index 5bda0213..085a737a 100644 --- a/open_issues/device_drivers_and_io_systems.mdwn +++ b/open_issues/device_drivers_and_io_systems.mdwn @@ -92,3 +92,9 @@ Also see [[user-space device drivers]]. * OSF Mach * Darwin + + * IRC, freenode, #hurd, 2013-08-26 + + < stargater> in haiku is a layer wraper for bsd driver + < stargater> + https://www.haiku-os.org/news/2007-05-08/haiku_getting_a_freebsd_network_driver_compatibility_layer diff --git a/open_issues/exec.mdwn b/open_issues/exec.mdwn index 36513453..05deaa7a 100644 --- a/open_issues/exec.mdwn +++ b/open_issues/exec.mdwn @@ -10,7 +10,10 @@ License|/fdl]]."]]"""]] [[!tag open_issue_hurd]] -IRC, unknown channel, unknown date. +[[!toc]] + + +# IRC, unknown channel, unknown date. <youpi> oh my, disabling gzip/bzip2 support makes apt preconfigure hang <youpi> support in exec* I meant @@ -30,6 +33,50 @@ Justus: This doesn't seem to be an issue anymore (2013-09-08): Also I've never encountered a problem with apt. + +## IRC, freenode, #hurd, 2013-08-01 + + <teythoon> uh, all the non trivial exec server code has #ifdef'd BFD code + all over it and it looks like that isn't even used anymore + <teythoon> that's too bad actually, I figured out how to get the values + from BFD, not so for the other elf parser that is used instead + + +## IRC, freenode, #hurd, 2013-08-05 + + <teythoon> btw, there is a Debian bug concerning zipped executables. now + I'm not sure if I understood the problem, but gziped and bzip2ed + executables work for me + <teythoon> (not that I'm a big fan of that particular feature) + <youpi> iirc these somehow got fixed yes + <youpi> something like a previous out of bound access + <teythoon> the exec server contains lot's of code that is unused and + probably bit rot (#ifdef BFD) or otherwise ignored (#if 0) + <youpi> yes :/ + <teythoon> and there's gunzipping and bunzip2ing, which we probably don't + want anyway + <pinotree> why not? + <teythoon> we should strip all that from exec and start adding features + <teythoon> pinotree: b/c it's slow and the gain is questionable + <teythoon> it breaks mmapping the code in + <teythoon> exec/exec.c is huge (~2300 lines) and complex and it is an + essential server + <teythoon> and I wonder if the unzipping is done securely, e. g. if it's + not possible to crash exec with an maliciously compressed executable + + +## IRC, freenode, #hurd, 2013-09-12 + + <rekado> The zip code in hurd/exec/ looks really complicated; does it + really just unpack zipped files in memory (which could be replaced by + library calls) or is there something else going on? + <braunr> rekado: + http://lists.gnu.org/archive/html/bug-hurd/2013-08/msg00049.html + <rekado> braunr: interesting. Thanks. + <rekado> Does this mean that the "small hack entry" on the contributing + page to use libz and libbz2 in exec is no longer valid? + <braunr> probably + --- May want to have a look at using BFD / libiberty/simpleobject. diff --git a/open_issues/exec_leak.mdwn b/open_issues/exec_leak.mdwn deleted file mode 100644 index b58d2c81..00000000 --- a/open_issues/exec_leak.mdwn +++ /dev/null @@ -1,57 +0,0 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -[[!tag open_issue_hurd]] - - -# IRC, freenode, #hurd, 2012-08-11 - - <braunr> the exec servers seems to leak a lot - <braunr> server* - <braunr> exec now uses 109M on darnassus - <braunr> it really leaks a lot - <pinotree> only 109mb? few months ago, exec on exodar was taking more than - 200mb after few days of uptime with builds done - <braunr> i wonder how much it takes on the buildds - - -# IRC, freenode, #hurd, 2012-08-17 - - <braunr> the exec leak is tricky - <braunr> bddebian: btw, look at the TODO file in the hurd source code - <braunr> bddebian: there is a not from thomas bushnell about that - <braunr> "*** Handle dead name notifications on execserver ports. ! - <braunr> not sure it's still a todo item, but it might be worth checking - <bddebian> braunr: diskfs_execboot_class = ports_create_class (0, 0); - This is what would need to change right? It should call some cleanup - routine in the first argument? - <bddebian> Would be ideal if it could just use deadboot() from exec. - <braunr> bddebian: possible - <braunr> bddebian: hum execboot, i'm not so sure - <bddebian> Execboot is the exec task, no? - <braunr> i don't know what execboot is - <bddebian> It's from libdiskfs - <braunr> but "diskfs_execboot_class" looks like a class of ports used at - startup only - <braunr> ah - <braunr> then it's something run in the diskfs users ? - <bddebian> yes - <braunr> the leak is in exec - <braunr> if clients misbehave, it shouldn't affect that server - <bddebian> That's a different issue, this was about the TODO thing - <braunr> ah - <braunr> i don't know - <bddebian> Me either :) - <bddebian> For the leak I'm still focusing on do-bunzip2 but I am baffled - at my results.. - <braunr> ? - <bddebian> Where my counters are zero if I always increment on different - vars but wild freaking numbers if I increment on malloc and decrement on - free diff --git a/open_issues/exec_memory_leaks.mdwn b/open_issues/exec_memory_leaks.mdwn index d504c4f0..67281bdc 100644 --- a/open_issues/exec_memory_leaks.mdwn +++ b/open_issues/exec_memory_leaks.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -12,8 +12,56 @@ License|/fdl]]."]]"""]] There are is some memory leak in [[`exec`|hurd/translator/exec]]. +[[!toc]] -# I + +# IRC, freenode, #hurd, 2012-08-11 + + <braunr> the exec servers seems to leak a lot + <braunr> server* + <braunr> exec now uses 109M on darnassus + <braunr> it really leaks a lot + <pinotree> only 109mb? few months ago, exec on exodar was taking more than + 200mb after few days of uptime with builds done + <braunr> i wonder how much it takes on the buildds + + +## IRC, freenode, #hurd, 2012-08-17 + + <braunr> the exec leak is tricky + <braunr> bddebian: btw, look at the TODO file in the hurd source code + <braunr> bddebian: there is a not from thomas bushnell about that + <braunr> "*** Handle dead name notifications on execserver ports. ! + <braunr> not sure it's still a todo item, but it might be worth checking + <bddebian> braunr: diskfs_execboot_class = ports_create_class (0, 0); + This is what would need to change right? It should call some cleanup + routine in the first argument? + <bddebian> Would be ideal if it could just use deadboot() from exec. + <braunr> bddebian: possible + <braunr> bddebian: hum execboot, i'm not so sure + <bddebian> Execboot is the exec task, no? + <braunr> i don't know what execboot is + <bddebian> It's from libdiskfs + <braunr> but "diskfs_execboot_class" looks like a class of ports used at + startup only + <braunr> ah + <braunr> then it's something run in the diskfs users ? + <bddebian> yes + <braunr> the leak is in exec + <braunr> if clients misbehave, it shouldn't affect that server + <bddebian> That's a different issue, this was about the TODO thing + <braunr> ah + <braunr> i don't know + <bddebian> Me either :) + <bddebian> For the leak I'm still focusing on do-bunzip2 but I am baffled + at my results.. + <braunr> ? + <bddebian> Where my counters are zero if I always increment on different + vars but wild freaking numbers if I increment on malloc and decrement on + free + + +# 2012-11-25 After twelve hours worth of `fork/exec` ([[GCC]]'s `check-c` part of the testsuite), we got: @@ -29,7 +77,7 @@ quite noticeable. In comparison: 276 0 3 1 1 344 442M 28.2M 0.6 48:09.36 91min /hurd/ext2fs /dev/hd2s5 -# II +# 2012-12-20 After running the libtool testsuite for some time: diff --git a/open_issues/fakeroot_eagain.mdwn b/open_issues/fakeroot_eagain.mdwn index 6b684a04..168ddf7d 100644 --- a/open_issues/fakeroot_eagain.mdwn +++ b/open_issues/fakeroot_eagain.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -132,7 +132,7 @@ License|/fdl]]."]]"""]] <braunr> or rather, a lot more <braunr> (or maybe not, since it leaks only in some cases) -[[exec_leak]]. +[[exec_memory_leaks]]. <braunr> pinotree: actually, the behaviour under linux is the same with the alternative correctly set, whereas faked-tcp is restarted (if used at diff --git a/open_issues/gcc.mdwn b/open_issues/gcc.mdwn index 3aa7b63b..2b772cfc 100644 --- a/open_issues/gcc.mdwn +++ b/open_issues/gcc.mdwn @@ -37,8 +37,8 @@ git diff --patience --stat=$COLUMNS,$COLUMNS --patch --src-prefix=./ --dst-prefi --> -Last reviewed up to the [[Git mirror's 0479dc77cf50ee78769b55563051cf72d39b3d60 -(2013-05-27) sources|source_repositories/gcc]]. +Last reviewed up to the [[Git mirror's 3a930d3fc68785662f5f3f4af02474cb21a62056 +(2013-06-06) sources|source_repositories/gcc]]. <http://gcc.gnu.org/install/configure.html> has documentation for the `configure` switches. @@ -48,20 +48,247 @@ Last reviewed up to the [[Git mirror's 0479dc77cf50ee78769b55563051cf72d39b3d60 * `configure.ac` - * `libgomp/configure.tgt` + * `libstdc++-v3` - * `libstdc++-v3/configure.host` + * `configure.host` - `abi_baseline_pair` etc. setting. + `abi_baseline_pair` etc. setting. `config/abi/post/*-linux-gnu`. + TODO. - * `libstdc++-v3/config/os/gnu-linux/*` + * `config/os/gnu-linux` - Is used for all GNU systems, as per `libstdc++-v3/configure.host`. - Should rename to `gnu-user` to reflect this? + Is used for all GNU systems, as per `configure.host`. Should + rename to `gnu-user` to reflect this? TODO. * `gcc/acinclude.m4`:`gcc_GAS_FLAGS`: always pass `--32` to assembler for x86 Linux. (Why?) + * `lib-prefix.m4` (present twice in GCC sources) contains one remaining + `linux`-only case. + + * `libjava` + + TODO: + + classpath/include/jni_md-x86-linux-gnu.h + + See below (`log_build`). + + Makefile.am:## _GNU_SOURCE defined for some Linux builds. It doesn't hurt to + Makefile.am:## always define it. Some systems, including Linux, need + Makefile.am:# certain linuxthread functions get linked: + Makefile.am:## This is specific to Linux/{Free,Net,Open}BSD/Hurd and perhaps few others. + Makefile.am: $(mkinstalldirs) $(DESTDIR)$(SDK_INCLUDE_DIR)/linux; \ + Makefile.am: $(DESTDIR)$(SDK_INCLUDE_DIR)/linux); \ + Makefile.am: $(DESTDIR)$(SDK_INCLUDE_DIR)/linux/$$headername.h; \ + classpath/NEWS: the epoll notification mechanism on Linux 2.6. + classpath/config.rpath: linux* | k*bsd*-gnu) + classpath/config.rpath: gnu* | linux* | k*bsd*-gnu) + classpath/config.rpath: linux*oldld* | linux*aout* | linux*coff*) + classpath/config.rpath: linux* | k*bsd*-gnu) + classpath/configure.ac: *linux*) + classpath/configure.ac: target_os=linux-gnu + classpath/configure.ac: AC_MSG_WARN(no, using x86-linux-gnu) + classpath/doc/cp-vmintegration.texinfo:has been primarily tested against Linux and lacks garbage collections, a + classpath/doc/cp-vmintegration.texinfo:Linux and Windows 2000. As of June, 2004, it does not appear that ORP + classpath/doc/cp-vmintegration.texinfo:This is a free Java Virtual Machine that is being developed on GNU/Linux + classpath/doc/cp-vmintegration.texinfo:Runs on the x86 and PowerPC architectures, on the AIX, Linux, and Mac + classpath/gnu/classpath/SystemProperties.java: && "Linux".equals(defaultProperties.get("os.name"))) + classpath/gnu/java/nio/EpollSelectorImpl.java: * notification mechanism on GNU/Linux. + classpath/java/io/File.java: * <strong>Implementation note</strong>: Unlike the RI, on Linux and UNIX + classpath/java/net/MimeTypeMapper.java: // On Linux this usually means /etc/mime.types. + classpath/ltcf-cxx.sh: linux*) + classpath/ltcf-cxx.sh: linux*) + classpath/ltconfig:# Transform linux* to *-*-linux-gnu*, to support old configure scripts. + classpath/ltconfig:linux-gnu*) ;; + classpath/ltconfig:linux*) host=`echo $host | sed 's/^\(.*-.*-linux\)\(.*\)$/\1-gnu\2/'` + classpath/ltconfig: version_type=linux + classpath/ltconfig: version_type=linux + classpath/ltconfig: version_type=linux + classpath/ltconfig: version_type=linux + classpath/ltconfig: version_type=linux + classpath/ltconfig: version_type=linux + classpath/ltconfig:# No shared lib support for Linux oldld, aout, or coff. + classpath/ltconfig:linux-gnuoldld* | linux-gnuaout* | linux-gnucoff*) + classpath/ltconfig:# This must be Linux ELF. + classpath/ltconfig:linux-gnu*) + classpath/ltconfig: version_type=linux + classpath/ltconfig: # powerpc, because MkLinux only supported shared libraries with the + classpath/ltconfig: # most powerpc-linux boxes support dynamic linking these days and + classpath/ltconfig: # assume the GNU/Linux dynamic linker is in use. + classpath/ltconfig: dynamic_linker='GNU/Linux ld.so' + classpath/ltconfig: version_type=linux + classpath/ltconfig: version_type=linux + classpath/ltconfig: version_type=linux + classpath/ltconfig: version_type=linux + classpath/ltconfig: dynamic_linker='GNU/Linux ld.so' + classpath/ltconfig: version_type=linux + classpath/ltconfig: version_type=linux + classpath/ltconfig: version_type=linux + classpath/ltmain.sh:# compiler flags: $LTCFLAGS + classpath/ltmain.sh: *-*-linux*) + classpath/ltmain.sh: darwin|linux|osf|windows|none) + classpath/ltmain.sh: # Like Linux, but with the current version available in + classpath/ltmain.sh: linux) + classpath/m4/lib-link.m4: dnl 2. if it's /usr/local/include and we are using GCC on Linux, + classpath/m4/lib-link.m4: linux* | gnu* | k*bsd*-gnu) haveit=yes;; + classpath/m4/lib-link.m4: dnl 2. if it's /usr/local/lib and we are using GCC on Linux, + classpath/m4/lib-link.m4: linux* | gnu* | k*bsd*-gnu) haveit=yes;; + classpath/m4/lib-prefix.m4: dnl 3. if it's /usr/local/include and we are using GCC on Linux, + classpath/m4/lib-prefix.m4: linux* | gnu* | k*bsd*-gnu) haveit=yes;; + classpath/m4/lib-prefix.m4: CPPFLAGS="${CPPFLAGS}${CPPFLAGS:+ }-I$additional_includedir" + classpath/m4/lib-prefix.m4: dnl 3. if it's /usr/local/lib and we are using GCC on Linux, + classpath/m4/lib-prefix.m4: linux*) haveit=yes;; + classpath/m4/lib-prefix.m4: LDFLAGS="${LDFLAGS}${LDFLAGS:+ }-L$additional_libdir" + classpath/m4/lib-prefix.m4: dnl On glibc systems, the current practice is that on a system supporting + classpath/native/jni/java-net/javanet.c: /* Not writable on Linux */ + classpath/native/jni/java-nio/gnu_java_nio_VMChannel.c: * vector based read call (currently readv on Linux). + classpath/native/jni/java-nio/gnu_java_nio_VMChannel.c: * vector based read call (currently readv on Linux). + classpath/vm/reference/java/lang/VMProcess.java: // Linux use a process-per-thread model, which means the same thread + + configure.ac: *-*-linux*) + configure.ac: AC_DEFINE(LINUX_THREADS, 1, [Define if using POSIX threads on Linux.]) + include/config.h.in:/* Define if using POSIX threads on Linux. */ + include/config.h.in:#undef LINUX_THREADS + include/posix-threads.h:# ifdef LOCK_DEBUG /* Assumes Linuxthreads */ + include/posix-threads.h:#ifndef LINUX_THREADS + include/posix-threads.h:// pthread_mutex_destroy does nothing on Linux and it is a win to avoid + include/posix-threads.h:#endif /* LINUX_THREADS */ + include/posix-threads.h: // For linux_threads this is really a pointer to its thread data + include/posix-threads.h:// E.g. on X86 Linux, pthread_self() is too slow for our purpose. + include/posix-threads.h:// This code should probably go away when Linux/X86 starts using a + posix-threads.cc:#if defined(LINUX_THREADS) || defined(FREEBSD_THREADS) + posix-threads.cc: // LinuxThreads (prior to glibc 2.1) usurps both SIGUSR1 and SIGUSR2. + posix-threads.cc:#else /* LINUX_THREADS */ + posix-threads.cc:#endif /* LINUX_THREADS */ + posix-threads.cc: // In older glibc's (prior to 2.1.3), the cond_wait functions may + posix-threads.cc: // glibc 2.1.3 doesn't set the value of `thread' until after start_routine + + configure.ac: # We can save a little space at runtime if the mutex has m_count + configure.ac: # or __m_count. This is a nice hack for Linux. + configure.ac: AC_COMPILE_IFELSE([AC_LANG_PROGRAM([[#include <pthread.h>]], [[ + configure.ac: extern pthread_mutex_t *mutex; int q = mutex->m_count; + + Makes sense to implement in our [[/libpthread]] ([[!taglink + open_issue_libpthread]])? + + configure.ac: i?86-*-linux*) + configure.ac: SIGNAL_HANDLER=include/i386-signal.h + configure.ac: SIGNAL_HANDLER_AUX=include/x86_64-signal.h + include/i386-signal.h:// on an i386 based Linux system. + include/i386-signal.h: directly rather than via glibc. The sigaction structure that the + include/i386-signal.h: * called _directly_ by the kernel, because linuxthreads wraps signal + include/i386-signal.h: * handler to a linuxthreads wrapper, we will lose the PC adjustment + include/i386-signal.h: * Also, there may not be any unwind info in the linuxthreads + + configure.ac: *-linux*) + configure.ac: host_os=linux;; + + configure.host: i[34567]86*-linux* | \ + configure.host: can_unwind_signal=yes + configure.host: libgcj_ld_symbolic='-Wl,-Bsymbolic' + configure.host: if test x$slow_pthread_self = xyes \ + configure.host: [...] + configure.host: i[34567]86*-kfreebsd*-gnu | x86_64*-kfreebsd*-gnu) + configure.host: libgcj_ld_symbolic='-Wl,-Bsymbolic' + configure.host: slow_pthread_self= + + java/lang/natObject.cc:// What follows currenly assumes a Linux-like platform. + java/lang/natObject.cc:// Some of it specifically assumes X86 or IA64 Linux, though that + java/lang/natObject.cc:# define INVALID_THREAD_ID 0 // Works for Linux? + java/lang/natObject.cc: const unsigned MIN_SLEEP_USECS = 2001; // Shorter times spin under Linux. + java/lang/natVMClassLoader.cc: // a module named (eg, on Linux) `lib-gnu-pkg-quux.so', followed + + libltdl/acinclude.m4:x86_64-*linux*|ppc*-*linux*|powerpc*-*linux*|s390*-*linux*|sparc*-*linux*) + libltdl/acinclude.m4: x86_64-*linux*) + libltdl/acinclude.m4: ppc64-*linux*|powerpc64-*linux*) + libltdl/acinclude.m4: LD="${LD-ld} -m elf32ppclinux" + libltdl/acinclude.m4: s390x-*linux*) + libltdl/acinclude.m4: sparc64-*linux*) + libltdl/acinclude.m4: x86_64-*linux*) + libltdl/acinclude.m4: ppc*-*linux*|powerpc*-*linux*) + libltdl/acinclude.m4: s390*-*linux*) + libltdl/acinclude.m4: sparc*-*linux*) + libltdl/acinclude.m4: # Under GNU Hurd, this test is not required because there is + libltdl/acinclude.m4: version_type=linux + libltdl/acinclude.m4: version_type=linux + libltdl/acinclude.m4: version_type=linux + libltdl/acinclude.m4: version_type=linux + libltdl/acinclude.m4: version_type=linux + libltdl/acinclude.m4: version_type=linux + libltdl/acinclude.m4: version_type=linux + libltdl/acinclude.m4:# No shared lib support for Linux oldld, aout, or coff. + libltdl/acinclude.m4:linux*oldld* | linux*aout* | linux*coff*) + libltdl/acinclude.m4:# This must be Linux ELF. + libltdl/acinclude.m4:linux*) + libltdl/acinclude.m4: version_type=linux + libltdl/acinclude.m4: # powerpc, because MkLinux only supported shared libraries with the + libltdl/acinclude.m4: # most powerpc-linux boxes support dynamic linking these days and + libltdl/acinclude.m4: # assume the GNU/Linux dynamic linker is in use. + libltdl/acinclude.m4: dynamic_linker='GNU/Linux ld.so' + libltdl/acinclude.m4: version_type=linux + libltdl/acinclude.m4: version_type=linux + libltdl/acinclude.m4: version_type=linux + libltdl/acinclude.m4: version_type=linux + libltdl/acinclude.m4: version_type=linux + libltdl/acinclude.m4: version_type=linux + libltdl/acinclude.m4: version_type=linux + libltdl/acinclude.m4:# This must be Linux ELF. + libltdl/acinclude.m4:linux*) + libltdl/acinclude.m4: linux*) + libltdl/acinclude.m4:linux*) + libltdl/acinclude.m4: linux*) + libltdl/acinclude.m4: # Linux and Compaq Tru64 Unix objects are PIC. + libltdl/acinclude.m4: # Linux and Compaq Tru64 Unix objects are PIC. + libltdl/acinclude.m4: linux*) + libltdl/acinclude.m4: linux*) + libltdl/acinclude.m4: gnu* | linux* | kfreebsd*-gnu | knetbsd*-gnu) + libltdl/acinclude.m4: # GNU and its variants, using gnu ld.so (Glibc) + libltdl/ltmain.sh: darwin|linux|osf|windows) + libltdl/ltmain.sh: # Like Linux, but with the current version available in + libltdl/ltmain.sh: linux) + shlibpath.m4: version_type=linux + shlibpath.m4: version_type=linux + shlibpath.m4: version_type=linux + shlibpath.m4: version_type=linux + shlibpath.m4: version_type=linux + shlibpath.m4: version_type=linux + shlibpath.m4:# No shared lib support for Linux oldld, aout, or coff. + shlibpath.m4:linux*oldld* | linux*aout* | linux*coff*) + shlibpath.m4:# This must be Linux ELF. + shlibpath.m4:linux*|k*bsd*-gnu) + shlibpath.m4: version_type=linux + shlibpath.m4: # powerpc, because MkLinux only supported shared libraries with the + shlibpath.m4: # most powerpc-linux boxes support dynamic linking these days and + shlibpath.m4: # assume the GNU/Linux dynamic linker is in use. + shlibpath.m4: dynamic_linker='GNU/Linux ld.so' + shlibpath.m4: version_type=linux + shlibpath.m4: version_type=linux + shlibpath.m4: version_type=linux + shlibpath.m4: version_type=linux + shlibpath.m4: version_type=linux + shlibpath.m4: version_type=linux + + testsuite/lib/libjava.exp: if { [regexp "linux" $target_triplet] } { + + Adds `-specs=libgcj-test.spec`, which is created by `configure`. *This + spec file is read by gcj when linking. It is only used by the testing + harnesses (in libjava and gdb).* TODO. [[!taglink open_issue_gdb]]. + + * `libgcc` + + TODO: + + * `config/t-linux` + * `config/i386/t-linux` + * `config/i386/linux-unwind.h` + + * `libitm` + + TODO: + + * `libitm/config/linux` + * `hurd/usr` `NATIVE_SYSTEM_HEADER_DIR`, `638454a19c1c08f01c10517bc72a114250fc4f33`, @@ -87,17 +314,20 @@ Last reviewed up to the [[Git mirror's 0479dc77cf50ee78769b55563051cf72d39b3d60 * `gcc/config/gnu-user.h` defines `*SPLIT_STACK*` macros -- which aren't valid for us (yet), I think. - * Also see [[sourceware_PR 10686]], glibc commit + * Also see [[!sourceware_PR 10686]], glibc commit ecbf434213c0333d81706074e4d107ac45011635 `Reserve new TLS field for x86 and x86_64` (`__private_ss`). * Might `-fsplit-stack` be useful for us with respect to our [[multithreaded|multithreading]] libraries? - * `--enable-languages=[...]` + * `gcc/ada`, `gcc/testsuite/ada`, `gcc/testsuite/gnat.dg`, `gnattools`, + `libada` (not reviewed) * [[Ada (GNAT)|GNAT]] support is work in progress. + * `gcc/go`, `gcc/testsuite/go.test`, `libgo` (not reviewed) + * The [[Google Go's libgo|gccgo]] (introduced in e440a3286bc89368b8d3a8fd6accd47191790bf2 (2010-12-03)) needs OS configuration / support. @@ -153,8 +383,6 @@ Last reviewed up to the [[Git mirror's 0479dc77cf50ee78769b55563051cf72d39b3d60 buildable out of the box)? See also 73905b5de0d9a086f22ded7638bb1c0ae1b91326. - * Various testsuite bits should include `*-*-gnu*`, too. - * [low] [[toolchain/cross-gnu]] toolchain bootstrap vs. `fenv.h` in libgcc's libbid: @@ -303,40 +531,18 @@ Last reviewed up to the [[Git mirror's 0479dc77cf50ee78769b55563051cf72d39b3d60 "20110609002620.GA16719@const.famille.thibault.fr"]]. commit 026e608ecebcb2a6193971006a85276307d79b00. - * 549e2197b118efb2d947aaa15d445b05c1b5ed62 `Import the asan runtime library - into GCC tree`. Linux-specific things: - `ASAN_USE_ALIAS_ATTRIBUTE_FOR_INDEX`, `ASAN_LINUX`, `ASAN_POSIX`, - `libsanitizer/asan/asan_linux.cc`, - `libsanitizer/asan/asan_malloc_linux.cc`, - `libsanitizer/asan/asan_posix.cc`, - `libsanitizer/interception/interception.h`, - `libsanitizer/interception/interception_linux.cc`, - `libsanitizer/interception/interception_linux.h`, - `libsanitizer/sanitizer_common/sanitizer_allocator.cc`, - `libsanitizer/sanitizer_common/sanitizer_linux.cc`, - `libsanitizer/sanitizer_common/sanitizer_posix.cc`, - `libsanitizer/sanitizer_common/sanitizer_procmaps.h`, - `libsanitizer/sanitizer_common/sanitizer_symbolizer_linux.cc`. - 4afab99bf0fe2d6905a9fa9d6ab886ca102312df `Enable libsanitizer just on x86 - linux for now`. 492e75a7336b4dbfe38207ea3abf8d5bd72376a9 `Move - libsanitizer configure logic to subdirectory`. - 6aea389d84c2172668af5f108e2b17e131120d0b `Add STATIC_LIBASAN_LIBS for - -static-libasan`. Further commits later on. - - * 9cf754572854d9d9cd43c277eb7afb12e4911358 `Import tsan runtime from - llvm`. Linux-specific things: `libsanitizer/tsan/tsan_platform.h`, - `libsanitizer/tsan/tsan_platform_linux.cc`, - `libsanitizer/tsan/tsan_symbolize_addr2line_linux.cc`. - a96132f29aa3dfe94141a87537f62ea73ce0fc19 `Set TSAN_SUPPORTED=yes for - x86_64/i686-linux for 64-bit multilib`. Further commits later on. + * [[`libsanitizer`|_san]] (not reviewed) + + A lot of Linux-specific things. + + # Build Here's a log of a GCC build run; this is from our [[Git repository's -dcdba5abca23716daa6aeb5c92f367e0978e4539 (2013-05-27; -0479dc77cf50ee78769b55563051cf72d39b3d60 (2013-05-27)), plus -`id:"87txlnlg0z.fsf@kepler.schwinge.homeip.net"` +2a3496bebfe9d89f11d0b7a591afac55e11d5263 (2013-06-06; +3a930d3fc68785662f5f3f4af02474cb21a62056 (2013-06-06)) sources|source_repositories/gcc]], run on kepler.SCHWINGE and coulomb.SCHWINGE. $ export LC_ALL=C @@ -349,10 +555,11 @@ sources|source_repositories/gcc]], run on kepler.SCHWINGE and coulomb.SCHWINGE. Different hosts may default to different shells and compiler versions; thus harmonized. -We're stuck with GCC 4.6 until there are Debian *gnat-4.7* packages avaible. +We're stuck with GCC 4.6 until there are Debian *gnat-4.7*/*gnat-4.8* packages +avaible. -This takes up around 3.5 GiB, and needs roughly 3.5 h on kepler.SCHWINGE and 15 -h on coulomb.SCHWINGE. +This takes up around 3.5 GiB, and needs roughly 3.5 h on kepler.SCHWINGE and +15.25 h on coulomb.SCHWINGE. <!-- @@ -370,7 +577,9 @@ h on coulomb.SCHWINGE. Addressed in Debian glibc. - * `host-linux.c` vs. `host-default.c` + * `gcc/config/host-linux.c` vs. `host-default.c` + + * `gcc/config/x-linux` * *fixincludes* stuff @@ -382,15 +591,16 @@ h on coulomb.SCHWINGE. Comes from `gcc/config.gcc`: `i386/t-pmm_malloc` vs. `i386/t-gmm_malloc` for `i[34567]86-*-linux*` vs. `i[34567]86-*-*`. - * *libgomp* + * `libgomp` - * `libgomp/config/linux/`, `libgomp/config/linux/x86` + * `libgomp/config/linux`, `libgomp/config/linux/x86` - `sed`ed away. + `sed`ed away in `log_build*`. TODO. - * `-ftls-model=initial-exec -march=i486 -mtune=i686` + * `-march=i486 -mtune=i686` - `sed`ed away. + `sed`ed away in `log_build*`. This comes from `libgomp/configure.tgt`, + where this is added to `XCFLAGS` for `i[456]86-*-linux*` only. TODO? * Missing `EOWNERDEAD`, `ENOTRECOVERABLE`. What're they used for? @@ -592,10 +802,10 @@ coulomb.SCHWINGE: $ make -k check-target 2>&1 | tee log_test_4_check-target [...] -This needs roughly 7.25 h on kepler.SCHWINGE and 3.5 h (`check-fixincludes`, -`gcc/check-ada`) + 14 h (`gcc/check-c`) + 4.5 h (`gcc/check-c++`) + 6 h +This needs roughly 7.5 h on kepler.SCHWINGE and 3.75 h (`check-fixincludes`, +`gcc/check-ada`) + 14 h (`gcc/check-c`) + 4.5 h (`gcc/check-c++`) + 7.25 h (`gcc/check-fortran`, `gcc/check-java`, `gcc/check-lto`, `gcc/check-objc`) + -9.75 h (`check-intl`, [...], `check-lto-plugin`, `check-target`) = 37.75 h on +10.25 h (`check-intl`, [...], `check-lto-plugin`, `check-target`) = 39.75 h on coulomb.SCHWINGE. @@ -611,34 +821,380 @@ coulomb.SCHWINGE. TODO. - * As of b401cb7ed15602d244a6807835b0b9d740a302a8 (2012-11-26; - 769bf18a20ee2540ca7601cdafabd62b18b9751b (2012-10-01)), all - `gcc.dg/guality` and `g++.dg/guality` and a few more are no longer tested - on coulomb.SCHWINGE and kepler.SCHWINGE. + * Some are correctly UNSUPPORTED: + + * [[IFUNC]] + + Also multiversioning, `g++.dg/ext/mv*`, for example (several of which + started FAILing (ICE) on kepler.SCHWINGE). + + * SSE2 (`sse2_runtime`) + + `g++.dg/other/i386-1.C`, `g++.dg/other/pr40446.C`, + `g++.dg/other/pr49133.C`, `gcc.dg/compat/union-m128-1_main.c`, + `gcc.dg/compat/vector-1a_main.c`, `gcc.dg/compat/vector-2a_main.c`, + `gcc.dg/pr36584.c`, `gcc.dg/pr37544.c`, `gcc.dg/torture/pr16104-1.c`, + `gcc.dg/torture/pr35771-1.c`, `gcc.dg/torture/pr50444.c`, + `gcc.dg/torture/stackalign/alloca-2.c`, + `gcc.dg/torture/stackalign/alloca-3.c`, + `gcc.dg/torture/stackalign/push-1.c`, + `gcc.dg/torture/stackalign/vararg-3.c`, `gcc.target/i386/pr39315-2.c`, + `gcc.target/i386/pr39315-4.c`, `gcc.target/i386/pr44948-2a.c`, + `gcc.target/i386/pr46880.c`, `gcc.target/i386/pr52736.c`, + `gcc.target/i386/pr54703.c`, `gcc.target/i386/sse2-extract-1.c`, + several from `gfortran.fortran-torture` + + * [[`asan.exp`|_san]] + + * missing profiling C library (`-lc_p`) + + `g++.old-deja/g++.law/profile1.C`, `gcc.dg/20021014-1.c`, + `gcc.dg/nest.c`, `gcc.dg/nested-func-4.c`, `gcc.dg/pr32450.c`, + `gcc.dg/pr43643.c` + + * other C libraries + + `gcc.target/i386/long-double-64-2.c`, + `gcc.target/i386/long-double-80-3.c` + + * `gcc` + + spawn [open ...] + FAIL: gcc.dg/split-2.c execution test + + FAIL: gcc.dg/split-5.c execution test + + TODO. + + xgcc: internal compiler error: Aborted (program cc1) + libbacktrace could not find executable to open + Please submit a full bug report, [...] + FAIL: largefile.c -O0 -g -I. -Dwith_PCH (internal compiler error) + [...] + + TODO. + + * `g++` + + spawn [open ...] + terminate called after throwing an instance of 'int' + FAIL: g++.dg/eh/sighandle.C -std=gnu++98 execution test + + FAIL: g++.dg/eh/sighandle.C -std=gnu++11 execution test + + TODO. + + spawn [open ...] + FAIL: g++.dg/cdce3.C -std=gnu++98 execution test + + FAIL: g++.dg/cdce3.C -std=gnu++11 execution test + + TODO. + + FAIL: g++.dg/tls/thread_local3.C -std=gnu++11 execution test + FAIL: g++.dg/tls/thread_local3g.C -std=gnu++11 execution test + FAIL: g++.dg/tls/thread_local4.C -std=gnu++11 execution test + FAIL: g++.dg/tls/thread_local4g.C -std=gnu++11 execution test + FAIL: g++.dg/tls/thread_local5.C -std=gnu++11 execution test + FAIL: g++.dg/tls/thread_local5g.C -std=gnu++11 execution test + + They used to PASS, but FAIL as of + 769bf18a20ee2540ca7601cdafabd62b18b9751b..be3860ba8df48cca3253da4f02fd2d42d856ce80. + TODO. + + -PASS: g++.dg/vect/pr36648.cc -std=c++98 execution test + -PASS: g++.dg/vect/pr36648.cc -std=c++11 execution test + + On kepler.SCHWINGE, executables are generated (and run), on + coulomb.SCHWINGE only assembler code is generated. TODO. Likewise for + execution tests from `gcc.dg/vect` and `gfortran.dg/vect`. + + * `gcc`, `g++` + + FAIL: gcc.dg/cleanup-10.c execution test + FAIL: gcc.dg/cleanup-11.c execution test + FAIL: gcc.dg/cleanup-8.c execution test + FAIL: gcc.dg/cleanup-9.c execution test + FAIL: g++.dg/ext/cleanup-10.C -std=gnu++98 execution test + FAIL: g++.dg/ext/cleanup-10.C -std=gnu++11 execution test + FAIL: g++.dg/ext/cleanup-11.C -std=gnu++98 execution test + FAIL: g++.dg/ext/cleanup-11.C -std=gnu++11 execution test + FAIL: g++.dg/ext/cleanup-8.C -std=gnu++98 execution test + FAIL: g++.dg/ext/cleanup-8.C -std=gnu++11 execution test + FAIL: g++.dg/ext/cleanup-9.C -std=gnu++98 execution test + FAIL: g++.dg/ext/cleanup-9.C -std=gnu++11 execution test + + TODO. + + spawn [open ...] + gdb: took too long to attach + testcase [...]/gcc/testsuite/gcc.dg/guality/guality.exp completed in 16 seconds + + spawn [open ...] + gdb: took too long to attach + testcase [...]/gcc/testsuite/g++.dg/guality/guality.exp completed in 20 seconds - * As of b401cb7ed15602d244a6807835b0b9d740a302a8 (2012-11-26; - 769bf18a20ee2540ca7601cdafabd62b18b9751b (2012-10-01)), there are - regressions (FAILs) in libgomp execution tests on coulomb.SCHWINGE. + TODO. The gfortran ones worked fine. - * 769bf18a20ee2540ca7601cdafabd62b18b9751b..be3860ba8df48cca3253da4f02fd2d42d856ce80 + * `[ARCH]/libgomp` - On GNU/Hurd: + As of dcdba5abca23716daa6aeb5c92f367e0978e4539 (2013-05-27; + 0479dc77cf50ee78769b55563051cf72d39b3d60 (2013-05-27)), plus + `id:"87txlnlg0z.fsf@kepler.schwinge.homeip.net"`, about a dozen of them + (but different ones per each run) FAIL on coulomb.SCHWINGE: - Running [...]/hurd/master/gcc/testsuite/g++.dg/tls/tls.exp ... - +FAIL: g++.dg/tls/thread_local3.C -std=gnu++11 execution test - +FAIL: g++.dg/tls/thread_local3g.C -std=gnu++11 execution test - +FAIL: g++.dg/tls/thread_local4.C -std=gnu++11 execution test - +FAIL: g++.dg/tls/thread_local4g.C -std=gnu++11 execution test - +FAIL: g++.dg/tls/thread_local5.C -std=gnu++11 execution test - +FAIL: g++.dg/tls/thread_local5g.C -std=gnu++11 execution test + spawn [open ...] + + Program aborted. Backtrace: + #0 0x1042523 + #1 0x1043D6F + #2 0x10F9BC7 + FAIL: libgomp.fortran/lib1.f90 -O1 execution test - They used to PASS. + All have basically the same backtrace. TODO. + + * `[ARCH]/libjava` + + spawn [open ...] + Exception in thread "main" java.io.IOException: Invalid argument + at gnu.java.nio.channels.FileChannelImpl.write(natFileChannelImpl.cc:202) + at java.io.FileOutputStream.write(libgcj.so.14) + at java.io.DataOutputStream.write(libgcj.so.14) + at java.io.RandomAccessFile.write(libgcj.so.14) + at LargeFile.main(LargeFile.exe) + FAIL: LargeFile execution - source compiled test + UNTESTED: LargeFile output - source compiled test + + FAIL: LargeFile -findirect-dispatch execution - source compiled test + UNTESTED: LargeFile -findirect-dispatch output - source compiled test + FAIL: LargeFile -O3 execution - source compiled test + UNTESTED: LargeFile -O3 output - source compiled test + FAIL: LargeFile -O3 -findirect-dispatch execution - source compiled test + UNTESTED: LargeFile -O3 -findirect-dispatch output - source compiled test + + TODO. + + spawn [open ...] + 1 + FAIL: Throw_2 execution - source compiled test + UNTESTED: Throw_2 output - source compiled test + + FAIL: Throw_2 -findirect-dispatch execution - source compiled test + UNTESTED: Throw_2 -findirect-dispatch output - source compiled test + FAIL: Throw_2 -O3 execution - source compiled test + UNTESTED: Throw_2 -O3 output - source compiled test + FAIL: Throw_2 -O3 -findirect-dispatch execution - source compiled test + UNTESTED: Throw_2 -O3 -findirect-dispatch output - source compiled test + + TODO. + + * `[ARCH]/libmudflap` + + spawn [open ...] + FAIL: libmudflap.cth/pass37-frag.c (-O0) execution test + FAIL: libmudflap.cth/pass37-frag.c (-O0) output pattern test + + FAIL: libmudflap.cth/pass37-frag.c (-O0) (rerun 1) execution test + FAIL: libmudflap.cth/pass37-frag.c (-O0) (rerun 1) output pattern test + [...] + + TODO. Seems like not just timeouts (though, reported before: [[!GCC_PR + 20003]]). If GDB is to believed, it seems like confusion between + libmudflap and glibc startup (while setting up the signal thread?): + + #0 getenv (name=0x12dabee "LANGUAGE") at getenv.c:81 + #1 0x011b2c78 in guess_category_value (categoryname=<optimized out>, category=<optimized out>) at dcigettext.c:1359 + #2 __dcigettext (domainname=0x12dab1b <_libc_intl_domainname> "libc", msgid1=0x12e1cd8 "Error in unknown error system: ", msgid2=0x0, plural=0, n=0, category=5) at dcigettext.c:575 + #3 0x011b1c53 in __dcgettext (domainname=0x12dab1b <_libc_intl_domainname> "libc", msgid=0x12e1cd8 "Error in unknown error system: ", category=5) at dcgettext.c:53 + #4 0x01203728 in __strerror_r (errnum=-1, buf=0x15ff648 "", buflen=1024) at ../sysdeps/mach/_strerror.c:57 + #5 0x011b0f30 in __assert_perror_fail (errnum=-1, file=0x1133969 "./pthread/cthreads-compat.c", line=45, function=0x1133985 <__PRETTY_FUNCTION__.5356> "cthread_fork") at assert-perr.c:62 + #6 0x011324d4 in cthread_fork (func=0x118b0b0 <_hurd_msgport_receive>, arg=0x0) at ./pthread/cthreads-compat.c:45 + #7 0x01192a96 in _hurdsig_init (intarray=0x102a000, intarraysize=5) at hurdsig.c:1499 + #8 0x0117b9f8 in _hurd_new_proc_init (argv=0x15ffb88, intarray=0x102a000, intarraysize=5) at hurdinit.c:138 + #9 0x0117bfef in _hurd_init (flags=8, argv=0x15ffb88, portarray=0x1029000, portarraysize=6, intarray=0x102a000, intarraysize=5) at hurdinit.c:94 + #10 0x011a47c4 in init1 (argc=1, arg0=0x1025000 "/media/erich/home/thomas/tmp/gcc/hurd/master.build/i686-unknown-gnu0.3/libmudflap/testsuite/pass37-frag.exe") at ../sysdeps/mach/hurd/i386/init-first.c:136 + #11 0x00001ec6 in _dl_start_user () from /lib/ld.so + + pthread/cthreads-compat.c: + + 38 cthread_t + 39 cthread_fork (cthread_fn_t func, void *arg) + 40 { + 41 pthread_t thread; + 42 int err; + 43 + 44 err = pthread_create (&thread, NULL, func, arg); + 45 assert_perror (err); + + Breakpoint 2, cthread_fork (func=0x118b0b0 <_hurd_msgport_receive>, arg=0x0) at ./pthread/cthreads-compat.c:44 + 44 err = pthread_create (&thread, NULL, func, arg); + (gdb) info threads + Id Target Id Frame + * 4 Thread 17597.16 cthread_fork (func=0x118b0b0 <_hurd_msgport_receive>, arg=0x0) at ./pthread/cthreads-compat.c:44 + (gdb) s + 40 { + (gdb) + 44 err = pthread_create (&thread, NULL, func, arg); + (gdb) + + Breakpoint 1, pthread_create (thr=0x15ffa70, attr=0x0, start=0x118b0b0 <_hurd_msgport_receive>, arg=0x0) at ../../../master/libmudflap/mf-hooks3.c:272 + 272 { + (gdb) s + 275 TRACE ("pthread_create\n"); + (gdb) + 278 si = CALL_REAL (malloc, sizeof (*si)); + (gdb) n + 279 si->user_fn = start; + (gdb) + 283 return CALL_REAL (pthread_create, thr, attr, __mf_pthread_spawner, si); + (gdb) s + 279 si->user_fn = start; + (gdb) + 280 si->user_arg = arg; + (gdb) + 283 return CALL_REAL (pthread_create, thr, attr, __mf_pthread_spawner, si); + (gdb) + 280 si->user_arg = arg; + (gdb) + 283 return CALL_REAL (pthread_create, thr, attr, __mf_pthread_spawner, si); + (gdb) + __mf_0fn_pthread_create (thr=thr@entry=0x15ffa70, attr=attr@entry=0x0, start=start@entry=0x1041070 <__mf_pthread_spawner>, arg=arg@entry=0x108e520 <__mf_0fn_bufs+12288>) at ../../../master/libmudflap/mf-hooks3.c:265 + 265 } + (gdb) s + pthread_create (thr=0x15ffa70, attr=0x0, start=0x118b0b0 <_hurd_msgport_receive>, arg=0x0) at ../../../master/libmudflap/mf-hooks3.c:284 + 284 } + (gdb) s + cthread_fork (func=0x118b0b0 <_hurd_msgport_receive>, arg=0x0) at ./pthread/cthreads-compat.c:45 + 45 assert_perror (err); + (gdb) s + __assert_perror_fail (errnum=-1, file=0x1133969 "./pthread/cthreads-compat.c", line=45, function=0x1133985 <__PRETTY_FUNCTION__.5356> "cthread_fork") at assert-perr.c:55 + + Is this `libmudflap/mf-hooks3.c:__mf_0fn_pthread_create`, *a special + bootstrap variant*, that indeed just returns `-1`? + + * `[ARCH]/libstdc++-v3` + + FAIL: libstdc++-abi/abi_check + + TODO. + + $ readelf --symbols --wide i686-unknown-gnu0.3/./libstdc++-v3/src/.libs/libstdc++.so | grep pthread_mutex + 1065: 00000000 0 FUNC WEAK DEFAULT UND pthread_mutex_unlock@GLIBC_2.13_DEBIAN_31 (37) + 2515: 00000000 0 FUNC WEAK DEFAULT UND pthread_mutex_lock@GLIBC_2.13_DEBIAN_31 (37) + 2978: 00068430 15 FUNC GLOBAL DEFAULT 11 _ZNSt12__basic_fileIcEC2EP15__pthread_mutex@@GLIBCXX_3.4 + 3790: 00068430 15 FUNC GLOBAL DEFAULT 11 _ZNSt12__basic_fileIcEC1EP15__pthread_mutex@@GLIBCXX_3.4 + 2085: 00000000 0 FUNC WEAK DEFAULT UND pthread_mutex_unlock@@GLIBC_2.13_DEBIAN_31 + 3535: 00000000 0 FUNC WEAK DEFAULT UND pthread_mutex_lock@@GLIBC_2.13_DEBIAN_31 + 3998: 00068430 15 FUNC GLOBAL DEFAULT 11 _ZNSt12__basic_fileIcEC2EP15__pthread_mutex + 4810: 00068430 15 FUNC GLOBAL DEFAULT 11 _ZNSt12__basic_fileIcEC1EP15__pthread_mutex + + `_ZNSt12__basic_fileIcEC1EP15__pthread_mutex` + (`std::__basic_file<char>::__basic_file(__pthread_mutex*)`), but + `_ZNSt12__basic_fileIcEC2EP15pthread_mutex_t` + (`std::__basic_file<char>::__basic_file(pthread_mutex_t*)`) is expected. + + FAIL: 22_locale/time_get/get_date/wchar_t/4.cc execution test + FAIL: 27_io/basic_filebuf/close/char/4879.cc execution test + FAIL: 27_io/basic_filebuf/close/char/9964.cc execution test + FAIL: 27_io/basic_filebuf/imbue/char/13171-2.cc execution test + FAIL: 27_io/basic_filebuf/imbue/wchar_t/14975-2.cc execution test + WARNING: program timed out. + FAIL: 27_io/basic_filebuf/open/char/9507.cc execution test + FAIL: 27_io/basic_filebuf/seekoff/char/26777.cc execution test + WARNING: program timed out. + FAIL: 27_io/basic_filebuf/showmanyc/char/9533-1.cc execution test + FAIL: 27_io/basic_filebuf/underflow/char/10097.cc execution test + FAIL: 27_io/objects/char/7.cc execution test + FAIL: 27_io/objects/char/9661-1.cc execution test + FAIL: 27_io/objects/wchar_t/7.cc execution test + FAIL: 27_io/objects/wchar_t/9661-1.cc execution test + FAIL: 30_threads/async/42819.cc execution test + FAIL: 30_threads/async/49668.cc execution test + FAIL: 30_threads/async/54297.cc execution test + FAIL: 30_threads/async/any.cc execution test + FAIL: 30_threads/async/async.cc execution test + FAIL: 30_threads/async/sync.cc execution test + FAIL: 30_threads/call_once/39909.cc execution test + FAIL: 30_threads/call_once/49668.cc execution test + FAIL: 30_threads/call_once/call_once1.cc execution test + FAIL: 30_threads/condition_variable/54185.cc execution test + FAIL: 30_threads/condition_variable_any/50862.cc execution test + FAIL: 30_threads/condition_variable_any/53830.cc execution test + FAIL: 30_threads/future/members/45133.cc execution test + FAIL: 30_threads/future/members/get.cc execution test + FAIL: 30_threads/future/members/get2.cc execution test + FAIL: 30_threads/future/members/share.cc execution test + FAIL: 30_threads/future/members/valid.cc execution test + FAIL: 30_threads/future/members/wait.cc execution test + FAIL: 30_threads/future/members/wait_for.cc execution test + FAIL: 30_threads/future/members/wait_until.cc execution test + FAIL: 30_threads/lock/2.cc execution test + FAIL: 30_threads/lock/4.cc execution test + FAIL: 30_threads/mutex/try_lock/2.cc execution test + FAIL: 30_threads/packaged_task/49668.cc execution test + FAIL: 30_threads/packaged_task/cons/3.cc execution test + FAIL: 30_threads/packaged_task/cons/alloc.cc execution test + FAIL: 30_threads/packaged_task/members/get_future.cc execution test + FAIL: 30_threads/packaged_task/members/invoke.cc execution test + FAIL: 30_threads/packaged_task/members/invoke2.cc execution test + FAIL: 30_threads/packaged_task/members/invoke3.cc execution test + FAIL: 30_threads/packaged_task/members/invoke4.cc execution test + FAIL: 30_threads/packaged_task/members/invoke5.cc execution test + FAIL: 30_threads/packaged_task/members/reset2.cc execution test + FAIL: 30_threads/promise/cons/alloc.cc execution test + FAIL: 30_threads/promise/cons/move.cc execution test + FAIL: 30_threads/promise/cons/move_assign.cc execution test + FAIL: 30_threads/promise/members/get_future.cc execution test + FAIL: 30_threads/promise/members/set_exception.cc execution test + FAIL: 30_threads/promise/members/set_exception2.cc execution test + FAIL: 30_threads/promise/members/set_value.cc execution test + FAIL: 30_threads/promise/members/set_value2.cc execution test + FAIL: 30_threads/promise/members/set_value3.cc execution test + FAIL: 30_threads/promise/members/swap.cc execution test + FAIL: 30_threads/shared_future/members/get.cc execution test + FAIL: 30_threads/shared_future/members/get2.cc execution test + FAIL: 30_threads/shared_future/members/valid.cc execution test + FAIL: 30_threads/shared_future/members/wait.cc execution test + FAIL: 30_threads/shared_future/members/wait_for.cc execution test + FAIL: 30_threads/shared_future/members/wait_until.cc execution test + FAIL: 30_threads/this_thread/3.cc execution test + FAIL: 30_threads/this_thread/4.cc execution test + FAIL: 30_threads/thread/cons/2.cc execution test + FAIL: 30_threads/thread/cons/3.cc execution test + FAIL: 30_threads/thread/cons/4.cc execution test + FAIL: 30_threads/thread/cons/49668.cc execution test + FAIL: 30_threads/thread/cons/5.cc execution test + FAIL: 30_threads/thread/cons/6.cc execution test + FAIL: 30_threads/thread/cons/7.cc execution test + FAIL: 30_threads/thread/cons/8.cc execution test + FAIL: 30_threads/thread/cons/9.cc execution test + FAIL: 30_threads/thread/cons/moveable.cc execution test + FAIL: 30_threads/thread/members/1.cc execution test + FAIL: 30_threads/thread/members/2.cc execution test + FAIL: 30_threads/thread/members/3.cc execution test + FAIL: 30_threads/thread/native_handle/cancel.cc execution test + FAIL: 30_threads/thread/swap/1.cc execution test + FAIL: 30_threads/timed_mutex/try_lock/2.cc execution test + FAIL: 30_threads/timed_mutex/try_lock_for/3.cc execution test + FAIL: 30_threads/timed_mutex/try_lock_until/2.cc execution test + FAIL: 30_threads/try_lock/2.cc execution test + FAIL: 30_threads/try_lock/4.cc execution test + + TODO. Perhaps just timeouts? [[!message-id + "200609052027.NAA09861@hpsje.cup.hp.com"]]. [[!message-id + "1227217275.6205.6.camel@janis-laptop"]]. If needed, can re-implement in + GCC DejaGnu's `remote.exp:remote_wait` to get rid of (that is, ignore) its + `timeout` parameter which, in DejaGnu code, is often invoked with a + hard-coded value (that we may want to override) (or is that what + `gcc/testsuite/lib/timeout.exp:standard_wait` is for?). While at it, + `libmudflap/testsuite/libmudflap.c++/ctors.exp` and + `libmudflap/testsuite/libmudflap.c/externs.exp` use hard-coded timeout + values in `remote_wait` calls (also, why don't these use the usual way of + running tests?). * What is `gcc/testsuite/gcc.test-framework/test-framework.exp` and should we define `CHECK_TEST_FRAMEWORK` to run these tests? - * TODO - ## Enhancements diff --git a/open_issues/gccgo.mdwn b/open_issues/gccgo.mdwn index 18be3d93..0b90ac75 100644 --- a/open_issues/gccgo.mdwn +++ b/open_issues/gccgo.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -31,11 +31,10 @@ First, make the language functional, have its test suite pass without errors. [[!inline pages=community/gsoc/project_ideas/gccgo feeds=no]] -## Svante's work +## `tschwinge/t/hurd/go` -Per [[!message-id "1335509732.3707.179.camel@hp.my.own.domain"]], Svante has -been working on this, has some (unpublished) patches, and this was blocked on -[[`getcontext`/`makecontext`/`setcontext`/`swapcontext`|glibc]]. +There is now a `tschwinge/t/hurd/go` branch in GCC's Git repository, where the +Hurd port for Go is being polished. ## `getcontext`/`makecontext`/`setcontext`/`swapcontext` usage analysis diff --git a/open_issues/gdb.mdwn b/open_issues/gdb.mdwn index 62d9e435..07b3da45 100644 --- a/open_issues/gdb.mdwn +++ b/open_issues/gdb.mdwn @@ -27,14 +27,14 @@ Here's what's to be done for maintaining GNU GDB. <!-- git checkout reviewed -git diff --patience --stat=$COLUMNS,$COLUMNS --patch --src-prefix=./ --dst-prefix=./ --find-renames --ignore-space-change ..sourceware/master | awk '/^diff/ { c = $0; } /^@@/ { print c; } { print; }' | less +git diff --patience --stat=$COLUMNS,$COLUMNS --patch --src-prefix=./ --dst-prefix=./ --find-renames --ignore-space-change ..sourceware/master | awk '/^diff/ { c = " " $0; } /^@@/ { print c; } { print; }' | less -i /^---.*/([^.]*|.*\.texi.*|[^/]*gnu[^/]*)$|hurd|linux|nacl|nptl|glibc|gs: --> -Last reviewed up to the [[Git mirror's 3eb2a51c46804f0be530893e94af559abee8b4f8 -(2013-05-27) sources|source_repositories/gdb]]. +Last reviewed up to the [[Git mirror's d19fd090b7ec0a60846c52cad9fc0c24c3729565 +(2013-09-18) sources|source_repositories/gdb]]. * Globally @@ -66,16 +66,23 @@ Last reviewed up to the [[Git mirror's 3eb2a51c46804f0be530893e94af559abee8b4f8 * Configure so that Debian system's `/usr/lib/debug/[...]` will be loaded automatically. + * old-style function definition + + ../../Ferry_Tagscherer/gdb/gnu-nat.c: In function 'trace_me': + ../../Ferry_Tagscherer/gdb/gnu-nat.c:2106:8: warning: old-style function definition [-Wold-style-definition] + void trace_me () + ^ + # Build Here's a log of a GDB build run; this is from our [[Git repository|source_repositories/gdb]]'s `tschwinge/Ferry_Tagscherer` branch, -commit 3eb2a51c46804f0be530893e94af559abee8b4f8 (2013-05-27), run on +commit d19fd090b7ec0a60846c52cad9fc0c24c3729565 (2013-09-18), run on kepler.SCHWINGE and coulomb.SCHWINGE. $ export LC_ALL=C - $ ../Ferry_Tagscherer/configure --prefix="$PWD".install SHELL=/bin/dash CC=gcc-4.7 CXX=g++-4.7 --disable-werror 2>&1 | tee log_build + $ ../Ferry_Tagscherer/configure --prefix="$PWD".install SHELL=/bin/dash CC=gcc-4.8 CXX=g++-4.8 --disable-werror 2>&1 | tee log_build [...] $ make 2>&1 | tee log_build_ [...] @@ -87,7 +94,7 @@ There are several occurences of *error: dereferencing type-punned pointer will break strict-aliasing rules* in the MIG-generated stub files; thus no `-Werror` until that is resolved ([[strict_aliasing]]). -This takes up around 220 MiB and needs roughly 8 min on kepler.SCHWINGE and 25 +This takes up around 230 MiB and needs roughly 8 min on kepler.SCHWINGE and 31 min on coulomb.SCHWINGE. <!-- @@ -168,11 +175,6 @@ formats and more emulation vectors. exc_request_S.c:157:24: warning: no previous prototype for 'exc_server' [] exc_request_S.c:193:28: warning: no previous prototype for 'exc_server_routine' [] - * `dlopen`/`-ldl` - - -checking for library containing dlopen... none required - +checking for library containing dlopen... -ldl - * `O_NOFOLLOW` First seen in @@ -183,8 +185,6 @@ formats and more emulation vectors. [[!taglink open_issue_glibc]]? - * Why does GNU/Linux have an additional `-ldl -rdynamic` when linking `gdb`? - * 00e1c4298d2b6fe040a9a970e98349602b12ddbf..6b25dae901ddedb3f330803542d3eac73cdcae4b: +../../Ferry_Tagscherer/gdb/gnu-nat.c: In function 'info_port_rights': @@ -200,7 +200,7 @@ formats and more emulation vectors. $ make install 2>&1 | tee log_install [...] -This takes up around 50 MiB, and needs roughly 1 min on kepler.SCHWINGE and 2 +This takes up around 50 MiB, and needs roughly 1 min on kepler.SCHWINGE and 3 min on coulomb.SCHWINGE. @@ -216,7 +216,7 @@ min on coulomb.SCHWINGE. $ make -k check 2>&1 | tee log_test [...] -This needs roughly 14 min on kepler.SCHWINGE and 70 min on coulomb.SCHWINGE. +This needs roughly 15 min on kepler.SCHWINGE and 72 min on coulomb.SCHWINGE. When running `make -k check 2>&1 | tee log_test`, at the end of the testsuite the `tee` process does not terminate if there are still stray leftover @@ -229,6 +229,12 @@ GNU/Hurd, these generally are `gdb.multi/watchpoint-multi`, and an unknown ## Analysis +The testsuite uses the system's default `gcc` (and similar) compilers, not +those specified on the `configure` line ([[!taglink open_issue_gdb]]?), see +`find_gcc` (and similar) usage in the testsuite and DejaGnu. Maybe something +like `gdb/testsuite/boards/cc-with-tweaks.exp` would help, or setting +`CC_FOR_TARGET` (and similar) per `gdb/testsuite/lib/future.exp`? + $ toolchain/logs/process gdb test * `gdb.base/attach-pie-misread.exp` @@ -289,6 +295,15 @@ GNU/Hurd, these generally are `gdb.multi/watchpoint-multi`, and an unknown At this point, the system hangs; no new processes can be spawned, so perhaps an issue with the exec server. + * `gdb.threads/manythreads.exp` + + [[!taglink open_issue_libpthread]]. Perhaps fails due to pthread + attributes usage? Doesn't execute properly: + + $ gdb/testsuite/gdb.threads/manythreads + manythreads: ../libpthread/sysdeps/mach/pt-thread-halt.c:51: __pthread_thread_halt: Unexpected error: (ipc/rcv) invalid name. + Killed + * Linux syscall usage, `<asm/unistd.h>` * `UNSUPPORTED: gdb.threads/ia64-sigill.exp: Couldn't compile ../../../master/gdb/testsuite/gdb.threads/ia64-sigill.c: unrecognized error` @@ -347,6 +362,18 @@ GNU/Hurd, these generally are `gdb.multi/watchpoint-multi`, and an unknown Cannot access memory at address 0x6c62616e (gdb) testcase ../../../Ferry_Tagscherer/gdb/testsuite/gdb.base/attach-pie-noexec.exp completed in 3 seconds + IRC, freenode, #hurd, 2013-09-06: + + <gnu_srs1> How to debug a program that works in the shell but Cannot + access memory at address ... in gdb? + <tschwinge> Build it without -pie -- but that is just a guess of what + might be going on. + * tschwinge clearly has spent enough time with obscure things to be + able to make such guesses. + <gnu_srs1> tschwinge: looks like -fPIE is used. + <gnu_srs1> verified: some (all?) executables compiled with -fPIE, -fpie + and linked with -pie cannot be debugged in gdb :( + * `solib-event stop` Running ../../../Ferry_Tagscherer/gdb/testsuite/gdb.mi/mi-catch-load.exp ... @@ -363,15 +390,203 @@ GNU/Hurd, these generally are `gdb.multi/watchpoint-multi`, and an unknown *stopped,reason="signal-received",signal-name="SIGSEGV",signal-meaning="Segmentation fault",frame={addr="0x00014add",func="??",args=[],from="/lib/ld.so"},thread-id="4",stopped-threads="all" - * `gdb.arch/i386-float.exp: info float` + * `gdb.base/call-signal-resume.exp` + + $ gdb -q gdb/testsuite/gdb.base/call-signals + (gdb) break stop_one + (gdb) r + (gdb) call gen_signal() + (gdb) bt + (gdb) frame [<function called from gdb>] + (gdb) return + (gdb) break handle_signal + (gdb) c + (gdb) c + + kepler.SCHWINGE: + + Breakpoint 2, handle_signal (sig=6) at ../../../Ferry_Tagscherer/gdb/testsuite/gdb.base/call-signals.c:28 + 28 } + (gdb) bt + #0 handle_signal (sig=6) at ../../../Ferry_Tagscherer/gdb/testsuite/gdb.base/call-signals.c:28 + #1 <signal handler called> + #2 0xb7fde416 in __kernel_vsyscall () + #3 0xb7dffd96 in kill () at ../sysdeps/unix/syscall-template.S:81 + #4 0x0804859c in gen_signal () at ../../../Ferry_Tagscherer/gdb/testsuite/gdb.base/call-signals.c:35 + #5 0x08048610 in main () at ../../../Ferry_Tagscherer/gdb/testsuite/gdb.base/call-signals.c:81 + + coulomb.SCHWINGE: + + Breakpoint 2, handle_signal (sig=6) at ../../../Ferry_Tagscherer/gdb/testsuite/gdb.base/call-signals.c:28 + 28 } + (gdb) bt + #0 handle_signal (sig=6) at ../../../Ferry_Tagscherer/gdb/testsuite/gdb.base/call-signals.c:28 + #1 0x010baac2 in trampoline () from /lib/i386-gnu/libc.so.0.3 + #2 0x00000006 in ?? () + #3 0x00000000 in ?? () + + kepler.SCHWINGE: + + (gdb) c + Continuing. + no signal + [Inferior 1 (process 10401) exited normally] + + coulomb.SCHWINGE: + + (gdb) c + Continuing. + no signal + + Program received signal SIGSEGV, Segmentation fault. + 0x00000000 in ?? () + (gdb) bt + #0 0x00000000 in ?? () + #1 0x01116c28 in _IO_acquire_lock_fct (p=<synthetic pointer>) at libioP.h:905 + #2 _IO_puts (str=0x80487e0 "no signal") at ioputs.c:45 + #3 0x080486d8 in gen_signal () at ../../../Ferry_Tagscherer/gdb/testsuite/gdb.base/call-signals.c:38 + #4 0x0804873d in main () at ../../../Ferry_Tagscherer/gdb/testsuite/gdb.base/call-signals.c:81 + + This is apparently new with the glibc 2.17 upgrade. If not doing the + manual `gen_signal` call, it works fine. TODO. + + * `gdb.base/relativedebug.exp` + + (gdb) PASS: gdb.base/relativedebug.exp: continue + bt + #0 0x010a1afc in ?? () from /lib/i386-gnu/libc.so.0.3 + #1 0x010a23be in mach_msg () from /lib/i386-gnu/libc.so.0.3 + #2 0x0126cd98 in msg_sig_post () from /lib/i386-gnu/libhurduser.so.0.3 + #3 0x010e2141 in ?? () from /lib/i386-gnu/libc.so.0.3 + #4 0x010e23ed in kill () from /lib/i386-gnu/libc.so.0.3 + #5 0x010e17f4 in raise () from /lib/i386-gnu/libc.so.0.3 + #6 0x010e5b7c in abort () from /lib/i386-gnu/libc.so.0.3 + #7 0x08048607 in handler (signo=14) at ../../../Ferry_Tagscherer/gdb/testsuite/gdb.base/relativedebug.c:25 + #8 0x010bdac2 in ?? () from /lib/i386-gnu/libc.so.0.3 + Backtrace stopped: previous frame inner to this frame (corrupt stack?) + (gdb) FAIL: gdb.base/relativedebug.exp: pause found in backtrace + + This is apparently new with the glibc 2.17 upgrade. Previously it said: + + (gdb) PASS: gdb.base/relativedebug.exp: continue + bt + #0 0x0107c85c in ?? () from /lib/i386-gnu/libc.so.0.3 + #1 0x0107d069 in mach_msg () from /lib/i386-gnu/libc.so.0.3 + #2 0x01220d4f in msg_sig_post () from /lib/i386-gnu/libhurduser.so.0.3 + #3 0x010bb683 in ?? () from /lib/i386-gnu/libc.so.0.3 + #4 0x010bb8f6 in kill () from /lib/i386-gnu/libc.so.0.3 + #5 0x010bad76 in raise () from /lib/i386-gnu/libc.so.0.3 + #6 0x010bf029 in abort () from /lib/i386-gnu/libc.so.0.3 + #7 0x08048597 in handler (signo=14) at ../../../Ferry_Tagscherer/gdb/testsuite/gdb.base/relativedebug.c:25 + #8 0x01098282 in ?? () from /lib/i386-gnu/libc.so.0.3 + #9 0x010bbe5a in sigsuspend () from /lib/i386-gnu/libc.so.0.3 + #10 0x0112fee1 in pause () from /lib/i386-gnu/libc.so.0.3 + #11 0x080485c5 in main () at ../../../Ferry_Tagscherer/gdb/testsuite/gdb.base/relativedebug.c:32 + (gdb) PASS: gdb.base/relativedebug.exp: pause found in backtrace + + TODO. + + * `gdb.gdb/selftest.exp` + + (gdb) PASS: gdb.gdb/selftest.exp: send SIGINT signal to child process + backtrace + #0 0x0146fafc in ?? () from /lib/i386-gnu/libc.so.0.3 + #1 0x014703be in mach_msg () from /lib/i386-gnu/libc.so.0.3 + #2 0x0163bd98 in msg_sig_post () from /lib/i386-gnu/libhurduser.so.0.3 + #3 0x014b0141 in ?? () from /lib/i386-gnu/libc.so.0.3 + #4 0x014b03ed in kill () from /lib/i386-gnu/libc.so.0.3 + #5 0x082cf471 in _rl_handle_signal (sig=2) at ../../Ferry_Tagscherer/readline/signals.c:221 + #6 0x0148bac2 in ?? () from /lib/i386-gnu/libc.so.0.3 + Backtrace stopped: previous frame inner to this frame (corrupt stack?) + (gdb) FAIL: gdb.gdb/selftest.exp: backtrace through signal handler + + This is apparently new with the glibc 2.17 upgrade. Previously it said: + + (gdb) PASS: gdb.gdb/selftest.exp: send SIGINT signal to child process + backtrace + #0 0x0144885c in ?? () from /lib/i386-gnu/libc.so.0.3 + #1 0x01449069 in mach_msg () from /lib/i386-gnu/libc.so.0.3 + #2 0x015ecd4f in msg_sig_post () from /lib/i386-gnu/libhurduser.so.0.3 + #3 0x01487683 in ?? () from /lib/i386-gnu/libc.so.0.3 + #4 0x014878f6 in kill () from /lib/i386-gnu/libc.so.0.3 + #5 0x082cf401 in _rl_handle_signal (sig=2) at ../../Ferry_Tagscherer/readline/signals.c:221 + #6 0x01464282 in ?? () from /lib/i386-gnu/libc.so.0.3 + #7 0x0144fce3 in ?? () from /lib/i386-gnu/libc.so.0.3 + #8 0x0153975b in poll () from /lib/i386-gnu/libc.so.0.3 + #9 0x081c91c2 in gdb_wait_for_event (block=1) at ../../Ferry_Tagscherer/gdb/event-loop.c:804 + #10 0x081c998f in gdb_do_one_event () at ../../Ferry_Tagscherer/gdb/event-loop.c:402 + #11 0x081c9b07 in start_event_loop () at ../../Ferry_Tagscherer/gdb/event-loop.c:431 + #12 0x081c2f42 in captured_command_loop (data=data@entry=0x0) at ../../Ferry_Tagscherer/gdb/main.c:260 + #13 0x081c0e57 in catch_errors (func=func@entry=0x81c2f30 <captured_command_loop>, func_args=func_args@entry=0x0, errstring=errstring@entry=0x83 + 5b81b "", mask=mask@entry=6) at ../../Ferry_Tagscherer/gdb/exceptions.c:546 + #14 0x081c388c in captured_main (data=data@entry=0x19ff150) at ../../Ferry_Tagscherer/gdb/main.c:1055 + #15 0x081c0e57 in catch_errors (func=func@entry=0x81c3130 <captured_main>, func_args=func_args@entry=0x19ff150, errstring=errstring@entry=0x835b + 81b "", mask=mask@entry=6) at ../../Ferry_Tagscherer/gdb/exceptions.c:546 + #16 0x081c43c0 in gdb_main (args=0x19ff150) at ../../Ferry_Tagscherer/gdb/main.c:1064 + #17 0x08099533 in main (argc=5, argv=0x19ff1e8) at ../../Ferry_Tagscherer/gdb/gdb.c:34 + (gdb) PASS: gdb.gdb/selftest.exp: backtrace through signal handler + + TODO. + + * `gdb.python/python.exp` + + >PASS: gdb.python/python.exp: post event insertion - gdb.post_event(Foo()) + end + ERROR: Process no longer exists + UNRESOLVED: gdb.python/python.exp: post event insertion - end + + This is apparently new with the glibc 2.17 upgrade. Previously it said: + + >PASS: gdb.python/python.exp: post event insertion - gdb.post_event(Foo()) + end + (gdb) PASS: gdb.python/python.exp: post event insertion - end + [More PASSes.] + + TODO. + + * `gdb.base/restore.exp`, `gdb.base/store.exp` + + Several FAILs, starting with GCC 4.8 usage: + + (gdb) PASS: gdb.base/restore.exp: caller3 calls callee1; return callee now + print l1 + $16 = <optimized out> + (gdb) FAIL: gdb.base/restore.exp: caller3 calls callee1; return restored l1 to 32492 + + [[!GCC_PR 55056]], [[!message-id + "20130126202645.GA4888@host2.jankratochvil.net"]], and maybe [[!message-id + "CAO2gOZXvCLdaKE2=ZKpjGVGq8A0wQ94-AUo7eKvvWHWncrU_yg@mail.gmail.com"]] look + related. - Running ../../../Ferry_Tagscherer/gdb/testsuite/gdb.arch/i386-float.exp ... - PASS: gdb.arch/i386-float.exp: first stepi - FAIL: gdb.arch/i386-float.exp: info float - PASS: gdb.arch/i386-float.exp: second stepi - PASS: gdb.arch/i386-float.exp: info float +TODO. - Only fails for GNU/Hurd: the floating point stack initially is not - all-zeroes, which is expected, and which it is on GNU/Linux. -TODO. +# Open Issues + +## [[tag/open_issue_gdb]] + +## `info files` SIGSEGV + +[[!tag open_issue_gdb]] + + +### IRC, freenode, #hurd, 2013-09-07 + + <rekado> I'm trying to debug pfinet, but I'm not very familiar with gdb. + Tried to attach to the running pfinet process (built with debug symbols), + set a breakpoint and ... when I ran "info files" the process segfaulted. + <teythoon> which process segfaults, pfinet or gdb? + <rekado> gdb segfaults. + + +## Watchpoints + +[[!tag open_issue_gdb]] + + +### IRC, freenode, #hurd, 2013-09-16 + + <gnu_srs> tschwinge: Is gdb watch known to fail on hurd? It hangs for me + when logged in via ssh. + <tschwinge> gnu_srs: Don't know about GDB's watch command. Are you sure it + is hanging? diff --git a/open_issues/gdb_gcore.mdwn b/open_issues/gdb_gcore.mdwn index 69211ac0..cadd9be1 100644 --- a/open_issues/gdb_gcore.mdwn +++ b/open_issues/gdb_gcore.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2009, 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2011, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -22,5 +23,8 @@ GDB's `gcore` command doesn't work / needs to be implemented / ported in GDB: Undefined command: "gcore". Try "help". gcore: failed to create core.8371 +Will probably need to implement `gdb/gdbarch.sh:gdb_signal_from_target`, +`gdb/gdbarch.sh:gdb_signal_to_target`. + If someone is working in this area, they may want to port <http://code.google.com/p/google-coredumper/>, too. diff --git a/open_issues/gdb_signal_handler.mdwn b/open_issues/gdb_signal_handler.mdwn new file mode 100644 index 00000000..3084f7e3 --- /dev/null +++ b/open_issues/gdb_signal_handler.mdwn @@ -0,0 +1,403 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_gdb open_issue_glibc]] + + +# IRC, freenode, #hurd, 2013-07-07 + + <zyg> Hi, I'm in GDB inside a handler for SIGHUP, after stepping out, gdb + will hang on instruction: <_hurd_sigstate_lock+88>: xchg + %edx,0x4(%eax) + <zyg> here is my signal test pasted: http://pastebin.com/U72qw3FC + #include <stdio.h> + #include <stdlib.h> + #include <signal.h> + + void * + my_handler(int signal, void *info, void *context) + { + printf("got SIGHUP\n"); + return NULL; + } + + void + install_handler (int signal) + { + struct sigaction sa; + sa.sa_sigaction = my_handler; + sa.sa_flags = SA_SIGINFO; + sigemptyset(&sa.sa_mask); + sigaction(signal, &sa, NULL); + } + + void test_sighup(void) + { + raise(SIGHUP); + } + + int main(int argc, char **argv){ + install_handler(SIGHUP); + test_sighup(); + exit(1); + } + <braunr> zyg: thanks + <braunr> zyg: what is the problem exactly ? + <braunr> zyg: i mean, does it hand before attaching with gdb ? + <zyg> braunr: it doesn't hang if runned without gdb. I've pasted here when + I step out of the handler, and get to the hanging instruction: + http://pastebin.com/nUyCx6Wj + $ gdb --args a.out + GNU gdb (GDB) 7.6-debian + Copyright (C) 2013 Free Software Foundation, Inc. + License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html> + This is free software: you are free to change and redistribute it. + There is NO WARRANTY, to the extent permitted by law. Type "show copying" + and "show warranty" for details. + This GDB was configured as "i486-gnu". + For bug reporting instructions, please see: + <http://www.gnu.org/software/gdb/bugs/>... + Reading symbols from /home/shrek/a.out...(no debugging symbols found)...done. + (gdb) + + (gdb) display/i $pc + (gdb) handle SIGHUP pass stop print + Signal Stop Print Pass to program Description + SIGHUP Yes Yes Yes Hangup + (gdb) + + (gdb) run + Starting program: /home/shrek/a.out + [New Thread 3571.5] + + Program received signal SIGHUP, Hangup. + 0x010548ec in mach_msg_trap () + at /build/buildd-eglibc_2.17-6-hurd-i386-g946kE/eglibc-2.17/build-tree/hurd-i386-libc/mach/mach_msg_trap.S:2 + 2 /build/buildd-eglibc_2.17-6-hurd-i386-g946kE/eglibc-2.17/build-tree/hurd-i386-libc/mach/mach_msg_trap.S: No such file or directory. + 1: x/i $pc + => 0x10548ec <mach_msg_trap+12>: ret + (gdb) + + (gdb) si + 0x0804862d in my_handler () + 1: x/i $pc + => 0x804862d <my_handler>: push %ebp + (gdb) x/20xi 0x804862d + => 0x804862d <my_handler>: push %ebp + 0x804862e <my_handler+1>: mov %esp,%ebp + 0x8048630 <my_handler+3>: sub $0x18,%esp + 0x8048633 <my_handler+6>: movl $0x8048750,(%esp) + 0x804863a <my_handler+13>: call 0x8048500 <puts@plt> + 0x804863f <my_handler+18>: mov $0x0,%eax + 0x8048644 <my_handler+23>: leave + 0x8048645 <my_handler+24>: ret + 0x8048646 <install_handler>: push %ebp + 0x8048647 <install_handler+1>: mov %esp,%ebp + 0x8048649 <install_handler+3>: sub $0x28,%esp + 0x804864c <install_handler+6>: movl $0x804862d,-0x14(%ebp) + 0x8048653 <install_handler+13>: movl $0x40,-0xc(%ebp) + 0x804865a <install_handler+20>: lea -0x14(%ebp),%eax + 0x804865d <install_handler+23>: add $0x4,%eax + 0x8048660 <install_handler+26>: mov %eax,(%esp) + 0x8048663 <install_handler+29>: call 0x80484d0 <sigemptyset@plt> + 0x8048668 <install_handler+34>: movl $0x0,0x8(%esp) + 0x8048670 <install_handler+42>: lea -0x14(%ebp),%eax + 0x8048673 <install_handler+45>: mov %eax,0x4(%esp) + (gdb) + + (gdb) break *0x804863f + Breakpoint 1 at 0x804863f + (gdb) c + Continuing. + got SIGHUP + + Breakpoint 1, 0x0804863f in my_handler () + 1: x/i $pc + => 0x804863f <my_handler+18>: mov $0x0,%eax + (gdb) + + (gdb) si + 0x08048644 in my_handler () + 1: x/i $pc + => 0x8048644 <my_handler+23>: leave + (gdb) + 0x08048645 in my_handler () + 1: x/i $pc + => 0x8048645 <my_handler+24>: ret + (gdb) + 0x010708b2 in trampoline () from /lib/i386-gnu/libc.so.0.3 + 1: x/i $pc + => 0x10708b2 <trampoline+2>: add $0xc,%esp + (gdb) + 0x010708b5 in trampoline () from /lib/i386-gnu/libc.so.0.3 + 1: x/i $pc + => 0x10708b5 <trampoline+5>: ret + (gdb) + __sigreturn (scp=0x102988c) at ../sysdeps/mach/hurd/i386/sigreturn.c:30 + 30 ../sysdeps/mach/hurd/i386/sigreturn.c: No such file or directory. + 1: x/i $pc + => 0x1096340 <__sigreturn>: push %ebp + (gdb) + 0x01096341 30 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096341 <__sigreturn+1>: push %edi + (gdb) + 0x01096342 30 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096342 <__sigreturn+2>: push %esi + (gdb) + 0x01096343 30 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096343 <__sigreturn+3>: push %ebx + (gdb) + 0x01096344 30 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096344 <__sigreturn+4>: sub $0x2c,%esp + (gdb) + 0x01096347 30 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096347 <__sigreturn+7>: mov 0x40(%esp),%esi + (gdb) + 0x0109634b 30 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x109634b <__sigreturn+11>: call 0x11a0609 <__x86.get_pc_thunk.bx> + (gdb) + 0x011a0609 in __x86.get_pc_thunk.bx () from /lib/i386-gnu/libc.so.0.3 + 1: x/i $pc + => 0x11a0609 <__x86.get_pc_thunk.bx>: mov (%esp),%ebx + (gdb) + 0x011a060c in __x86.get_pc_thunk.bx () from /lib/i386-gnu/libc.so.0.3 + 1: x/i $pc + => 0x11a060c <__x86.get_pc_thunk.bx+3>: ret + (gdb) + 0x01096350 in __sigreturn (scp=0x102988c) at ../sysdeps/mach/hurd/i386/sigreturn.c:30 + 30 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096350 <__sigreturn+16>: add $0x15ccb0,%ebx + (gdb) + 35 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096356 <__sigreturn+22>: test %esi,%esi + (gdb) + 0x01096358 35 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096358 <__sigreturn+24>: je 0x10964f0 <__sigreturn+432> + (gdb) + 0x0109635e 35 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x109635e <__sigreturn+30>: testl $0x10100,0x4(%esi) + (gdb) + 0x01096365 35 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096365 <__sigreturn+37>: jne 0x10964f0 <__sigreturn+432> + (gdb) + __hurd_threadvar_location_from_sp (__sp=<optimized out>, __index=<optimized out>) at ../hurd/hurd/threadvar.h:94 + 94 ../hurd/hurd/threadvar.h: No such file or directory. + 1: x/i $pc + => 0x109636b <__sigreturn+43>: mov -0x38(%ebx),%ebp + (gdb) + __hurd_threadvar_location (__index=_HURD_THREADVAR_SIGSTATE) at ../hurd/hurd/threadvar.h:116 + 116 in ../hurd/hurd/threadvar.h + 1: x/i $pc + => 0x1096371 <__sigreturn+49>: mov %esp,%edx + (gdb) + __hurd_threadvar_location_from_sp (__sp=0x1029848, __index=_HURD_THREADVAR_SIGSTATE) at ../hurd/hurd/threadvar.h:94 + 94 in ../hurd/hurd/threadvar.h + 1: x/i $pc + => 0x1096373 <__sigreturn+51>: cmp 0x0(%ebp),%esp + (gdb) + 0x01096376 94 in ../hurd/hurd/threadvar.h + 1: x/i $pc + => 0x1096376 <__sigreturn+54>: jae 0x10964d0 <__sigreturn+400> + (gdb) + 0x0109637c 94 in ../hurd/hurd/threadvar.h + 1: x/i $pc + => 0x109637c <__sigreturn+60>: mov -0x15c(%ebx),%eax + (gdb) + 0x01096382 94 in ../hurd/hurd/threadvar.h + 1: x/i $pc + => 0x1096382 <__sigreturn+66>: and (%eax),%edx + (gdb) + 0x01096384 94 in ../hurd/hurd/threadvar.h + 1: x/i $pc + => 0x1096384 <__sigreturn+68>: mov -0x90(%ebx),%eax + (gdb) + 0x0109638a 94 in ../hurd/hurd/threadvar.h + 1: x/i $pc + => 0x109638a <__sigreturn+74>: add (%eax),%edx + (gdb) + _hurd_self_sigstate () at ../hurd/hurd/signal.h:165 + 165 ../hurd/hurd/signal.h: No such file or directory. + 1: x/i $pc + => 0x109638c <__sigreturn+76>: mov 0x8(%edx),%edi + (gdb) + 0x0109638f 165 in ../hurd/hurd/signal.h + 1: x/i $pc + => 0x109638f <__sigreturn+79>: test %edi,%edi + (gdb) + 0x01096391 165 in ../hurd/hurd/signal.h + 1: x/i $pc + => 0x1096391 <__sigreturn+81>: je 0x1096598 <__sigreturn+600> + (gdb) + __sigreturn (scp=0x102988c) at ../sysdeps/mach/hurd/i386/sigreturn.c:42 + 42 ../sysdeps/mach/hurd/i386/sigreturn.c: No such file or directory. + 1: x/i $pc + => 0x1096397 <__sigreturn+87>: mov %edi,(%esp) + (gdb) + 0x0109639a 42 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x109639a <__sigreturn+90>: call 0x1051d70 <_hurd_sigstate_lock@plt> + (gdb) + 0x01051d70 in _hurd_sigstate_lock@plt () from /lib/i386-gnu/libc.so.0.3 + 1: x/i $pc + => 0x1051d70 <_hurd_sigstate_lock@plt>: jmp *0x864(%ebx) + (gdb) + _hurd_sigstate_lock (ss=ss@entry=0x1244008) at hurdsig.c:170 + 170 hurdsig.c: No such file or directory. + 1: x/i $pc + => 0x106bb90 <_hurd_sigstate_lock>: sub $0x1c,%esp + (gdb) + 0x0106bb93 170 in hurdsig.c + 1: x/i $pc + => 0x106bb93 <_hurd_sigstate_lock+3>: mov %ebx,0x14(%esp) + (gdb) + 0x0106bb97 170 in hurdsig.c + 1: x/i $pc + => 0x106bb97 <_hurd_sigstate_lock+7>: call 0x11a0609 <__x86.get_pc_thunk.bx> + (gdb) + 0x011a0609 in __x86.get_pc_thunk.bx () from /lib/i386-gnu/libc.so.0.3 + 1: x/i $pc + => 0x11a0609 <__x86.get_pc_thunk.bx>: mov (%esp),%ebx + (gdb) + 0x011a060c in __x86.get_pc_thunk.bx () from /lib/i386-gnu/libc.so.0.3 + 1: x/i $pc + => 0x11a060c <__x86.get_pc_thunk.bx+3>: ret + (gdb) + 0x0106bb9c in _hurd_sigstate_lock (ss=ss@entry=0x1244008) at hurdsig.c:170 + 170 in hurdsig.c + 1: x/i $pc + => 0x106bb9c <_hurd_sigstate_lock+12>: add $0x187464,%ebx + (gdb) + 0x0106bba2 170 in hurdsig.c + 1: x/i $pc + => 0x106bba2 <_hurd_sigstate_lock+18>: mov %esi,0x18(%esp) + (gdb) + 170 in hurdsig.c + 1: x/i $pc + => 0x106bba6 <_hurd_sigstate_lock+22>: mov 0x20(%esp),%esi + (gdb) + sigstate_is_global_rcv (ss=0x1244008) at hurdsig.c:162 + 162 in hurdsig.c + 1: x/i $pc + => 0x106bbaa <_hurd_sigstate_lock+26>: lea 0x57c0(%ebx),%eax + (gdb) + 0x0106bbb0 162 in hurdsig.c + 1: x/i $pc + => 0x106bbb0 <_hurd_sigstate_lock+32>: mov (%eax),%eax + (gdb) + 163 in hurdsig.c + 1: x/i $pc + => 0x106bbb2 <_hurd_sigstate_lock+34>: test %eax,%eax + (gdb) + 0x0106bbb4 163 in hurdsig.c + 1: x/i $pc + => 0x106bbb4 <_hurd_sigstate_lock+36>: je 0x106bbbc <_hurd_sigstate_lock+44> + (gdb) + 0x0106bbb6 163 in hurdsig.c + 1: x/i $pc + => 0x106bbb6 <_hurd_sigstate_lock+38>: cmpl $0x1,0x18(%esi) + (gdb) + 0x0106bbba 163 in hurdsig.c + 1: x/i $pc + => 0x106bbba <_hurd_sigstate_lock+42>: je 0x106bbe0 <_hurd_sigstate_lock+80> + (gdb) + _hurd_sigstate_lock (ss=ss@entry=0x1244008) at hurdsig.c:172 + 172 in hurdsig.c + 1: x/i $pc + => 0x106bbe0 <_hurd_sigstate_lock+80>: lea 0x4(%eax),%ecx + (gdb) + __spin_try_lock (__lock=0x124480c) at ../sysdeps/mach/i386/machine-lock.h:59 + 59 ../sysdeps/mach/i386/machine-lock.h: No such file or directory. + 1: x/i $pc + => 0x106bbe3 <_hurd_sigstate_lock+83>: mov $0x1,%edx + (gdb) + 0x0106bbe8 59 in ../sysdeps/mach/i386/machine-lock.h + 1: x/i $pc + => 0x106bbe8 <_hurd_sigstate_lock+88>: xchg %edx,0x4(%eax) + (gdb) + <braunr> zyg: i don't get what you mean + <braunr> are you starting it with gdb ? + <zyg> braunr: yes: "gdb --args a.out" + <braunr> ok + <braunr> can't reproduce it + <braunr> i get "Program received signal SIGHUP, Hangup. + <braunr> " + <braunr> then continue, then the program has exited + <zyg> braunr: do you run it in gdb or without? + <zyg> braunr: Ah "Program received signal SIGHUP, Hangup." is from + gdb.. try issue continue, not sure why gdb stops at SIGHUP (default?). + <braunr> 10:34 < braunr> then continue, then the program has exited + <braunr> gdb stops at signals + <zyg> braunr: yes, try repeating that, but instead of continue, just issue + "si" + <zyg> braunr: sorry.. you would need to remove that printf/fprintf, else it + gets too long. That's why I put a breakpoint. + <braunr> a breakpoint ? + <braunr> on the signal handler ? + <zyg> braunr: yes, put a break after having entered the handler. Or edit + the pasted C code an remove that printf("got SIGHUP\n"); + <braunr> i'm not sure that's correctly supported + <braunr> and i can see why glibc would deadlock on the sigstate lock + <braunr> don't do that :p + <zyg> braunr: why does it deadlock? + <braunr> because both the signal handler and messages from gdb will cause + common code paths to be taken + <zyg> braunr: oh.. when I step instruction I'm inside an SIGTRAP handler + probably? + <braunr> possible + <braunr> i don't know the details but that's the kind of things i expect + <braunr> and signals on the hurd are definitely buggy + <braunr> i don't know if we support nesting them + <braunr> i'd say we don't + <zyg> braunr: I'll try to put a break beyond that xchg and continue + <braunr> xhcg is probably the sigstate spinlock + <braunr> xchg* + <braunr> you'd need to reach the unlock instruction, which is probably + executed once the handler has finished running + <zyg> braunr: yes :) ... one instruction beyond didn't help + <zyg> braunr: thanks alot, putting a break in __sigreturn, after that + function has called _hurd_sigstate_unlock@plt works! + <braunr> works ? + <braunr> what did you want to do ? + <zyg> braunr: I want to trace user code inside the signal handler, also how + we enter and how we leave. + <braunr> well you can't do that inside, so no it doesn't work for you :/ + <braunr> but that's a start i guess + <zyg> braunr: I seem to do most normal things inside the handler, + step-instruction and put breaks. + <braunr> ? + <braunr> i thought that's what made the program deadlock + <zyg> braunr: as you said earlier, the deadlock came when i "step + instruction" into the area between _hurd_sigstate_lock and + _hurd_sigstate_unlock. Otherwise I havn't had any issues. + <braunr> but isn't the sigstate locked during the handler execution ? + <zyg> braunr: no it locks and unlocks in __sigreturn which is done when + leaving the handler. + <braunr> than how could it deadlock on handler entry ? + <braunr> or perhaps the fact your handler was empty made the entry point + directly reach __sigreturn + <braunr> hm no i don't buy it + <braunr> the sigstate must also be locked on entry + <zyg> braunr: there was never any problem with entering + <braunr> then describe the problem with more details please + <braunr> ah sorry + <zyg> braunr: are you sure? there is minimal user-code run before the + signal is going into the handler. + <braunr> you "step out of the handler" diff --git a/open_issues/glibc.mdwn b/open_issues/glibc.mdwn index d45f0d14..b453b44f 100644 --- a/open_issues/glibc.mdwn +++ b/open_issues/glibc.mdwn @@ -281,14 +281,55 @@ Last reviewed up to the [[Git mirror's 0323d08657f111267efa47bd448fbf6cd76befe8 [[glibc_madvise_vs_static_linking]]. + IRC, OFTC, #debian-hurd, 2013-09-09: + + <gg0> does hurd MADV_DONTNEED or MADV_FREE or none? + http://sources.debian.net/src/jemalloc/3.4.0-1/include/jemalloc/jemalloc_defs.h.in#L239 + <gg0> seems it builds by defining JEMALLOC_PURGE_MADVISE_DONTNEED + but i don't know what i'm talking about, so it could build with + JEMALLOC_PURGE_MADVISE_FREE as well + + IRC, OFTC, #debian-hurd, 2013-09-10: + + <youpi> gg0: it implements none, even if it defines DONTNEED (but + not FREE) + + See also: + + gnash (0.8.11~git20130903-1) unstable; urgency=low + + * Git snapshot. + + Embedded jemalloc copy has been replaced by system one. + [...] + - Disable jemalloc on hurd and kfreebsd-*. No longer disabled upstream. + * `msync` Then define `_POSIX_MAPPED_FILES`, `_POSIX_SYNCHRONIZED_IO`. - * `sys/epoll.h` + * `epoll`, `sys/epoll.h` Used by [[wayland]], for example. + IRC, freenode, #hurd, 2013-08-08: + + <nalaginrut> is there any possible to have kquque/epoll alike + things in hurd? or there is one? + <braunr> nalaginrut: use select/poll + <nalaginrut> is it possible to implement epoll? + <braunr> it is + <braunr> we don't care enough about it to do it + <braunr> (for now) + <nalaginrut> well, since I wrote a server with Guile, and it could + take advantage of epoll, never mind, if there's no, it'll use + select automatically + <nalaginrut> but if someday someone care about it, I'll be + interested on it + <braunr> epoll is a scalability improvement over poll + <braunr> the hurd being full of scalability issues, this one is + clearly not a priority + <nalaginrut> ok + * `sys/eventfd.h` * `sys/inotify.h` @@ -390,6 +431,429 @@ Last reviewed up to the [[Git mirror's 0323d08657f111267efa47bd448fbf6cd76befe8 libgc, libsigsegv, luatex, mono, nspr, pth, ruby1.8, texlive-bin, uim, and more. + IRC, OFTC, #debian-hurd, 2013-09-08: + + <pinotree> oh, and even ruby2.0 suffers because of fixed-stack + threads + <youpi> yes, we definitely need to finish fixing it + <youpi> my current work is in our glibc repo, youpi/tls-threadvar + <pinotree> | *** makecontext: a stack at 0xbc000 with size 0x40000 + is not usable with threadvars + <pinotree> all 8 failing tests with that + <youpi> maybe we can hand-disable the use of contexts in ruby for + now? + <pinotree> gg0: ↑ :) + <gg0> after the pseudo-patch i RFCed, i don't deserve to say + anything else about that :) + <pinotree> i mean, feel free to investigate and "fix" ruby2.0 as + above :) + <gg0> eh maybe i'd just be able to hand-disable failing + thread-related _tests_ :) + <gg0> i'm still hoping some real developer picks and actually fixes + it, seems it's not enough interesting though + <azeem> 21:37 < youpi> yes, we definitely need to finish fixing it + <gg0> afaiu youpi is working on threadvars-tls migration, which + would mean fixing them all. i just meant fixing ruby, which would + mean having puppet btw + <youpi> gg0: "actually fixing" means fixing threadvars-tls + migration + <youpi> "just fixing" ruby can be done by simply disabling context + use in ruby + + IRC, OFTC, #debian-hurd, 2013-09-10: + + <gg0> this one fixes make test by disabling context and giving more + time to timing related tests http://paste.debian.net/plain/37977/ + <gg0> make test-all is another story + <youpi> gg0: AIUI, the sleep part should get fixed by the next + glibc upload, which will include the getclk patch + <youpi> but the disabling context part could be good to submit to + the debian ruby package, mentioning that this is a workaround for + now + <gg0> unfortunately still not enough, test-all still fails + <youpi> does it make the package not build? + <gg0> test-all is the second part of what we call tests + <gg0> they build and package (they produce all ruby packages), + after that they run debian/run-test-suites.bash which is make + test + make test-all + <gg0> well after or during the build doesn't matter, it's their + testsuite + <gg0> ok just failed: + <gg0> TestBug4409#test_bug4409 = Illegal instruction + <gg0> make: *** [yes-test-all] Error 132 + <gg0> what to do with Illegal instruction? + <gg0> just found 2 words that make everybody shut up :p + <pinotree> same as above: debug it + <teythoon> gg0: have you confirmed that this is reproducible? I've + once had a process die with SIGILL and it was not and I figured + it might have been a (qemu?) glitch + <gg0> seems i'm running tests which are disabled on _all_ archs, + better so + <gg0> well, this should be reproducible. i just got it on a qemu, i + could try to reproduce it on real hardware but as just said, i + was testing tests disabled by maintainer so completely useless + <teythoon> gg0: yeah, I'm running all my hurd instances on qemu/kvm + as well, I meant did you get this twice in a row? + <gg0> to be honest i got another illegal instruction months ago but + don't recall doing what + <gg0> nope not twice, i've commented it out. then run the remaining + and then found out i should not have done what i was doing + <gg0> but i could try to reproduce it + <gg0> ok now i recall i got it another one few hours ago on real + hardware, from logs: + <gg0> TestIO#test_copy_stream_socket = Illegal instruction + <gg0> teythoon: on real hardware though + <gg0> and this is the one i should debug once it finishes, still + running + + IRC, freenode, #hurd, 2013-09-11: + + <gg0_> ../sysdeps/mach/hurd/jmp-unwind.c:53: _longjmp_unwind: + Assertion `! __spin_lock_locked (&ss->critical_section_lock)' + failed. + <gg0_> and + <gg0_> ../libpthread/sysdeps/mach/pt-thread-halt.c:51: + __pthread_thread_halt: Unexpected error: (ipc/send) invalid + destination port. + <tschwinge> gg0_: Which libpthread source are these? Stock Debian + package? + <gg0_> tschwinge: everything debian, ruby rebuilt with + http://paste.debian.net/plain/38519/ which should disable + *context + + IRC, OFTC, #debian-hurd, 2013-09-11: + + <gg0_> wrt ruby, i'd propose a patch that disables *context and + comments out failed tests (a dozen). most of them are timing + related, don't always fail + <gg0_> if they failed gracefully, we could leave them enabled and + just ignoring testsuite result, but most of them block testsuite + run when fail + <gg0_> anyone against? any better idea (and intention to implement + it? :p)? + <gg0_> youpi: is disabling some tests acceptable? ^ + <youpi> it'd be good to at least know what is failing + <youpi> so as to know what impact hiding these failures will have + <youpi> remember that hiding bugs usually means getting bitten by + them even harder later :) + <gg0_> many of them use pipes + <gg0_> here the final list, see commented out ones + http://paste.debian.net/plain/38426 + <gg0_> and as said some don't always fails + <gg0_> test_copy_stream_socket uses a socket + <youpi> note that we can still at least build packages with notest + <youpi> at least to get the binaries uploaded + <youpi> disabling *context should however really be done + <youpi> and the pipe issues are concerning + <youpi> I don't remember other pipe issues + <youpi> so maybe it's a but in the ruby bindings + <gg0_> i just remember they didn't die, then something unknown + fixed it + <youpi> I see something frightening in io.c + <youpi> #if BSD_STDIO + <youpi> preserving_errno(fseeko(f, lseek(fileno(f), + (off_t)0, SEEK_CUR), SEEK_SET)); + <youpi> #endif + <youpi> this looks very much like a workaround for an odd thing in + BSD + <youpi> it happens that that gets enabled on hurd too, since + __MACH__ is defined + <youpi> you could try to drop these three lines, just to see + <youpi> this is very probably very worth investigating, at any rate + <youpi> even just test_gets_limit_extra_arg is a very simple test, + that I fail to see why it should ever fail on hurd-i386 + <youpi> starting debugging it would be a matter of putting printfs + in io.c, to check what gets called, with what parameters, etc. + <youpi> just a matter of taking the time to do it, it's not very + complex + <gg0_> youpi: are you looking at 1.8? no BSD_STDIO here + <youpi> yes, 1.8 + <gg0_> 1.9.3.448 + <gg0_> landed to sid few days ago + <youpi> ah, I have 1.87 + <youpi> +. + <gg0_> my favourites are TestIO#test_copy_stream_socket and + TestIO#test_cross_thread_close_fd -> Illegal instruction + <gg0_> TestIO#test_io_select_with_many_files sometimes Illegal + instruction, sometimes ruby1.9.1: + ../sysdeps/mach/hurd/jmp-unwind.c:53: _longjmp_unwind: Assertion + `! __spin_lock_locked (&ss->critical_section_lock)' failed. + + [[thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock]]? + + <gg0_> trying to debug illegal instruction + http://paste.debian.net/plain/38585/ + <gg0_> (yes, i'm not even good at gdbing) + <gg0_> any hint? + <gg0_> oh found out there's an intree .gdbinit, that might + complicate things + + IRC, OFTC, #debian-hurd, 2013-09-13: + + <gg0_> where should it be implemented MAP_STACK? plus, is it worth + doing it considering migration to tls, wouldn't it be useless? + <gg0_> sysdeps/mach/hurd/mmap.c i should reduce stupid questions + frequency from daily to weekly basis + + IRC, OFTC, #debian-hurd, 2013-09-14: + + <gg0_> say i managed to mmap 0x200000-aligned memory + <gg0_> now i get almost the same failed tests i get disabling + *context + <gg0_> that would mean they don't depend on threading + + IRC, freenode, #hurd, 2013-09-16: + + <gg0> i get many ../sysdeps/mach/hurd/jmp-unwind.c:53: + _longjmp_unwind: Assertion `! __spin_lock_locked + (&ss->critical_section_lock)' failed. + <gg0> by running ruby testsuite, especially during test_read* tests + http://sources.debian.net/src/ruby1.9.1/1.9.3.448-1/test/ruby/test_io.rb#L972 + <gg0> read/write operations with pipes + <braunr> gg0: that's weird + <braunr> gg0: debian glibc ? + <gg0> braunr: yep, debian 2.17-92 + <gg0> sometimes assertion above, sometimes tests in question get + stuck reading + <gg0> it would be nice reproducing it w/o ruby + <gg0> probably massive io on pipes could do the job + <gg0> also more nice finding someone who finds it interesting to + fix :p + <gg0> ruby is rebuilt with http://paste.debian.net/plain/40755/, no + *context + <gg0> pipe function in tests above creates one thread for write, + one for read + http://sources.debian.net/src/ruby1.9.1/1.9.3.448-1/test/ruby/test_io.rb#L26 + <tschwinge> gg0: About the jmp-unwind assertion failure: is it be + chance this issue: + <http://www.gnu.org/software/hurd/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.html>? + I didn't look in detail. + <braunr> tschwinge: that's what i thought too about the assertion, + which is why i find it strange + <gg0> asserting it's not locked then locking it doesn't exclude + race conditions + + IRC, OFTC, #debian-hurd, 2013-09-17: + + <gg0> youpi: i guess no one saw it anymore since + tg-thread-cancel.diff patch + <gg0> it = + http://www.gnu.org/software/hurd/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.html + <gg0> this one comes from sysdeps/mach/hurd/jmp-unwind.c:53 though + <gg0> another assertion to remove? + <youpi> gg0: it's not exactly the same: in hurd_thread_cancel we + hold no lock at all at the assertion point + <youpi> in jmp-unwind.c, we do hold a lock + <youpi> and the assertion might be actually true because all other + threads are supposed to hold the first lock before taking the + other one + <youpi> you could check for that in other places + <youpi> and maybe it's the other place which wouldhave to be fixed + <youpi> also look for documentation which would say that + + IRC, freenode, #hurd, 2013-09-17: + + <braunr_> gg0: is that what we do ?? + <gg0> braunr: well, i was looking at + http://sources.debian.net/src/eglibc/2.17-92/debian/patches/hurd-i386/tg-thread-cancel.diff + <gg0> which afaics fixes + http://www.gnu.org/software/hurd/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.html + <gg0> the one i get now is + http://sources.debian.net/src/eglibc/2.17-92/sysdeps/mach/hurd/jmp-unwind.c#L53 + <gg0> 09:12 < youpi> gg0: it's not exactly the same: in + hurd_thread_cancel we hold no lock at all at the assertion point + <gg0> 09:13 < youpi> in jmp-unwind.c, we do hold a lock + <gg0> 09:13 < youpi> and the assertion might be actually true + because all other threads are supposed to hold the first lock + before taking the other one + <braunr> gg0: that assertion is normal + <braunr> it says there is a deadlock + <braunr> ss->critical_section_lock must be taken before ss->lock + <gg0> you mean ss->lock before ss->critical_section_lock + <braunr> no + <gg0> ah ok got it + <braunr> that's a bug + <braunr> longjmp + <braunr> ugh + <braunr> you could make a pass through the various uses of those + locks and check what the intended locking protocol should be + <braunr> i inferred ss->critical_section_lock before ss->lock from + hurd_thread_cancel + <braunr> this might be wrong too but considering this function is + used a lot, i doubt it + <gg0> (no, i hadn't got it, i was looking at jmp-unwind.c where + lock is before critical_section_lock) + <gg0> could we get useful info from gdb'ing the assertion? + <tschwinge> gg0: Only if you first get an understanding why it is + happening, what you expect to happen instead/why it shall not + happen/etc. Then you can perhaps use GDB to verify that. + <gg0> i can offer an irc interface if anyone is interested, it's + ready, just to attach :) + <gg0> this is the test + http://sources.debian.net/src/ruby1.9.1/1.9.3.448-1/test/ruby/test_io.rb#L937 + <gg0> pipe function creates two threads + http://sources.debian.net/src/ruby1.9.1/1.9.3.448-1/test/ruby/test_io.rb#L26 + <gg0> Attaching to pid 15552 + <gg0> [New Thread 15552.1] + <gg0> [New Thread 15552.2] + <gg0> (gdb) + + IRC, freenode, #hurd, 2013-09-21: + + <youpi> gg0: it seems the assert (! __spin_lock_locked + (&ss->critical_section_lock)); is bogus + <youpi> but it'd be good to catch a call trace + <youpi> well, it may not be bogus, in case that lock is only ever + taken by the thread itself + <youpi> in that case, inside longjmp_unwind we're not supposed to + have it already + <youpi> ok, that's what we had tried to discuss with Roland + <youpi> it can happen when playing with thread cancelation + <braunr> youpi: the assertion isn't exactly bogus + <braunr> the lock ordering is + <youpi> braunr: which one are you talking about? + <youpi> the one in hurd_thread_cancel looks really wrong + <youpi> and some parts of the code keep the critical section lock + without ss->lock held, so I don't see how lock ordering can help + + IRC, OFTC, #debian-hurd, 2013-09-22: + + <gg0> how much does this patch suck on a scale from 1 to 10? + http://paste.debian.net/plain/44810/ + <youpi> well, the stack allocation issue will go away once I get + the threadvars away + <youpi> I'm working on it right now + <youpi> about the lib paths, it makes sense to add the gnu case, + but i386-gnu shouldn't be put in the path + <gg0> that's great + <gg0> so seems the wrong moment for what i've already done + ie. asking terceiro what he thinks about patch above :/ + <gg0> any distro-independent way to get libc.so and libm.so path? + <gg0> ruby as last resource takes them from "ldd ruby" + <pinotree> gg0: should work fine then + <gg0> well it does. but gnu doesn't have a case so it hits default + which is broken + http://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/40235/entry/test/dl/test_base.rb + <gg0> btw even linux and kfreebsd with debian multipath have broken + cases but they don't hit default and get fixed by ldd later + <pinotree> why it is broken? are arguments passed to that script? + <gg0> i'm not sure about what propose. a broken case so it doesn't + hit default like linux and kfbsd + <gg0> yes they are :/ + <pinotree> and which ones are? who executes that script and which + arguments does it pass to it? + <gg0> other ruby scripts which have nothing to do with libc/libm + <pinotree> well, if they pass arguments which should be the paths + to libc and libm, they must be getting such paths, aren't they? + <gg0> they don't. arguments are other ruby scripts, don't know why, + maybe something else broken before + <gg0> but that would mean that before there's a smarter path + detection way, i doubt + <pinotree> then add the case for hurd, but setting both libc and + libm as nil + <pinotree> so they will be fetched again + <gg0> yep and would really ugly + <gg0> +be + <gg0> "please commit this one which wrongly sets paths." + <gg0> an alternative would be removing default case + <gg0> or pointing it out by proposing ldd in hurd case might make + them review the whole detection + <gg0> by setting correct paths like in patch above it wouldn't + break a possible hurd-amd64, it would work due to ldd + <youpi> gg0: that's why I said the patch is fine, but without the + i386-gnu part of the path + <youpi> just like it happens to be on linux & kfreebsd + <gg0> i might take ldconfig -p output + <gg0> to make it uselessly correct from start + <gg0> http://bugs.ruby-lang.org/issues/8937 + <pinotree> note thar ruby 1.8 is EOL + <pinotree> *that + <gg0> -- If you're reporting a bug in both Ruby 1.9/2.0 and Ruby + 1.8: ruby-trunk, and write like "this bug can be reproduced in + Ruby 1.8 as well." -- + <gg0> i suspect this one won't be the only one i'll file. unless + upcoming youpi's tls and braunr's thread destruction patches fix + all ruby tests + <pinotree> did you check ruby2.0 too, btw? + <gg0> switched to ruby2 few hours ago. i pointed out 2nd part of + testsuite is not enabled, probably terceiro will enable it soon + <gg0> by applying my patch above we'd completely fix current + ruby2.0 build (yes because tests are not completely enabled) + <pinotree> what you run those extra tests? + <gg0> + http://anonscm.debian.org/gitweb/?p=collab-maint/ruby1.9.1.git;a=blob;f=debian/run-test-suites.bash + <gg0> make test + make test-all + <gg0> (test-all is 2nd part) + <gg0> many are problematic. i didn't finish yet to suppress them + one-by-one. one i suppress, another one pops up + <gg0> either get stuck or well known assertion + <pinotree> check those that get stuck :) + <gg0> which kind of check? + <pinotree> "check" as in "debug" + <gg0> btw i tested puppet few days ago (with ruby1.8), it seems to + be working, at least at trasferring files from master + <gg0> don't know about any advanced usage + <pinotree> ruby 1.8 is going to die soon, so testing things against + it is not totally useful + <gg0> so you assume 1.8 is less broken than 1.9/2.0, right? + <pinotree> no + <gg0> i just can see it's been built without tests itself too + <pinotree> erm no + <gg0> well ok, if i can be wrong, i'll be wrong + <gg0> i say that after a quick check time ago, might be wrong + <pinotree> `getbuildlogs ruby1.8 last hurd-i386`, see the last + build log + <gg0> ah from pkg-kde-tools + <gg0> i hate kde :) + <pinotree> no? + <gg0> no what? + <pinotree> devscripts: /usr/bin/getbuildlog + <gg0> pkg-kde-tools: /usr/bin/pkgkde-getbuildlogs + <pinotree> which is not what i said + <gg0> wait that's what apt-file found + <gg0> maybe i should update it + <gg0> is it so recent? + <pinotree> no + <pinotree> i just added an 's' more at the end of the command, but + typing getbu<tab> could have been helpful anyway... + <gg0> yeah just got it + <gg0> my fault not to have tried to run it before looking for it + <pinotree> and btw, i don't see what hating kde has to do with + tools developed by qt/kde debian packagers + <gg0> j/k i simply don't use kde, never used and apt-file search + told me it was from pkg-kde-tools + <gg0> btw build log says "make test" fails, doesn't even start. and + its failure doesn't block the build + <pinotree> exactly + <gg0> s/make test/make test-all/ + <gg0> "make test" (aka "1st part" above) doesn't run. i guess it's + missing in packaging + + IRC, freenode, #hurd, 2013-09-22: + + <braunr> youpi: i mean the lock order where the assertion occurs is + reserved compared to the one in hurd_thread_cancel + <braunr> (and the one in hurd_thread_cancel is the same used in + hurd condition-related functions) + <youpi> "reserved" ? + <braunr> reversed + <braunr> :) + <youpi> by "the assertion occurs", you mean gg0's spot? + <braunr> yes + <youpi> well , the assertion also happens in hurd_thread_cancel + <braunr> it does oO + <braunr> i didn't see that + <youpi> but otherwise yes, it's completely bogus that we have both + locking in different orders + <youpi> could you submit the fix for jmp-unwind.c to upstream? + <braunr> what fix ? + <youpi> reversing the lock order + <braunr> ah, simply that + <youpi> (well, provided that hurd_thread_cancel is right) + <braunr> that's what i suggested to gg0 + <braunr> to check where those locks are held and determine the + right order + * `recvmmsg`/`sendmmsg` (`t/sendmmsg`) From [[!message-id "20120625233206.C000A2C06F@topped-with-meat.com"]], @@ -401,13 +865,140 @@ Last reviewed up to the [[Git mirror's 0323d08657f111267efa47bd448fbf6cd76befe8 Then perhaps the Linux fallback case should be that instead of stubs, too.* + * `SOCK_CLOEXEC` + + IRC, freenode, #hurd, 2013-09-02: + + <gnu_srs1> Do we support accept4 with the SOCK_CLOEXEC flag? + <gnu_srs1> According to the code in sysdeps/mach/hurd/accept4.c + that case is not covered + <gnu_srs1> (only O_NONBLOCK, not SOCK_NONBLOCK??)) + <pinotree> gnu_srs1: we do + <pinotree> but only for accept4, not for socket and socketpair + <gnu_srs1> pinotree: cannot find the case for O_CLOEXEC covered in + __libc_accept4() + <pinotree> gnu_srs1: no, you need SOCK_* + <gnu_srs1> The only code for accept4() is in sysdeps/mach/hurd/ and + it uses O_* for flags ? + <pinotree> flags = sock_to_o_flags (flags); + <pinotree> tried checking it? + <gnu_srs1> Aha, tks:-D + <pinotree> and you don't need an explicit case of O_CLOEXEC, since + it is handled in other ways + + [[!message-id "1378154151.21738.15.camel@G3620.my.own.domain"]]. + + IRC, freenode, #hurd, 2013-09-03: + + <gnu_srs> any ideas about the SOCK_CLOEXEC issue? + <pinotree> didn't i tell already about it? + <gnu_srs> I did not find any hurd related code in tschwinges + branches. + <pinotree> you didn't check deep then... + <gnu_srs> so why does socket/socketpair not return ENOSYS then? + <pinotree> why should it, since they are implemented? + <braunr> ... + <gnu_srs> for socket/socketpair? + <braunr> gnu_srs: enosys means no system call + <gnu_srs> s/ENOSYS/EINVAL/ + <gnu_srs> see the mail to the bug-hurd/debian-hurd ML for more info + <gnu_srs> and tschwinges reply + <pinotree> which is what i knew already? + <gnu_srs> pinotree: please reply on the mailing list on the EINVAL + vs EPROTOTYPE issue to clarify things + <pinotree> gnu_srs: + https://sourceware.org/ml/libc-alpha/2013-02/msg00092.html + <pinotree> gnu_srs: things were clear already... + <gnu_srs> pinotree: I've read that patch and still pflocal/pf.c + returns EPROTOTYPE not changed by the __socket wrapper in eglibc + <pinotree> gnu_srs: what about realizing SOCK_CLOEXEC and friends + are NOT posix? + <gnu_srs> since socket/socketpair does not return EINVAL the dbus + code has to be patched then? + <pinotree> pflocal should never ever get such flags mixed to the + protocol, so any invalid value of protocol correctly returns + EPROTOTYPE + <gnu_srs> this is the question I need answered: Which way to go? + <pinotree> all of them + <gnu_srs> ? + <pinotree> - applications should not assume that because you have + accept4 (which is not posix) then SOCK_CLOEXEC and SOCK_NONBLOCK + (flags for it) are usable to socket and socketpair + <pinotree> - glibc should (as the idea of my patch) handle + implementations providing SOCK_* but not supporting them for + socket/socketpair + <pinotree> - finally the hurd part of glibc could implement them + <gnu_srs> to conclude: should I send a bug report for dbus then? + <gnu_srs> pinotree: yes or no? + <pinotree> gnu_srs: *shrug* i wrote it above, so an *upstream* + report (not a debian one) + + IRC, freenode, #hurd, 2013-09-06: + + <gnu_srs> I've found another error code issue, now in glib2.0 (and + dbus). Are you really sure the error code + <gnu_srs> for protocol of pflocal/pf.c should be + EPROTONOSUPPORT. The code expects EINVAL for a protocol with + <gnu_srs> SOCK_CLOEXEC, which is a flag. Maybe pf.c should add + this case and return EINVAL instead of + <gnu_srs> submitting bug reports upstream. Yes, I know this is not + POSIX, but it is defined for Hurd too, + <gnu_srs> currently only supported for accept4, not socket or + socketpair. + <pinotree> gnu_srs: no, and i explained already why it is wrong + this way + <pinotree> pflocal shouldn't even get such flags + <pinotree> (pflocal or any other server implementing socket_create) + <gnu_srs> (20:19:35) pinotree: pflocal shouldn't even get such + flags + <gnu_srs> then the glibc wrapper code is missing to catch this + flag:( + <gnu_srs> youpi: ? + <pinotree> gnu_srs: because, as told many times, socket and + socketpair do not support such flags + <pinotree> given they don't do, they filter nothing + <pinotree> and no, you need to file bugs upstream, since having + SOCK_* and accept4 does not imply at all that socket and + socketpair support them + + IRC, freenode, #hurd, 2013-09-07: + + <gnu_srs> A correction from yesterdays discussion: + s/EPROTONOSUPPORT/EPROTOTYPE + + IRC, freenode, #hurd, 2013-09-10: + + <gnu_srs> for dbus2.0 I found out that the third SOCK_CLOEXEC case + needs a patch too (more working tests), + <gnu_srs> the updated patch is at http://paste.debian.net/37948/ if + you have the time, otherwise I'll do it. + <pinotree> gnu_srs: which is what i wrote in my bug report... + <gnu_srs> Yes you wrote that, but the patch is not updated yet? + <pinotree> it refers to a different socket access, recently added, + which is not used by default + <gnu_srs> I got two more tests running when adding that patch:-/ + <pinotree> tests of what? + <gnu_srs> run-test.sh and run-test-systemserver.sh:P + <pinotree> tests of what? + <pinotree> i don't have the universal knowledge of the files in all + the sources + <gnu_srs> dbus-1.6.14/test/name-test/* + + [[!message-id "523A3D6C.2030200@gmx.de"]]. + + IRC, OFTC, #debian-hurd, 2013-09-19: + + <pinotree> tschwinge: ehm, regarding the SOCK_* patches for + socket/socketpair, didn't we talk about them when i worked on + eglibc 2.17? + For specific packages: * [[octave]] * Create `t/cleanup_kernel-features.h`. - * Add tests from Linux kernel commit messages for `t/dup3` et al. + * [[Secure_file_descriptor_handling]]. * In `sysdeps/unix/sysv/linux/Makefile`, there are a bunch of `-DHAVE_SENDFILE` -- but we do have `sendfile`, too. @@ -927,6 +1518,31 @@ Last reviewed up to the [[Git mirror's 0323d08657f111267efa47bd448fbf6cd76befe8 <tschwinge> Yeah, that's known for years... :-D <tschwinge> Probably not too difficult to resolve, though. + * IRC, OFTC, #debian-hurd, 2013-08-16: + + <pinotree> http://paste.debian.net/25934/ ← _hurd_thread_sigstate calls + malloc, boom + + * `conformtest` + + IRC, OFTC, #debian-hurd, 2013-09-22: + + <youpi> btw, I noticed that glibc has a head conformance test which we + happily fail quite a bit :) + <youpi> it's not so awful, we don't have twice as many failures as + linux, but not so far + <pinotree> youpi: do you mean "header" for "head", right? + <youpi> err, where ? :) + <pinotree> <youpi> btw, I noticed that glibc has a head conformance + test which we happily fail quite a bit :) + <youpi> ah, yes + <pinotree> noticed that too + <youpi> I had a quick look at the POSIX part, some things are probably + not too hard to change (e.g. exposing pthread_kill in signal.h) + <youpi> others will by quite hard to fix (short type instead of int + type for some flock structure field) + <youpi> s/by/be/ + * Verify baseline changes, if we need any follow-up changes: * a11ec63713ea3903c482dc907a108be404191a02 @@ -1253,6 +1869,13 @@ TODO. [[!message-id "20120723195143.7F8142C0B9@topped-with-meat.com"]]. + IRC, freenode, #hurd, 2013-08-27: + + < gnu_srs> Hi, is this fixed by now? + < gnu_srs> ../hurd/hurd.h:72:5: warning: case value ‘0’ not in + enumerated type ‘error_t’ [-Wswitch] + < pinotree> nope + * baseline fd5bdc0924e0cfd1688b632068c1b26f3b0c88da..2ba92745c36eb3c3f3af0ce1b0aebd255c63a13b (or probably Samuel's mmap backport) introduces: @@ -1389,6 +2012,23 @@ Failures, mostly in order of appearance: Due to `ext2fs --no-atime`. + * IRC, OFTC, #debian-hurd, 2013-05-08 + + <youpi> bah, tst-atime failure :) + <pinotree> do you have its output? + <youpi> well it's very simple + <youpi> I have the noatime option on / :) + <pinotree> oh + <youpi> fortunately fsysopts works :) + <pinotree> the test checks whether ST_NOATIME is in the mount + options, maybe it would be a good idea to provide it + <youpi> yes + <pinotree> unfortunately it isn't in posix, so i'm not sure whether + adding it to the general bits/statvfs.h would be welcome + <pinotree> or whether we should fork it, like it is done for linux + <pinotree> oh no, we fork it already + <pinotree> \o/ + `dirent/tst-fdopendir.out`: directory atime changed diff --git a/open_issues/glibc/0.4.mdwn b/open_issues/glibc/0.4.mdwn index ceb5ea21..f864469d 100644 --- a/open_issues/glibc/0.4.mdwn +++ b/open_issues/glibc/0.4.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,6 +10,8 @@ License|/fdl]]."]]"""]] [[!tag open_issue_glibc open_issue_libpthread]] +Things to consider doing when bumping the glibc SONAME. + # IRC, freenode, #hurd, 2012-12-14 diff --git a/open_issues/glibc/debian.mdwn b/open_issues/glibc/debian.mdwn index 331632f3..2ef2c474 100644 --- a/open_issues/glibc/debian.mdwn +++ b/open_issues/glibc/debian.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -24,6 +24,43 @@ locale stuff. `--disable-compatible-utmp`? +## IRC, freenode, #hurd, 2013-08-28 + + <youpi> uh, the i686 profiles have much more progression than i386 + <youpi> it seems they don't actually run these + <pinotree> youpi: what do you mean with "we don't run those"? + <pinotree> iirc there are three build profiles done, but there are 4 + regression test files + <youpi> yes, but some failing tests are not run in the three build profiles + <youpi> even if they are built for all of them + <pinotree> not even run? which ones? + <youpi> see for instance test-ifloat.out + <youpi> test-ifloat is built in all profiles, but only run in the libc one + <pinotree> don't have a glibc built tree around atm, sorry :/ + <youpi> perhaps because glibc thinks it's not useful to run it again if it + fails on i386 + <youpi> you can check the logs + <pinotree> do you think glibc's build system is that smart? :) + <pinotree> all the builds are done in separate builddirs, so theorically + they should not touch each other... + <youpi> yes + <youpi> that's why I'm surprised + <pinotree> could it be they get not run in optimized/particular builds? + <pinotree> what about linux/kfreebsd i386? + <youpi> I don't see what makes them not run + <youpi> or at least be treated particularly by th eMakefile + <youpi> not run on kfreebsd either + <youpi> pinotree: also, most of the tests now working have been marked as + failing by your patches for 2.17, would it be possible to retry them on + the box you used at that time? + <pinotree> that's the vm on my machine + <youpi> which kind of vm? + <youpi> kvm? + <pinotree> y + <youpi> they are working here + <youpi> with kvm + + # Building Run `debian/rules patch` to apply patches (instead of having it done during the @@ -62,3 +99,70 @@ apter applying patches. If the Debian symbol versioning file is not up to date and the build of Debian packages fails due to this, putting `DPKG_GENSYMBOLS_CHECK_LEVEL=0` in the environment \`\`helps''; see `man dpkg-gensymbols`. + + +# IRC, freenode, #hurd, 2013-07-01 + + <braunr> something seems to have changed with regard to patch handling in + eglibc 2.17 + <braunr> pinotree: when i add a patch to series and use dpkg-buildpackage, + i'm told there are local modifications and the build stops :/ + <braunr> any idea what i'm doing wrong ? + <pinotree> which steps do you do? + <braunr> i extract the sources, copy the patch to debian/patches/hurd-i386, + add the appropriate line to debian/patches/series, call dch -i, then + dpkg-buildpackage + <pinotree> eglibc is a "3.0 (quilt)" format source package + <pinotree> this means its default patches are in a quilt-style system, and + they are applied on extraction + <braunr> ok + <braunr> and it can't detect new patches ? + <pinotree> so if you add a new patch to the global serie, you have to push + it manually + <braunr> i have to revert them all ? + <braunr> ok + <braunr> how do i do that ? + <pinotree> quilt push -a + <braunr> ok + <braunr> thanks + <pinotree> remember to do that before starting the build, since the rest + assumes the quilt-style patches are fully applied + <bddebian> No push applies them, quilt pop -a reverts them + <pinotree> yeah, and he has to push the new over the dpkg-applied ones + <bddebian> Oh, aye + <braunr> does quilt change series ? + <pinotree> no + <braunr> ok + <pinotree> i mean, some commands do that + <braunr> so i do everything i did, with an additional push, right ? + <pinotree> ok, screw me, i didn't get your question above :P + <braunr> does that change your answer ? + <pinotree> <braunr> does quilt change series ? + <braunr> yes + <pinotree> if you import or create a new patch, it changes series indeed + <braunr> ok + <pinotree> push or pop of patches does not + <braunr> i'm doing it wron + <braunr> g + <pinotree> btw, in a quilt patch stack you can easily import a new patch + using the import command + <pinotree> so for example you could do + <pinotree> apt-get source eglibc # or get it somehow else + <pinotree> cd eglibc-* + <pinotree> quilt import /location/of/my/patch + <pinotree> quilt push # now your patch is applied + <braunr> ah thanks + <pinotree> dpkg-buildpackage as usual + <braunr> that's what i was looking for + <bddebian> quilt new adds a new entry in series + <pinotree> y + <bddebian> or import, aye + <pinotree> braunr: if you want to learn quilt, a very good doc is its own, + eg /usr/share/doc/quilt/quilt.txt.gz + * bddebian has never actually used import + <braunr> ok + <pinotree> it is basically a simple stack of patches + + <youpi> braunr: yes, patch handling is a bit different + <youpi> the arch-independant patches are applied by dpkg-source -x + <youpi> and the arch-dependent patches are applied during build diff --git a/open_issues/glibc/debian/experimental.mdwn b/open_issues/glibc/debian/experimental.mdwn index 8d117e99..5168479d 100644 --- a/open_issues/glibc/debian/experimental.mdwn +++ b/open_issues/glibc/debian/experimental.mdwn @@ -11,6 +11,7 @@ License|/fdl]]."]]"""]] [[!tag open_issue_glibc]] Issues with the current 2.17 version of glibc/EGLIBC in Debian experimental. +Now in unstable. # IRC, OFTC, #debian-hurd, 2013-03-14 @@ -113,3 +114,62 @@ Issues with the current 2.17 version of glibc/EGLIBC in Debian experimental. eventually? (Some experimental package(s), but which?) <youpi> that was libc0.3 packages <youpi> which are indeed known to break the network + + +# IRC, freenode, #hurd, 2013-06-18 + + <braunr> root@darnassus:~# dpkg-reconfigure locales + <braunr> Generating locales (this might take a + while)... en_US.UTF-8...Segmentation fault + <braunr> is it known ? + <youpi> uh, no + + +## IRC, OFTC, #debian-hurd, 2013-06-19 + + <pinotree> btw i saw too the segmentation fault when generating locales + + +# IRC, OFTC, #debian-hurd, 2013-06-20 + + <youpi> damn + <youpi> hang at ext2fs boot + <youpi> static linking issue, clearly + + +## IRC, freenode, #hurd, 2013-06-30 + + <youpi> Mmm + <youpi> __access ("/etc/ld.so.nohwcap", F_OK) at startup of ext2fs + <youpi> deemed to fail.... + <pinotree> when does that happen? + <youpi> at hwcap initialization + <youpi> at least that's were ext2fs.static linked against libc 2.17 hangs + at startup + <youpi> and this is indeed a very good culprit :) + <pinotree> ah, a debian patch + <youpi> does anybody know a quick way to know whether one is the / ext2fs ? + :) + <pinotree> isn't the root fs given a special port? + <youpi> I was thinking about something like this, yes + <youpi> ok, boots + <youpi> I'll build a 8~0 that includes the fix + <youpi> so people can easily build the hurd package + <youpi> Mmm, no, the bootstrap port is also NULL for normally-started + processes :/ + <youpi> I don't understand why + <youpi> ah, only translators get a bootstrap port :/ + <youpi> perhaps CRDIR then + <youpi> (which makes a lot of sense) + + +## IRC, freenode, #hurd, 2013-07-01 + + <braunr> youpi: what is local-no-bootstrap-fs-access.diff supposed to fix ? + <youpi> ext2fs.static linked againt debian glibc 2.17 + <youpi> well, as long as you don't build & use ext2fs.static with it... + <braunr> that's thing, i want to :) + <braunr> +the + <youpi> I'd warmly welcome a way to detect whether being the / translator + process btw + <youpi> it seems far from trivial diff --git a/open_issues/glibc/t/tls-threadvar.mdwn b/open_issues/glibc/t/tls-threadvar.mdwn index 105a07c7..7ce36f41 100644 --- a/open_issues/glibc/t/tls-threadvar.mdwn +++ b/open_issues/glibc/t/tls-threadvar.mdwn @@ -64,3 +64,55 @@ dropped altogether, and `__thread` directly be used in glibc. <youpi> I saw the mails, but didn't investigate at all [[!message-id "878vdyqht3.fsf@kepler.schwinge.homeip.net"]]. + + +# IRC, freenode, #hurd, 2013-07-08 + + <youpi> tschwinge: apparently there were a lot of changes missing in the + threadvars branch I had commited long time ago + <youpi> I'm gathering things + <tschwinge> youpi: t/tls-threadvar you mean? + <youpi> yes + <youpi> tschwinge: yes, there were a lot other occurences of threadvars + stuff in various places + <youpi> I'm building libc again, and will see what issue would remain + + +## IRC, freenode, #hurd, 2013-07-12 + + <youpi> braunr: about the per-thread ports, there is also the mig reply + port + <youpi> (stored in _HURD_THREADVAR_MIG_REPLY) + + +## IRC, freenode, #hurd, 2013-07-15 + + <braunr> and with the branch youpi pushed where he removes threadvars, it + shouldn't get "too" hard + <braunr> (save for the tricky bugs you may encounter) + <youpi> well, that branch is not working yet + + +## IRC, OFTC, #debian-hurd, 2013-09-22 + + <youpi> I'm currently tracking bugs with my threadvars changes + <youpi> some of them seem fine, others, not + <youpi> of course the most complex ones are the most probable culprits for + the issues I'm getting + <youpi> fortunately they're after the process bootstrap + <youpi> so basically that works + <youpi> just a few dozen tests fail + <youpi> mostly about loading .so or raising signals + <youpi> dlopen("bug-dlsym1-lib1.so"): bug-dlsym1-lib1.so: cannot open + shared object file: Function not implemented + <youpi> after having changed errno a bit + <youpi> doesn't that look odd ? :) + <youpi> good, I found an issue with the sigstate + <youpi> now running testsuite again, to see whether there are other issues + with it :) + <youpi> s/sigstate/reply_port/ actually + + +## IRC, OFTC, #debian-hurd, 2013-09-23 + + <youpi> yay, errno threadvar conversion success diff --git a/open_issues/glibc/t/tls.mdwn b/open_issues/glibc/t/tls.mdwn index 68db2cc1..a92a21fb 100644 --- a/open_issues/glibc/t/tls.mdwn +++ b/open_issues/glibc/t/tls.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2012, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -14,7 +15,8 @@ License|/fdl]]."]]"""]] * Discuss d2431f633e6139a62e1575ec18830f7e81160cf0 with Samuel. - * `TLS_INIT_TP_EXPENSIVE` is unused; Hurd def. can be removed. + * Validate our implementation against + <https://sourceware.org/glibc/wiki/TLSandSignals>. # Documentation diff --git a/open_issues/gnumach_integer_overflow.mdwn b/open_issues/gnumach_integer_overflow.mdwn index 2166e591..08a29268 100644 --- a/open_issues/gnumach_integer_overflow.mdwn +++ b/open_issues/gnumach_integer_overflow.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -15,3 +15,36 @@ License|/fdl]]."]]"""]] <braunr> yes, we have integer overflows on resident_page_count, but luckily, the member is rarely used + +See also [[gnumach_vm_object_resident_page_count]]. + + +## IRC, freenode, #hurd, 2013-06-04 + + <elmig> this is declared as int on vm_object.h + <elmig> and as it as counter it's always positive + <braunr> yes it should be unsigned + <elmig> ok + <braunr> but leave it as it is for consistency with the rest + <elmig> i send patch :) + <braunr> please no + <braunr> unless you've fully determined the side effects + <elmig> i've grepped the vars and saw only comparisons > and = 0 + <elmig> never less than 0 + <braunr> > 0 is the same + <braunr> well + <braunr> > not, but >= would be a problem + <elmig> http://paste.debian.net/plain/8527 + <elmig> asctually no >=0 + <braunr> still, i don't want to change that unless it's strictly necessary + <braunr> hum, you're grepping ref_count, not resident_page_count + <elmig> i did both + <elmig> on resident_page_count theres resident_page_count >= 0 + <elmig> = 0, == 0 + <braunr> this isn't the only possible issue + <braunr> anyway + <braunr> for now there is no reason to change anything unless you do a full + review + <elmig> only place i see resdent_page_count and page_count being decreased + it's on vm/vm_resident.c + <elmig> vm_page_remove() and vm_page_replace() diff --git a/open_issues/gnumach_vm_object_resident_page_count.mdwn b/open_issues/gnumach_vm_object_resident_page_count.mdwn index 9b16fc1d..2ffe5753 100644 --- a/open_issues/gnumach_vm_object_resident_page_count.mdwn +++ b/open_issues/gnumach_vm_object_resident_page_count.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -22,5 +22,33 @@ License|/fdl]]."]]"""]] `vm/vm_object.h:vm_object:resident_page_count`. +## IRC, freenode, #hurd, 2013-06-03 + + <elmig> regarding + https://www.gnu.org/software/hurd/open_issues/gnumach_vm_object_resident_page_count.html, + this is fixed. it's an int. what should happen do this page? /dev/null + <elmig> ? + <youpi> I guess so + + +## IRC, freenode, #hurd, 2013-06-04 + + <elmig> + http://darnassus.sceen.net/~hurd-web/open_issues/gnumach_vm_object_resident_page_count/ + <elmig> this is a int + <elmig> how to deal with the page? delete it? archive it? + <braunr> ? + <elmig> the issue originallu reported was fixed, right? + <braunr> i think so, yes + <braunr> for now at least + <elmig> so this stays on the open_issues on the wiki anyway? + <braunr> no, it should go away + <elmig> i dont know how to suggest deletion on the wiki + <braunr> don't + <braunr> i'll do it later + + +## 2013-06-04 + resident_page_count it's now an int. The topic it's fixed. diff --git a/open_issues/hurd_init.mdwn b/open_issues/hurd_init.mdwn new file mode 100644 index 00000000..b0b58a70 --- /dev/null +++ b/open_issues/hurd_init.mdwn @@ -0,0 +1,216 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + + +# [[!message-id "20130625154749.17799.36923@thinkbox.jade-hamburg.de"]] + + +## IRC, freenode, #hurd, 2013-07-22 + + <teythoon> ok, so back to the drawing board for the next big issue, the + potential proc and init merge + <teythoon> Roland had some harsh words for that proposal, but noone else + raised concerns + <youpi> noone else does not mean much + <youpi> I guess only Roland actually understands the matter + <youpi> so I'd tend to believe him + <teythoon> even though, his criticism was so superficial, he could at least + be a bit more specific... + <braunr> i agree that the argument, being simply based on vague principle, + isn't very convincing + <teythoon> so, what should I do? + <braunr> you can either keep them separate, or fight with roland + <teythoon> common braunr, I need a little more guidance in these kind of + social issues + <teythoon> a statement like this is of little use ;) + <braunr> that's the best i can give you + <teythoon> :/ + <braunr> i have one patch "fixing" HZ on the hurd, and i even get to fight + about it + <teythoon> I understand Roland has been around forever and keeps an eye on + stuff + <teythoon> but could/would he block a patch for hurd if e.g. youpi would + accept it + <teythoon> i.e. how much control has he in practice? + <teythoon> me fighting with him over a patch is of little value for anyone + and I don't care to do so + <braunr> not much i suppose now + <braunr> but we also have to agree with the change + <braunr> with *real* arguments + <braunr> (well, if it was up to me, i'd even merge exec with proc so ..) + <teythoon> ok, so I whip up a patch to see how it goes in practice and + present it so we could talk about the issue with something to look at + first + <braunr> although maybe not ;p + <braunr> you'll hit the same reaction + <teythoon> from Roland? + <braunr> yes + <braunr> and youpi said he tends to trust what roland says + <braunr> so let's discuss the pros and cons a bit more + <teythoon> yes, but I'd honor his concerns if they were properly + presented. just telling me to hack on linux instead even though I think I + have demonstrated that I do want to work on Hurd is so childish in my + eyes that I do not consider that a valid argument at the moment + <teythoon> sure, shoot + <braunr> well, functionally, they're unrelated + <teythoon> head -n1 init/init.c + <teythoon> /* Start and maintain hurd core servers and system run state + <youpi> and thus it makes sense to make them separate, even if it does not + seem to bring anything useful now + <youpi> history has shown that it makes a bed for nice things later + <braunr> teythoon: that's not what proc is about + <teythoon> braunr: I know + <teythoon> braunr: that's what init is about in its own words ;) + <youpi> teythoon: also, "simplifying the code" is not necessarily an + argument that would be considered + <youpi> depending on the simplification + <youpi> linux made it all simple by using a monolithic kernel :) + <youpi> separating concerns is complex + <youpi> but in the end it usually pays off on the Hurd + <youpi> personally, I'd be fine with Guillem's solution, and renumbering + init's pid in Debian + <youpi> there's a pending question from Roland actually: what information + is exchanged between init and proc in the end? + <youpi> that's actually the point of the discussion: is that information + really big or not + <teythoon> I'm sorry, you lost me, where did he ask that question? + <pinotree> $ git grep proc_getmsgport | egrep '[0-9]' ← /hurd/init as pid 1 + is hardcoded in few places + <youpi> teythoon: he didn't ask it this way, but that's the question I had + to be able to answer his + <youpi> Date: Mon, 15 Jul 2013 10:36:35 -0700 (PDT) + <youpi> > That's not what he said. He said there is a lot of information + <youpi> > propagated from init to proc, and thus the separation is + questionable. + <youpi> Are you talking about bootstrap, or what? + <youpi> as I haven't investigated much, I couldn't answer this + <youpi> pinotree: right. We could patch these in Debian + <teythoon> youpi: so, shall I refresh, test and refine Guillems patch and + resend it? + <youpi> it's probably an easier way + <teythoon> ok, I start by doing that + + +## IRC, freenode, #hurd, 2013-07-25 + + <teythoon> pinotree: btw, there are two /sbin/init processes even with my + hacked up init/proc variant where /sbin/init gets to be pid 1 + <pinotree> never seen that + <pinotree> what are their parents? + <teythoon> pinotree: well, pid 1 is /sbin/init now, pid 13 or something has + the parent 1 + <teythoon> looks like init forks or something + <pinotree> i guess your sysvinit is compiled without INITDEBUG? + <pinotree> nothing in syslog either? + <teythoon> pinotree: it's compiled like the sysvinit shipped with debian + <pinotree> teythoon: do you have custom additions in inittab? + <teythoon> pinotree: a terminal for my serial console + <teythoon> *getty + <pinotree> are the getty started correctly for you, btw? + <teythoon> pinotree: yes + <pinotree> interesting + <pinotree> teythoon: back then, they were costantly respawning, with hurd's + getty's failing to start when exec'ed by (sysv)init + <pinotree> wonder what changed + <teythoon> pinotree: cool, magically went away then :) + + +## IRC, freenode, #hurd, 2013-07-29 + + <teythoon> youpi: I need some feedback on the not freezing translators + issue, more specifically whether I understood you correctly in your mail + from wednesday (20130724131552.GG9576@type.bordeaux.inria.fr) + <teythoon> oh yeah, and I had some questions yesterday too, about rpctrace + and dead-name notifications, specifically why /hurd/init is not receiving + any for the root translator and the exec server + <braunr> teythoon: more details please + <teythoon> ok, so /hurd/init is registering for dead name notifications for + essential tasks + <teythoon> the rootfs and exec both register as essential tasks at init and + init requests successfully dead name notifications for them + <teythoon> if you e.g. kill the auth server, /hurd/init will notice and + crash the system + <teythoon> if you kill exec or the rootfs, /hurd/init does not get notified + <teythoon> I verified this with gdb and an subhurd + <teythoon> I'm puzzled by this, as the kernel is the one who sends the + notifications, right? + <braunr> yes + <braunr> teythoon: where is the problem ? + <teythoon> and it is not that the system is not sending any messages, it + is, I see the msgcount increase over time + <teythoon> braunr: dunno, as far as I can tell the kernel does not deliver + the notification for rootfs and exec + <braunr> oh + <teythoon> those are the two processes loaded by grub, maybe they are + different somehow + <braunr> is that affecting your work ? + <teythoon> no, not directly, I strayed around at the weekend, trying to + think of cool stuff hurd could do + <teythoon> youpi: I need some feedback on the not freezing translators + issue, more specifically whether I understood you correctly in your mail + from wednesday (20130724131552.GG9576@type.bordeaux.inria.fr) + <youpi> teythoon: ok, now I'm available for the not-freezing-translators + thing :) + + +## IRC, freenode, #hurd, 2013-08-05 + + <teythoon> youpi: I'm in the process of producing a unified + sysvinit-as-pid1 and please-dont-kill-important-processes patch series + <teythoon> youpi: there is one issue with changing /hurd/inits pid, libcs + reboot() also assumes that it has the pid 1 + <youpi> argl + <youpi> that's bad, because it's then an ABI, not just an internal thing + <teythoon> hardcoding the pid is the worst way of getting a handle of any + server :/ + <teythoon> I've been thinking to make it explicit by binding it to + /servers/startup or something + <youpi> that would be more hurdish than using a pid, yes + <teythoon> yes, and not only does it break the abi, but in a bad way + too. if the libc is updated before the hurd, the shutdown sequence is + broken in a way that the translators aren't synced :/ + <teythoon> youpi: as a workaround, we could make reboot() signal both pid 1 + and 2 + <youpi> at worse pid 1 shouldn't get harmed by receiving a startup_reboot + RPC indeed + <teythoon> yes + + +## IRC, freenode, #hurd, 2013-08-16 + + <teythoon> grml, the procfs hardcodes the kernels pid :/ + <teythoon> there's always one more thing to fix... + <teythoon> uh, and we made pids.h a private header, so no nice constant for + the procfs translator :/ + <teythoon> server lookup by hardcoding the pid should be banned... + + +## IRC, freenode, #hurd, 2013-09-16 + + <teythoon> youpi: I'm thinking about splitting /hurd/init into /hurd/init + and /hurd/startup + <teythoon> that way, you could also merge the init as pid1 patches + <teythoon> that should be doable within the week + <youpi> that would probably be better received by Roland than merging init + into proc :) + <teythoon> yes, I suppose so :D + <youpi> perhaps you should start the discussion on the list about it + already, with just a sketch of which would do what + <teythoon> ok + <teythoon> fwiw I like the name startup b/c it speaks the startup protocol + <braunr> teythoon: +1 startup + + +## IRC, freenode, #hurd, 2013-09-23 + + <teythoon> I've been hacking on init/startup, I've looked into cleaning it + up diff --git a/open_issues/libc_variant_selection.mdwn b/open_issues/libc_variant_selection.mdwn index afcd9ae0..71370a43 100644 --- a/open_issues/libc_variant_selection.mdwn +++ b/open_issues/libc_variant_selection.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -28,6 +29,28 @@ On Thu, Oct 07, 2010 at 11:22:46AM +0200, Samuel Thibault wrote: > Yes, you need to copy it by hand. Same for libc0.3-i686, we just need to > steal the cpuid code from the kfreebsd port of glibc. + +# IRC, freenode, #hurd, 2013-06-30 + + <pinotree> other than that, the hwcap system is not working for us yet, + right? + <youpi> no but we'd like to use e.g. cpuid for that + <youpi> like kfreebsd does + <pinotree> do they use cpuid for that? + <pinotree> i kind of lost myself in glibc's loading internals, trying to + find out where the hwcap bits come from + <youpi> on linux it comes from the kernel + <youpi> on kfreebsd aiui they use cpuid to figure it out from the process + itself + <pinotree> do you have any pointer to the kfreebsd way? iirc i had a look + in their sysdeps, but found nothing related to that + <youpi> it's in local-sysdeps.diff aiui + <youpi> +dl_platform_kfreebsd_i386_init + <youpi> which fills dl_hwcap + <youpi> called at _dl_sysdep_start + <pinotree> interesting + + --- Having working CPUID code inside [[glibc]] is also a prerequisite for proper diff --git a/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn b/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn index 670c82cb..11bebd6e 100644 --- a/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn +++ b/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn @@ -133,3 +133,29 @@ License|/fdl]]."]]"""]] <braunr> (imo, the mach_debug interface should be adjusted to be used with privileged ports only) <braunr> (well, maybe not all mach_debug RPCs) + + +# `gnumach.defs` + +[[!message-id +"CAEvUa7nd2LSUsMG9axCx5FeaD1aBvNxE4JMBe95b9hbpdqiLdw@mail.gmail.com"]]. + + +## IRC, freenode, #hurd, 2013-05-13 + + <braunr> youpi: what's the point of the last commit in the upstream hurd + repository (utils/vmstat: Use gnumach.defs from gnumach) ? + <braunr> or rather, i think i see the point, but then why do it only for + gnumach and not fot the rest ? + <braunr> for* + <youpi> most probably because nobody did it, probably + <braunr> aiui, it makes the hurd build process not rely on system headers + <youpi> (and nobody had any issue with it) + <braunr> well yes, that's why i'm wondering :) + <braunr> it looks perfectly fine to me to use system headers instead of + generating them + <youpi> ah right + <youpi> I thought there was actually a reason + <youpi> I'll revert + <youpi> could you answer David about it? + <braunr> sure diff --git a/open_issues/libnetfs_passive_translators.mdwn b/open_issues/libnetfs_passive_translators.mdwn new file mode 100644 index 00000000..c89d27f7 --- /dev/null +++ b/open_issues/libnetfs_passive_translators.mdwn @@ -0,0 +1,55 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + + +# IRC, freenode, #hurd, 2013-07-15 + + <teythoon> is there a libnetfs based translator that supports passive + translators? + <youpi> I don't see any at the top of my head, only with active ones such + as hostmux + <teythoon> I suspected as much since as far as I can tell libnetfs lacks + some bits to make that even work + <braunr> teythoon: the problem with passive translators is persistence + <braunr> well, it's easy to store volatile passive translators in a + libnetfs server + <braunr> but usually, there is no backing store for them + <braunr> ext2fs is the only one actually providing space to store their + command line + <teythoon> sure, but at least file_get_translator needs to work so that + procfs can serve a mounts node + <braunr> silly idea but + <braunr> don't you want to directly add it to the procfs translator ? + <teythoon> no, I think it's useful on its own + <braunr> ok + <braunr> then you may need to add the required support + <teythoon> it even doubles as normal command line tool + <teythoon> yes, I almost got it... or so I hope ;) + <braunr> ok + <teythoon> also, netfs_get_translator exists, so not supporting that feels + like a bug to me + <teythoon> could also be useful for a potential devfs translator + <braunr> yes + + <teythoon> uh, the code duplication in lib*fs is really bad :/ + <teythoon> the code is mostly similar, though they have diverged and many + little things are different so diffing them is very noisy + <teythoon> stuff like file names or identifiers + <teythoon> and I cannot figure out why my shiny passive translators are not + started :/ + <teythoon> % showtrans tmp/mounts + <teythoon> /hurd/mtab.fixed / + <teythoon> % wc --bytes tmp/mounts + <teythoon> 0 tmp/mounts + <teythoon> and no mtab translator around either + +[[hurd/translator/mtab/discussion]]. diff --git a/open_issues/libnetfs_vs_libdiskfs.mdwn b/open_issues/libnetfs_vs_libdiskfs.mdwn new file mode 100644 index 00000000..2fcfbea5 --- /dev/null +++ b/open_issues/libnetfs_vs_libdiskfs.mdwn @@ -0,0 +1,118 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +[[!toc]] + + +# Argument Parsing + +## IRC, freenode, #hurd, 2013-06-27 + + <teythoon> the arg parsing in libdiskfs and libnetfs differ :/ + <teythoon> afaics libdiskfs gets it right, libnetfs does not + <pinotree> what do you mean? + <teythoon> wrt to *_std_{runtime,startup}_argp + <teythoon> see eg netfs.h + <teythoon> libdiskfs/opts-std-runtime.c:const struct argp + diskfs_std_runtime_argp = + <teythoon> libdiskfs/opts-std-runtime.c-{ + <teythoon> libdiskfs/opts-std-runtime.c- std_runtime_options, parse_opt, + 0, 0, children + <teythoon> libdiskfs/opts-std-runtime.c-}; + <teythoon> but + <teythoon> libnetfs/std-runtime-argp.c:const struct argp + netfs_std_runtime_argp = { 0 }; + <pinotree> well there are no common startup/runtime options provided by + netfs + <pinotree> usually netfs-based translators put netfs_std_startup_argp as + child as their options, so if netfs starts providing options they would + work + <teythoon> ah + <pinotree> if you have a test showing issues, we can certainly look it :) + <teythoon> ok, m/b I was confused... + <pinotree> no worries, feel free to ask anytime + <teythoon> I thought about providing --update as common runtime flag, like + diskfs does, you think it's the right thing to do? + <pinotree> what would it do? + <teythoon> or should it be left for each translator to implement? + <teythoon> nothing by default I guess + <pinotree> options provided in libdiskfs are implemented and handled mostly + in libdiskfs itself + <pinotree> so imho a new option for libnetfs would be there because its + behaviour is implemented mostly within libnetfs itself + <teythoon> libdiskfs calls diskfs_reload_global_state + <teythoon> libnetfs could do the same, allowing translators to plug in + anything they wish + <teythoon> but I'll implement it in procfs for the time being + <pinotree> ah, its alias is remount + <teythoon> yes + <teythoon> I need that working for procfs + <teythoon> btw, I think I got your mount confusion thing figured out + <pinotree> but procfs has nothing to update/flush, all the information are + fetched at every rpc + <teythoon> yes + <teythoon> but we still need to ignore the flag + <teythoon> otherwise the set_options rpc fails + <teythoon> http://paste.debian.net/12938/ + <teythoon> whee, remounting proc works :) + <braunr> :) + + +# IRC, freenode, #hurd, 2013-07-29 + + <teythoon> so, what do you folks think about refactoring libdiskfs and + libnetfs to be more alike? + <pinotree> what do you mean? + <teythoon> ah, I mentioned that in the context of my mtab prototype + 1374247519-26589-1-git-send-email-4winter@informatik.uni-hamburg.de + <teythoon> they are hard to diff against each other b/c they differ in file + names and identifier names + <teythoon> while working on the mtab stuff I encountered stuff that was + implemented in libdiskfs, but never in libnetfs + <teythoon> mostly support for binding translators to libnetfs nodes + <braunr> teythoon: sure, but looks a little out of scope + <teythoon> braunr: I do not mean now, more in general + <braunr> ok + <tschwinge> teythoon: I wondered about this, too. I don't know if it's + possible to literally merge them (and build the backend-based (libdiskfs) + vs. volatile-backend one (libnetfs) based on a pre-processor define or + similar), or just structure the source code (files) in a way such that + »diff -ru libdiskfs/ libnetfs/« gives meaningful results, figuratively + spoken. + <teythoon> tschwinge: my thoughts exactly + + +# IRC, freenode, #hurd, 2013-08-28 + + <teythoon> braunr: do you think another lib*fs would be frowned uppon? I + like the way procfs is structured and that could be refactored and + generalized into a library + <braunr> i think we need more lib*fs libraries + <braunr> and better integration + <braunr> that's one of the strengths in linux + <braunr> it makes writing file systems very easy + <teythoon> cool :) + <teythoon> now we only need a snappy name, any suggestions? + <braunr> i don't know what you like specificlaly in procfs + <braunr> libpseudofs ? + <teythoon> well, it's not perfect, but i like the way you just have to + implement a function for a node and it magically gains the ability to + being read + <teythoon> for example + <braunr> yes i see + <pinotree> lacks a bit of caching though + <braunr> no caching for such file systems + <teythoon> indeed + <braunr> why would you want caching ? + <pinotree> you might have files that don't change at all, or rarely do + <braunr> the premise is that it's meant for files generated on the fly + <braunr> but are they big ? diff --git a/open_issues/libpthread.mdwn b/open_issues/libpthread.mdwn index e2fda122..0b426884 100644 --- a/open_issues/libpthread.mdwn +++ b/open_issues/libpthread.mdwn @@ -974,7 +974,7 @@ Most of the issues raised on this page has been resolved, a few remain. <braunr> exec weights 164M eww, we definitely have to fix that leak <braunr> the leaks are probably due to wrong mmap/munmap usage -[[exec_leak]]. +[[exec_memory_leaks]]. ### IRC, freenode, #hurd, 2012-08-29 @@ -1260,7 +1260,7 @@ Most of the issues raised on this page has been resolved, a few remain. <braunr> i'll add traces to know which step causes the error -### IRC, freenode, #hurd, 2012-12-11 +#### IRC, freenode, #hurd, 2012-12-11 <youpi> braunr: mktoolnix seems like a reproducer for the libports thread priority issue @@ -1273,7 +1273,7 @@ Most of the issues raised on this page has been resolved, a few remain. <youpi> that's it, yes -### IRC, freenode, #hurd, 2013-03-01 +#### IRC, freenode, #hurd, 2013-03-01 <youpi> braunr: btw, "unable to adjust libports thread priority: (ipc/send) invalid destination port" is actually not a sign of fatality @@ -1284,6 +1284,34 @@ Most of the issues raised on this page has been resolved, a few remain. <braunr> weird sentence, agreed :p +#### IRC, freenode, #hurd, 2013-06-14 + + <gnu_srs> Hi, when running check for gccgo the following occurs (multiple + times) locking up the console + <gnu_srs> unable to adjust libports thread priority: (ipc/send) invalid + destination port + <gnu_srs> (not locking up the console, it was just completely filled with + messages)) + <braunr> gnu_srs: are you running your translator as root ? + <braunr> or, do you have a translator running as an unprivileged user ? + <braunr> hm, invalid dest port + <braunr> that's a bug :p + <braunr> but i don't know why + <braunr> i'll have to take some time to track it down + <braunr> it might be a user ref overflow or something similarly tricky + <braunr> gnu_srs: does it happen everytime you run gccgo checks or only + after the system has been alive for some time ? + <braunr> (some time being at least a few hours, more probably days) + +#### IRC, freenode, #hurd, 2013-07-05 + + <braunr> ok, found the bug about invalid ports when adjusting priorities + <braunr> thhe hurd must be plagued with wrong deallocations :( + <braunr> i have so many problems when trying to cleanly destroy threads + +[[libpthread/t/fix_have_kernel_resources]]. + + ### IRC, freenode, #hurd, 2013-03-11 <braunr> youpi: oh btw, i noticed a problem with the priority adjustement @@ -1296,6 +1324,171 @@ Most of the issues raised on this page has been resolved, a few remain. <youpi> uh <youpi> indeed +### IRC, freenode, #hurd, 2013-07-01 + + <youpi> braunr: it seems as if pfinet is not prioritized enough + <youpi> I'm getting network connectivity issues when the system is quite + loaded + <braunr> loaded with what ? + <braunr> it could be ext2fs having a lot more threads than other servers + <youpi> building packages + <youpi> I'm talking about the buildds + <braunr> ok + <braunr> ironforge or others ? + <youpi> they're having troubles uploading packages while building stuff + <youpi> ironforge and others + <youpi> that happened already in the past sometimes + <youpi> but at the moment it's really pronounced + <braunr> i don't think it's a priority issue + <braunr> i think it's swapping + <youpi> ah, that's not impossible indeed + <youpi> but why would it swap? + <youpi> there's a lot of available memory + <braunr> a big file is enough + <braunr> it pushes anonymous memory out + <youpi> to fill 900MiB memory ? + <braunr> i see 535M of swap on if + <braunr> yes + <youpi> ironforge is just building libc + <braunr> and for some reason, swapping is orders of magnitude slower than + anything else + <youpi> not linking it yet + <braunr> i also see 1G of free memory on it + <youpi> that's what I meant with 900MiB + <braunr> so at some point, it needed a lot of memory, caused swapping + <braunr> and from time to time it's probably swapping back + <youpi> well, pfinet had all the time to swap back already + <youpi> I don't see why it should be still suffering from it + <braunr> swapping is a kernel activity + <youpi> ok, but once you're back, you're back + <youpi> unless something else pushes you out + <braunr> if the kernel is busy waiting for the default pager, nothing makes + progress + <braunr> (eccept the default pager hopefully) + <youpi> sure but pfinet should be back already, since it does work + <youpi> so I don't see why it should wait for something + <braunr> the kernel is waiting + <braunr> and the kernel isn't preemptibl + <braunr> e + <braunr> although i'm not sure preemption is the problem here + <youpi> well + <youpi> what I don't understand is what we have changed that could have so + much impact + <youpi> the only culprit I can see is the priorities we have changed + recently + <braunr> do you mean it happens a lot more frequently than before ? + <youpi> yes + <youpi> way + <braunr> ok + <youpi> ironforge is almost unusable while building glibc + <youpi> I've never seen that + <braunr> that's weird, i don't have these problems on darnassus + <braunr> but i think i reboot it more often + <braunr> could be a scalability issue then + <braunr> combined with the increased priorities + <braunr> if is indeed running full time on the host, whereas swapping + issues show the cpu being almost idle + <braunr> loadavg is high too so i guess there are many threads + <braunr> 0 971 3 -20 -20 1553 305358625 866485906 523M 63M * S<o + ? 13hrs /hurd/ext2fs.static -A /dev/hd0s2 + <braunr> 0 972 3 -20 -20 1434 125237556 719443981 483M 5.85M * S<o + ? 13hrs /hurd/ext2fs.static -A /dev/hd0s3 + <braunr> around 1k5 each + <youpi> that's quite usual + <braunr> could be the priorities then + <braunr> but i'm afraid that if we lower them, the number of threads will + grow out of control + <braunr> (good thing is i'm currently working on a way to make libpthread + actually remove kernel resources) + <youpi> but the priorities should be the same in ext2fs and pfinet, + shouldn't they? + <braunr> yes but ext2 has a lot more threads than pfinet + <braunr> the scheduler only sees threads, we don't have a grouping feature + <youpi> right + <braunr> we also should remove priority depressing in glibc + <braunr> (in sched_yield) + <braunr> it affects spin locks + + <braunr> youpi: is it normal to see priorities of 26 ? + <youpi> braunr: we have changed the nice factor + <braunr> ah, factor + <youpi> Mm, I'm however realizing the gnumach kernel running these systems + hasn't been upgraded in a while + <youpi> it may not even have the needed priority levels + <braunr> ar euare you using top right now on if ? + <braunr> hm no i don't see it any more + <braunr> well yes, could be the priorities .. + <youpi> I've rebooted with an upgraded kernel + <youpi> no issue so far + <youpi> package uploads will tell me on the long run + <braunr> i bet it's also a scalability issue + <youpi> but why would it appear now only? + <braunr> until the cache and other data containers start to get filled, + processing is fast enough that we don't see it hapenning + <youpi> sure, but I haven't seen that in the past + <braunr> oh it's combined with the increased priorities + <youpi> even after a week building packages + <braunr> what i mean is, increased priorities don't affect much if threads + porcess things fast + <braunr> things get longer with more data, and then increased prioritis + give more time to these threads + <braunr> and that's when the problem appears + <youpi> but increased priorities give more time to the pfinet threads too, + don't they? + <braunr> yes + <youpi> so what is different ? + <braunr> but again, there are a lot more threads elsewhere + <braunr> with a lot more data to process + <youpi> sure, but that has alwasy been so + <braunr> hm + <youpi> really, 1k5 threads does not surprise me at all :) + <youpi> 10k would + <braunr> there aren't all active either + <youpi> yes + <braunr> but right, i don't know why pfinet would be given less time than + other threads .. + <braunr> compared to before + <youpi> particularly on xen-based buildds + <braunr> libpthread is slower than cthreads + <youpi> where it doesn't even have to wait for netdde + <braunr> threads need more quanta to achieve the same ting + <braunr> perhaps processing could usually be completed in one go before, + and not any more + <braunr> we had a discussion about this with antrik + + <braunr> youpi: concerning the buildd issue, i don't think pfinet is + affected actually + <braunr> but the applications using the network may be + <youpi> why using the network would be a difference ? + <braunr> normal applications have a lower priority + <braunr> what i mean is, pfinet simply has nothing to do, because normal + applications don't have enough cpu time + <braunr> (what you said earlier seemed to imply pfinet had issues, i don't + think it has) + <braunr> it should be easy to test by pinging the machine while under load + <braunr> we should also check the priority of the special thread used to + handle packets, both in pfinet and netdde + <braunr> this one isn't spawned by libports and is likely to have a lower + priority as well + + <braunr> youpi: you're right, something very recent slowed things down a + lot + <braunr> perhaps the new priority factor + <braunr> well not the factor but i suppose the priority range has been + increased + +[[nice_vs_mach_thread_priorities]]. + + <youpi> braunr: haven't had any upload issue so far + <youpi> over 20 uploads + <youpi> while it was usually 1 every 2 before... + <youpi> so it was very probably the kernel missing the priorities levels + <braunr> ok + <braunr> i think i've had the same problem on another virtual machine + <braunr> with a custom kernel i built a few weeks ago + <braunr> same kind of issue i guess + <braunr> it's fine now, and always was on darnassus + ## IRC, freenode, #hurd, 2012-12-05 diff --git a/open_issues/libpthread/t/fix_have_kernel_resources.mdwn b/open_issues/libpthread/t/fix_have_kernel_resources.mdwn index 10577c1e..6f09ea0d 100644 --- a/open_issues/libpthread/t/fix_have_kernel_resources.mdwn +++ b/open_issues/libpthread/t/fix_have_kernel_resources.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,7 +10,9 @@ License|/fdl]]."]]"""]] [[!tag open_issue_libpthread]] -`t/have_kernel_resources` +`t/fix_have_kernel_resources` + +Address problem mentioned in [[/libpthread]], *Threads' Death*. # IRC, freenode, #hurd, 2012-08-30 @@ -19,3 +21,395 @@ License|/fdl]]."]]"""]] <braunr> tschwinge: i.e. the ability to tell the kernel where the stack is, so it's unmapped when the thread dies <braunr> which requiring another thread to perform this deallocation + + +## IRC, freenode, #hurd, 2013-05-09 + + <bddebian> braunr: Speaking of which, didn't you say you had another "easy" + task? + <braunr> bddebian: make a system call that both terminates a thread and + releases memory + <braunr> (the memory released being the thread stack) + <braunr> this way, a thread can completely terminates itself without the + assistance of a managing thread or deferring work + <bddebian> braunr: That's "easy" ? :) + <braunr> bddebian: since it's just a thread_terminate+vm_deallocate, it is + <braunr> something like thread_terminate_self + <bddebian> But a syscall not an RPC right? + <braunr> in hurd terminology, we don't make the distinction + <braunr> the only real syscalls are mach_msg (obviously) and some to get + well known port rights + <braunr> e.g. mach_task_self + <braunr> everything else should be an RPC but could be a system call for + performance + <braunr> since mach was designed to support clusters, it was necessary that + anything not strictly machine-local was an RPC + <braunr> and it also helps emulation a lot + <braunr> so keep doing RPCs :p + + +## IRC, freenode, #hurd, 2013-05-10 + + <braunr> i'm not sure it should only apply to self though + <braunr> youpi: can we get a quick opinion on this please ? + <braunr> i've suggested bddebian to work on a new RPC that both terminates + a thread and releases its stack to help fix libpthread + <braunr> and initially, i thought of it as operating only on the calling + thread + <braunr> do you see any reason to make it work on any thread ? + <braunr> (e.g. a real thread_terminate + vm_deallocate) + <braunr> (or any reason not to) + <youpi> thread stack deallocation is always a burden indeed + <youpi> I'd tend to think it'd be useful, but perhaps ask the list + + +## IRC, freenode, #hurd, 2013-06-26 + + <braunr> looks like there is a port right leak in libpthread + <braunr> grmbl, the port leak seems to come from mach_port_destroy being + buggy :/ + <braunr> hum, apparently we're not the only ones to suffer from port leaks + wrt mach_port_destroy + <braunr> ew, libpthread is leaking + <pinotree> memory or ports? + <braunr> both + <pinotree> sounds great ;) + <braunr> as it is, libpthread doesn't destroy threads + <braunr> it queues them so they're recycled late + <braunr> r + <braunr> but there is confusion between the thread structure itself and its + internal resources + <braunr> i.e. there is pthread_alloc which allocates a thread structure, + and pthread_create which allocates everything else + <braunr> but on pthread_exit, nothing is destroyed + <braunr> when a thread structure is reused, its internal resources are + replaced by new instances + <pinotree> oh + <braunr> it's ok for joinable threads but most of our threads are detached + <braunr> pinotree: as expected, it's bigger than expected :p + <braunr> so i won't be able to write a quick fix + <braunr> the true way to fix this is make it possible for threads to free + their own resources + <braunr> let's do that :p + <braunr> ok, got the new thread termination function, i'll build eglibc + package providing it, then experiment with libpthread + <pinotree> braunr: iirc there's also a tschwinge patch in the debian eglibc + about that + <braunr> ah + <pinotree> libpthread_fix.diff + <braunr> i see + <braunr> thanks for the notice + <braunr> bddebian: + http://www.sceen.net/~rbraun/0001-thread_terminate_deallocate.patch + <braunr> bddebian: this is what it looks like + <braunr> see, short and easy + <bddebian> Aye but didn't youpi say not to bother with it?? + <braunr> he did ? + <braunr> i don't remember + <bddebian> I thought that was the implication. Or maybe that was the one I + already did!? + <braunr> i'd be interested in reading that + <braunr> anyway, there still are problems in libpthread, and this call is + one building block to fix some of them + <braunr> some important ones + <braunr> (big leaks) + + +## IRC, freenode, #hurd, 2013-06-29 + + <braunr> damn, i fix leaks in libpthread, only to find out leaks somewhere + else :( + <braunr> bddebian: ok, actually it was a bit more complicated than what i + showed you + <braunr> because in addition to the stack, the call must also release the + send right in the caller's ipc space + <braunr> (it can't be released before since there would be no mean to + reference the thread to destroy) + <braunr> or perhaps it should strictly be reserved to self termination + <braunr> hmm + <braunr> yes it would probably be simpler + <braunr> but it should be a decent compromise + <braunr> i'm close to having a libpthread that doesn't leak anything + <braunr> and that properly destroys threads and their resources + + +## IRC, freenode, #hurd, 2013-06-30 + + <braunr> bddebian: ok, it was even more tricky, because the kernel would + save the return value on the user stack (which is released by the call + and then invalid) before checking for asynchronous software traps (ASTs, + a kind of software interrupts in mach), and terminating the calling + thread is done by a deferred AST ... :) + <braunr> hmm, making threads able to terminate themselves makes rpctrace a + bit useless :/ + <braunr> well, more restricted + + <braunr> ok so, tough question : + <braunr> i have a small test program that creates a thread, and inspect its + state before any thread dies + <braunr> i can see msg_report_wait requests when using ps + <braunr> (one per thread) + <braunr> one of these requests create a new receive right, apparently for + the second thread in the test program + <braunr> each time i use ps, i can see the sequence numbers of two receive + rights increase + <braunr> i guess these rights are related to proc and signal handling per + thread + <braunr> but i can't find what create them + <braunr> does anyone know ? + <braunr> tschwing_: ^ :) + + <braunr> again, too many things wrong elsewhere to cleanly destroy threads + .. + <braunr> something is deeply wrong with controlling terminals .. + + +## IRC, freenode, #hurd, 2013-07-01 + + <braunr> youpi: if you happen to notice what receive right is created for + each thread (beyond the obvious port used for blocking and waking up), + please let me know + <braunr> it's the only port leak i have with thread destruction + <braunr> and i think it's related to the proc server since i see the + sequence number increase every time i use ps + + <braunr> pinotree: my change doesn't fix all the pthread leaks but it's a + lot better + <braunr> bddebian: i've spent almost the whole week end trying to find the + last port leak without success + <braunr> there is some weird bug related to the controlling tty that hits + me every time i try to change something + <braunr> it's the same bug that prevents ttys from being correctly closed + when using ssh or screen + <braunr> well maybe not the same, but it's close + <braunr> some stale receive right kept around for no apparent reason + <braunr> and i can't find its source + + +## IRC, freenode, #hurd, 2013-07-02 + + <braunr> and btw, i don't think i can make my libpthread patch work + <braunr> i'll just aim at avoiding leaks, but destroying threads and their + related resources depends on other changes i don't clearly see + + +## IRC, freenode, #hurd, 2013-07-03 + + <braunr> grmbl, i don't want to give up thread destruction .. + + +## IRC, freenode, #hurd, 2013-07-15 + + <braunr> btw, my work on thread destruction is currently stalled + <braunr> i don't have much free time right now + + +## IRC, freenode, #hurd, 2013-09-13 + + <braunr> i think i know why my thread_terminate_deallocate patches leak one + receive port :> + <braunr> but now i'm not sure of the proper solution + <braunr> every time a thread is created and destroyed, a receive right is + leaked + <braunr> i guess it's simply the reply port .. + <braunr> grmbl + <braunr> i guess i have to make it a simpleroutine ... + <braunr> hm too bad, it's not the reply port :( + <braunr> it's also leaking some memory + <braunr> it doesn't seem related to my changes though + <braunr> stacks, rights, and threads are correctly destroyed + <braunr> some obscure state is left behind + <braunr> i wonder how exception ports are dealt with + <braunr> vminfo seems to confirm memory is leaking in the heap + <braunr> humpf + <braunr> oh silly me + <braunr> i don't detach threads + <teythoon> well, detach them ;) + <braunr> hm worse :p + <braunr> now i get additional dead names + <braunr> but it's a step forward + + +## IRC, freenode, #hurd, 2013-09-16 + + <braunr> that thread port leak is so strange + <braunr> the leaked port seems to be created when the new thread starts + running + <braunr> so it looks like a port the kernel would implicitely create + <braunr> hm could it be a thread-specific reply port ? + <youpi> ah, yes, there is one of those + <braunr> how come mach/mig-reply.c in glibc isn't thread-safe ? + <youpi> it is overriden by sysdeps/mach/hurd/img-reply.c I guess + <youpi> which uses a threadvar for the mig reply port + <braunr> oh + <youpi> talking of which, there is also last_value in + sysdeps/mach/strerror_l.c + <youpi> strerror_thread_freeres is supposed to get called, but who knows + <braunr> it does look to be that port + <youpi> iirc that's the issue which prevents from letting us make threads + exit on idleness? + <braunr> one of them + <youpi> ok + <braunr> maybe the only one, yes + <braunr> i see memory leaks but they could be related/normal + <braunr> (i.e. not actual leaks) + <braunr> on the other hand, i also can't boot a hurd with my patch + <braunr> but i consider removing such leaks a priority + <braunr> does anyone know the semantic difference between + __mig_put_reply_port and __mig_dealloc_reply_port ? + <braunr> i guess __mig_dealloc_reply_port is actually a destruction + operation, right ? + <youpi> AIUI, dealloc is used when one wants the port not to be reused at + all + <youpi> because it has been used as a reference for something, and can + still be currently in use + <youpi> while put_reply would be when we're really done with it, and won't + use it again, and can thus be used as such + <youpi> or at least something like that + <braunr> heh + <braunr> __mig_dealloc_reply_port calls __mach_port_mod_refs, which is a + RPC, and creates a new reply port when destroying the current one + <youpi> bah + <youpi> that's fine, it's a deref of the old port, which is not in the + reply_port variable any more + <braunr> it's fine, but still a leak + <youpi> well, dealloc does not completely deallocs, yes + <braunr> that's not really the problem here + <braunr> i've introduced a case that wasn't considered at the time, namely + that a thread can destroy itself + <youpi> we probably need another function to be called from the thread exit + <braunr> i'll simply try with mach_port_destroy + <braunr> mach_port_destroy seems to be a RPC too ... + <braunr> grmbl + <youpi> isn't there a trap version somehow ? + <braunr> not in libc + <youpi> erf + <braunr> at least i know what's wrong now :) + <braunr> there still is a small memory leak i have to investigate + <braunr> but outside the stack + <braunr> the stack, the thread name and the thread are correctly destroyed + <braunr> slabinfo confirms only one port leak and nothing else is leaked + <braunr> ok so the port leak was indeed the thread-specific reply port, + taken care of + <braunr> there are also memory leaks too + + +## IRC, freenode, #hurd, 2013-09-17 + + <braunr> teythoon: on my side, i'm getting to know our threading + implementation better + <braunr> closing to clean thread destruction + <braunr> x15 ipc will hide reply ports ;p + <braunr> memory leaks solved \o/ + <braunr> now, have to fix memory release when joining + <braunr> proper reference counting on detach/join/exit, let's see how it + goes .. + <braunr> seems to work fine + + +## IRC, freenode, #hurd, 2013-09-18 + + <braunr> ok i'll soon have gnumach and libc packages including proper + thread destruction :> + <teythoon> braunr: why did you have to touch gnumach? + <braunr> to add a call allowing threads to release ports and memory + <braunr> i.e. their last self reference, their reply port and their stack + <braunr> let me public my current patches + <teythoon> braunr: thread_commit_suicide ? + <braunr> hehe + <braunr> initially thread_terminate_self but + <braunr> it can be used by other threads too + <braunr> to i named it thread_terminate_release + <braunr> http://darnassus.sceen.net/~rbraun/0001-pthread_thread_halt.patch + <braunr> + http://darnassus.sceen.net/~rbraun/0001-thread_terminate_release.patch + <braunr> the pthread patch needs to be polished because it changes the + semantics of pthread_thread_halt + <braunr> but other than that, it should be complete + <pinotree> pthread_thread_halt_reallyhalt + <braunr> ok let's try these libc packages + <braunr> old static ext2fs for the root, but other than that, it boots + <braunr> let's try iceweasel + <braunr> (i'll need to build a hurd package against this new libc, removing + the libports_stability patch which prevents thread destruction in servers + on the way) + <teythoon> prevents thread destruction o_O + <braunr> yes + <braunr> in libports only ;p + <teythoon> oh, *only* in libports, I assumed for a moment that it affected + almost every component of the Hurd... + <teythoon> *phew( + <braunr> ... :) + <braunr> that's why, after a burst of messages, say because of aptitude + (select), you may see a few hundred threads still hanging around + <braunr> also why unused servers remain running even after several minutes, + where the normal timeout is 2mins + <teythoon> I wondered about that, some servers (symlink comes to mind) seem + to go away if unused (or that's how I read the code) + <braunr> symlinks are usually not servers, since most of them actually + exist in file systems, and are implemented through an optimization + <teythoon> yes I know that + <teythoon> trans/symlink.c reads: + <teythoon> /* The timeout here is 10 minutes */ + <teythoon> err = mach_msg_server_timeout (fsys_server, 0, control, + <teythoon> MACH_RCV_TIMEOUT, 1000 * 60 * 10); + <teythoon> if (err == MACH_RCV_TIMED_OUT) + <teythoon> exit (0); + <braunr> ok + <teythoon> hm, /hurd/symlink doesn't feel at all like a symlink... but + works like one + <braunr> well, starting iceweasel makes X on my host freeze oO + <braunr> bbl + <teythoon> /hurd/symlink translators do go away after being unused for 10 + minutes... this is funny if they are set up by hand instead of being + started from a passive translator record + <teythoon> magically vanishing symlinks ;) + + +## IRC, freenode, #hurd, 2013-09-19 + + <braunr> hum, i can't rebuild a hurd package :( + <teythoon> braunr: with your thread destruction patches in libc? + <braunr> yes but it's unrelated + <braunr> In file included from ../../libdiskfs/boot-start.c:38:0: + <braunr> ./fsys_reply_U.h:173:15: error: conflicting types for + ‘fsys_get_children’ + <braunr> i didn't see a new libc debian release + <teythoon> hm, David reported that as well + <teythoon> + id:CAEvUa7=QzOiS41G5Vq8k4AiaN10jAPm+CL_205OHJnL0xpJXbw@mail.gmail.com + <teythoon> uh oh + <teythoon> it seems I didn't add a _reply suffix to the reply routines :/ + <teythoon> there's quite a bit of fallout from my patches, I kinda feel bad + :( + <braunr> teythoon: what i'm wondering is what youpi did too, since he got + hurd binary packages + <teythoon> braunr: well neither he nor I noticed that b/c for us the + declarations were just missing + <braunr> from libc you mean ? + <braunr> or hum gnumach-common ? + <teythoon> not sure actually + <braunr> no it's not a gnumach thing + <braunr> hurd-dev then + <teythoon> the build system should have cought these, or mig... + <braunr> also, i see you changed fsys_reply.defs, but nothing about + fsys_request.defs + <teythoon> I have no fsys_requests.defs + <braunr> looks like there was no fsys_request.defs in the first place + ... *sigh* + <braunr> do you know an application that often creates and destroys threads + ? + <teythoon> no, sorry + <pinotree> maybe some test suite + <braunr> ah right + <braunr> sysbench maybe + <braunr> also, i've been hit by a lot more network deadlocks than usual + lately + <braunr> fixing netdde has gained some priority in my todo list + + +## IRC, freenode, #hurd, 2013-09-20 + + <braunr> oh, git is multithreaded + <braunr> great + <braunr> so i've actually tested my libpthread patch quite a lot diff --git a/open_issues/libpthread_assertion_thread_prevp.mdwn b/open_issues/libpthread_assertion_thread_prevp.mdwn index e8160528..f93f07d6 100644 --- a/open_issues/libpthread_assertion_thread_prevp.mdwn +++ b/open_issues/libpthread_assertion_thread_prevp.mdwn @@ -87,3 +87,23 @@ failed"]] <braunr> removing the libports_stability patch exposed bugs in libpthread, triggering assertions when queueing/dequeue threads from a queue (but i don't know which one / in which function) + + +## IRC, freenode, #hurd, 2013-06-25 + + <pinotree> braunr: + https://buildd.debian.org/status/fetch.php?pkg=libmemcached&ver=1.0.17-2&arch=hurd-i386&stamp=1372165732 + <pinotree> make: ./pthread/pt-internal.h:122: __pthread_enqueue: Assertion + `thread->prevp == 0' failed. \o/ + <pinotree> (it should rather be /o\, but better pretend not) + <braunr> pinotree: yes, we regularly see it + <braunr> pinotree: how long has the machine been running at this point ? + <pinotree> dunno, you should ask samuel about that + <pinotree> does it happen after N hours/days? + <braunr> a few days of moderate to high activity yes + <pinotree> ah ok + <braunr> and i actually see this error much more often when i disable the + libports stability patch in the hurd debian package + <braunr> so i guess something is wrong with thread recycling + <braunr> but i wanted to completely rewrite that part with the new kernel + call i asked bddebian to work on :) diff --git a/open_issues/libpthread_dlopen.mdwn b/open_issues/libpthread_dlopen.mdwn index 05a07ef2..3c36eb26 100644 --- a/open_issues/libpthread_dlopen.mdwn +++ b/open_issues/libpthread_dlopen.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2012, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -12,6 +13,7 @@ License|/fdl]]."]]"""]] [[!toc]] +# [[FAQ entry|faq/libpthread_dlopen]] # IRC, freenode, #hurd, 2010-01-24 @@ -112,7 +114,18 @@ IRC, freenode, #hurd, 2011-08-17 [[packaging_libpthread]] ---- + +# IRC, freenode, #hurd, 2013-09-03 + + <gnu_srs> iceweasel: ./pthread/../sysdeps/generic/pt-mutex-timedlock.c:70: + __pthread_mutex_timedlock_internal: Assertion `__pthread_threads' failed. + <pinotree> LD_PRELOAD libpthread + <gnu_srs> why + <pinotree> missing link to pthread? + <pinotree> and yes, it's known already, just nobody worked on solving it + + +# libthreads vs. libpthread The same symptom appears in an odd case, for instance: diff --git a/open_issues/llvm.mdwn b/open_issues/llvm.mdwn index 2a4b4ed5..4da58579 100644 --- a/open_issues/llvm.mdwn +++ b/open_issues/llvm.mdwn @@ -97,6 +97,39 @@ a06fe9183fbffb78798a444da9bc3040fdd444aa (2013-03-23), test-suite 2012 Debian project, <http://wiki.debian.org/SummerOfCode2012/StudentApplications/AndrejBelym>. + * [[sanitizers|_san]] + + A lot of Linux-specific things. + + * IRC, OFTC, #debian-hurd, 2013-09-05: + + <gg0> how can this fix it on {kf,hurd}-i386? + http://anonscm.debian.org/viewvc/pkg-llvm/llvm-toolchain/branches/3.3/debian/patches/libstdc%2B%2B-header-i386.diff?view=markup&sortby=date&pathrev=830 + <pinotree> what makes you think it does? + <pinotree> it fixes #714890, which has nothing to do with hurd or + kfreebsd + <gg0> i simple wouldn't add a patch that fixes it on one i386 arch + only, being aware there are others + <pinotree> meet sylvestre + + * IRC, freenode, #hurd, 2013-09-05: + + <pinotree> tschwinge: iirc you were working on llvm/clang, weren't you? + <tschwinge> pinotree: That's right. I have patches to + follow-up/rework. Stalled at the moment, as you probably already + guessed... %-) + <pinotree> tschwinge: <Sylvestre> by the way, pinotree if you have time + for hurd stuff, I would be glad to have your help to port + llvm-toolchain-3.3 to hurd. I am having some issues with threading + aspects https://paste.debian.net/35466/ + <pinotree> he's the debian packager of llvm + <tschwinge> That paste is for LLDB -- which I'd not assume to be in a + shape usable for Hurd. + <tschwinge> (I didn't look at it at all.) + <pinotree> tschwinge: if you look at the latest llvm-toolchain-3.3 + debian source, there's a lldb-hurd.diff patch, which starts some + include header dance + # Build diff --git a/open_issues/mach_migrating_threads.mdwn b/open_issues/mach_migrating_threads.mdwn index c14ce95a..bbc6ac45 100644 --- a/open_issues/mach_migrating_threads.mdwn +++ b/open_issues/mach_migrating_threads.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -15,3 +15,89 @@ License|/fdl]]."]]"""]] * [[microkernel/mach/memory_object/discussion]] * [[resource_management_problems]] + + +# IRC, freenode, #hurd, 2013-08-13 + +In context of [[resource_management_problems]]. + + <braunr> and thread migration itself is something very confusing + <braunr> it's better to think of it as scheduling context inheritance + <teythoon> braunr: I read the paper I mentioned and then I wanted to find + the sources they modified + <teythoon> I failed + <teythoon> I hate scientific paper about software that fail to provide the + source code + <teythoon> that's not science imho b/c it's not reproducible + <braunr> i have some osf source code here + <braunr> i'll send it if you want + <teythoon> ah interesting + <braunr> but really, when you dive into it, thread migration is merely + scheduling context inheritance with kernel upcalls + <braunr> it's good + <teythoon> I searched for osf mach but google didn't turned up anything + <braunr> but it has nothing to do with resource accounting + <braunr> (well, it may hepl better account for cpu time actually) + <braunr> help* + <braunr> but that's all + <teythoon> why is that all? wouldn't that be transitive and could also be + used for i/o accounting? + <teythoon> also I tried to find alternative mach implementations + <teythoon> I wasn't terribly successful, and some sites are gone or + unmaintained for years :/ + <braunr> we don't need that for io accountin + <braunr> g + <braunr> thread migration is a kernel property + <braunr> on mach with userspace drivers, io isn't + <braunr> mach should only control cpu and memory + <braunr> and you though you can account physical memory, you can't transfer + virtual memory accounting from one task to another + <teythoon> yes, but once all of those resources can be accounted to the + thread initiating whatever it needs doing, shouldn't that be much easier? + <braunr> teythoon: it's not required for that + <braunr> teythoon: keep in mind userspace sees activations + <braunr> in a thread migration enabled kernel, activations are what we + usually call threads, and threads are scheduling contexts + <teythoon> braunr: ok, so TM is not required for accounting, but surely + it's a good thing to have, no? + <braunr> teythoon: it's required for cpu accounting only + <braunr> which is very important :) + <braunr> if you look carefully, you'll see hurd servers are what use most + cpu + <braunr> there is now easy way to know which application actually uses the + server + <braunr> i personally tend to think more and more that servers *should* + impersonate clients + <braunr> TM (or rather, scheduling context inheritance) is one step + <braunr> it's not enough exactly because it doesn't help with resource + accounting + <braunr> teythoon: + ftp://ftp.mklinux.org/pub/mklinux-pre-R1/SRPMS/sources/osfmk.tar.gz + + +# IRC, freenode, #hurd, 2013-09-02 + +[[!taglink open_issue_documentation]]: move information to +[[microkernel/mach/history]]. + + <teythoon> braunr: btw, I just noticed lot's of #ifdef MIGRATING_THREADS in + gnumach, so there was some work being done in that direction? + <braunr> gnumach is a fork of mach4 + <braunr> at a stage whre migration was being worked on, yes + <teythoon> from what I've gathered, gnumach is the only surviving mach4 + fork, right? + <braunr> yes + <braunr> well + <braunr> the macos x version is probably one too + <braunr> i don't know + <teythoon> oh? I read that it was based on mach3 + <braunr> it is + <braunr> i can't tell how much of mach3 versus mach4 it has, and if it's + relevant at all + <teythoon> and the osfmach, was that also based on mach4? + <braunr> yes + <teythoon> ok, fair enough + <braunr> that's why i think macos x is based on it too + <braunr> i initially downloaded osfmach sources to see an example of how + thread migration was used from userspace + <braunr> and they do have a special threading library for that diff --git a/open_issues/magic_translator_machtype.mdwn b/open_issues/magic_translator_machtype.mdwn index 9c1a0d76..3ae16cf0 100644 --- a/open_issues/magic_translator_machtype.mdwn +++ b/open_issues/magic_translator_machtype.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2008, 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2010, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -23,4 +24,5 @@ License|/fdl]]."]]"""]] Segmentation fault tschwinge@clubber:~ $ l mach<TAB>Connection to clubber.bddebian.com closed. -Justus: This is most likely just the shell not handling SIGLOST, see: <https://savannah.gnu.org/bugs/?19479> +Justus: This is most likely just the shell not handling SIGLOST, see +[[!GNU_Savannah_bug 19479]]. diff --git a/open_issues/memory_object_model_vs_block-level_cache.mdwn b/open_issues/memory_object_model_vs_block-level_cache.mdwn index 7da5dea4..22db9b86 100644 --- a/open_issues/memory_object_model_vs_block-level_cache.mdwn +++ b/open_issues/memory_object_model_vs_block-level_cache.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,6 +10,8 @@ License|/fdl]]."]]"""]] [[!tag open_issue_documentation open_issue_hurd open_issue_gnumach]] +[[!toc]] + # IRC, freenode, #hurd, 2012-02-14 @@ -271,3 +273,242 @@ License|/fdl]]."]]"""]] <mcsim> slpz: When mo_data_return is called, once the memory manager no longer needs supplied data, it should be deallocated using vm_deallocate. So this way pagers acknowledges the end of flush. + + +# IRC, freenode, #hurd, 2013-08-26 + + < Spyro> Ok, so + < Spyro> idiot question: in a nutshell, what is a memory object? + < Spyro> and how is swapping/paging handled? + < braunr> Spyro: a memory object is how the virtual memory system views a + file + < braunr> so it's a sequence of bytes with a length + < braunr> "swapping" is just a special case of paging that applies to + anonymous objects + < braunr> (which are named so because they're not associated with a file + and have no name) + < Spyro> Who creates a memory object, and when? + < braunr> pagers create memory objects when needed, e.g. when you open a + file + < Spyro> and this applies both to mmap opens as well as regular I/O opens + as in for read() and write()? + < braunr> basically, all file systems capable of handling mmap requests + and/or caching in physical memory are pagers + < braunr> yes + < braunr> read/write will go through the page cache when possible + < Spyro> and who owns the page cache? + < Spyro> also, who decides what pages ot evict to swap/file if physical + memory gets tight? + < braunr> the kernel + < braunr> that's one of the things that make mach a hybrid + < Spyro> so the kernel owns the page cage? + < Spyro> ...fml + < Spyro> cache! + < braunr> yes + + +## IRC, freenode, #hurd, 2013-08-27 + + < Spyro> so braunr: So, who creates the memory object, and how does it get + populated? + < Spyro> and how does a process accessing a file get hooked up to the + memory object? + < braunr> Spyro: i told you, pagers create memory objects + < braunr> memory objects are how the VM system views files, so they're + populated from the content of files + < braunr> either true files or virtual files such as in /proc + < braunr> Spyro: processes don't directly access memory objects unless + memory mapping them with vm_map() + < braunr> pagers (basically = file systems) do + <Spyro> ok, so how is a pager/fs involved in handling a fault? + + +## IRC, freenode, #hurd, 2013-08-28 + + <braunr> Spyro: each object is linked to a pager + <braunr> Spyro: when a fault occurs, the kernel looks up the VM map (kernel + or a user one), and the address in this map, then the map entry, checks + access and lots of other details + <Spyro> ok, so it's pager -> object -> vmem + <Spyro> ? + <braunr> Spyro: then finds the object mapped at that address (similar to + how a file is mapped with mmap) + <braunr> from the object, it finds the pager + <Spyro> ok + <braunr> and asks the pager about the data at the appropriate offset + <Spyro> so how does a user process do normal file I/O? is faulting just a + special case of it? + <braunr> it's completely separate + <Spyro> eww + <braunr> normal I/O is done with message passing + <braunr> the hurd io interface + <Spyro> ok + <Spyro> so who talks to who on a file I/O? + <braunr> a client (e.g. cat) talks to a file system server (e.g. ext2fs) + <Spyro> ok so + <Spyro> it's client to the pager for regular file I/O? + <braunr> Spyro: i don't understand the question + <braunr> Spyro: it's client to server, the server might not be a pager + <Spyro> ok + <Spyro> just trying to figure out the difference between paging/faulting + and regular I/O + <braunr> regular I/O is just message passing + <braunr> page fault handling is dealt with by pagers + <Spyro> and I have a hunch that the fs/pager is involved somehow in both, + because the server is the source of the data + <Spyro> I'm getting a headache + <braunr> nalaginrut: a server like ext2fs is both a file server and a pager + <Spyro> oh! + <Spyro> oh btw, does a file server make use of memory objects for caching? + <braunr> Spyro: yes + <Spyro> or rather, can it? + <Spyro> does it have to? + <braunr> memory objects are for caching, and thus for page faults + <braunr> Spyro: for caching, it's a requirement + <braunr> for I/O, it's not + <braunr> you could have I/O without memory objects + <Spyro> ok + <Spyro> so how does the pager/fileserver use memory objects for caching? + <Spyro> does it just map and write to them? + <braunr> basically yes but there is a complete protocol with the kernel for + that + <braunr> + http://www.gnu.org/software/hurd/gnumach-doc/External-Memory-Management.html#External-Memory-Management + <Spyro> heh, lucky guess + <Spyro> ty + <Spyro> I am in way over my head here btw + <Spyro> zero experience with micro kernels in practice + <braunr> it's not trivial + <braunr> that's not a microkernel thing at all + <braunr> that's how it works in monolithic kernels too + <braunr> i recommend netbsd uvm thesis + <braunr> there are nice pictures describing the vm system + <Spyro> derrr...preacious? + <Spyro> wow + <braunr> just ignore the anonymous memory handling part which is specific + to uvm + <Spyro> @_@ + <braunr> the rest is common to practically all VM systems out there + <Spyro> I know about the linux page cache + <braunr> well it's almost the same + <Spyro> with memory objects being the same thing as files in a page cache? + <braunr> memory objects are linux "address spaces" + <braunr> and address spaces are how the linux mm views a file, yes + <Spyro> derp + <Spyro> ... + <Spyro> um... + <braunr> struvt vm_page == struct page + * Spyro first must learn what an address_space is + <braunr> struct vm_map == struct mm_struct + <braunr> struct vm_map_entry == struct vm_area_struct + * Spyro isn't a linux kernel vm expert either + <braunr> struct vm_object == struct address_space + <braunr> roughly + <braunr> details vary a lot + <Spyro> and what's an address_space ? + <braunr> 11:41 < braunr> and address spaces are how the linux mm views a + file, yes + <Spyro> ok + <braunr> see include/linux/fs.h + <braunr> struct address_space_operations is the pager interface + * Spyro should look at the linux kernel sources perhaps, unless you have an + easier reference + <Spyro> embarrassingly, RVR hired me as an editor for the linux-mm wiki + <Spyro> I should know this stuff + <braunr> see + http://darnassus.sceen.net/~rbraun/design_and_implementation_of_the_uvm_virtual_memory_system.pdf + <braunr> page 42 + <braunr> page 66 for another nice view + <braunr> i wouldn't recommend using linux source as refernece + <braunr> it's very complicated, filled with a lot of code dealing with + details + <Spyro> lmao + <braunr> and linux guys have a habit of choosing crappy names + <Spyro> I was only going to + <Spyro> stoppit + <braunr> except for "linux" and "git" + <Spyro> ...make me laugh any more and I'll need rib surgery + <braunr> laugh ? + <Spyro> complicated and crappy + <braunr> seriously, "address space" for a file is very very confusing + <Spyro> oh I agree with that + <braunr> yes, names are crappy + <braunr> and the code is very complicated + <braunr> it took me half an hour to find where readahead is done once + <braunr> and i'm still not sure it was the right code + <Spyro> so in linkern, there is an address_space for each cached file? + <braunr> takes me 30 seconds in netbsd .. + <braunr> yes + <Spyro> eww + <Spyro> yeah, BAD name + <Spyro> but thanks for the explanation + <Spyro> now I finally know what an address space is + <braunr> many linux core developers admit they don't care much about names + <Spyro> so, in hurd, a memory object is to hurd, what an address_space is + to linux? + <braunr> yes + <braunr> notto hurd + <Spyro> ok + <braunr> to mach + <Spyro> you know what I mean + <Spyro> :P + <Spyro> easier than for linux I can tell you that much + <braunr> and the bsd vm system is a stripped version of the mach vm + <Spyro> ok + <braunr> that's why i think it's important to note it + <Spyro> good, I learned something abou tthe linux vm...from the mach guys + <Spyro> this is funny + <braunr> linux did too + <braunr> there is a paper about linux page eviction that directly borrows + the mach algorithm and improves it + <braunr> mach is the historic motivation behind mmap on posix + <Spyro> oh nice! + <Spyro> but yes, linux picked a shitty name + <braunr> is all that clearer to you ? + <Spyro> I think that address_space connection was a magic bolt of + understanding + <braunr> and do you see how I/O and paging are mostly unrelated ? + <Spyro> almost + <Spyro> but how does a file I/O take advantage of caching by a memory + object? + <Spyro> does the file server just nudge the core for a hint? + <braunr> the file system copies from the memory object + * Spyro noddles + <Spyro> I think I understand a bit better now + <braunr> it's message passing + <Spyro> but I havfe too much to digest already + <braunr> memory copying + <braunr> if the memory is already there, good, if not, the kernel will ask + the file system to bring the data + <braunr> if message passing uses zero copy, data retrieval can be deferred + until the client actually accesses it + <Spyro> which is a fancy way of saying demand paging? :P + <braunr> it's always demand paging + <braunr> what i mean is that the file system won't fetch data as soon as it + copies memory + <braunr> but when this data is actually needed by the client + <Spyro> uh... + <Spyro> whta's a precious page? + <braunr> let me check quickly + <braunr> If precious is FALSE, the kernel treats the data as a temporary + and may throw it away if it hasn't been changed. If the precious value is + TRUE, the kernel treats its copy as a data repository and promises to + return it to the manager + <braunr> basically, it's used when you want the kernel to keep cached data + in memory + <braunr> the cache becomes a lossless container for such pages + <braunr> the kernel may flush them, but not evict them + <Spyro> what's the difference? + <braunr> imagine a ramfs + <Spyro> point made + <braunr> ok + <Spyro> would be pretty hard to flush something that doesn't have a backing + store + <braunr> that was quick :) + <braunr> well + <braunr> the normal backing store for anonymous memory is the default pager + <braunr> aka swap + <Spyro> eww + <braunr> but if you want your data *either* in swap or in memory and never + in both + <braunr> it may be useful diff --git a/open_issues/mig_portable_rpc_declarations.mdwn b/open_issues/mig_portable_rpc_declarations.mdwn index 084d7454..ecfa06ae 100644 --- a/open_issues/mig_portable_rpc_declarations.mdwn +++ b/open_issues/mig_portable_rpc_declarations.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -56,3 +56,112 @@ License|/fdl]]."]]"""]] <antrik> braunr: we discussed the problem of expressing structs with MIG in the libburn thread <antrik> (which I still need to follow up on... [sigh]) + + +# IRC, freenode, #hurd, 2013-06-25 + + <teythoon> is there a nice way to get structured data through mig that I + haven't found yet? + <teythoon> say an array of string triples + <braunr> no + <teythoon> :/ + <braunr> but you shouldn't need that + <teythoon> my use case is getting info about fs translators from init to + procfs + +[[hurd/translator/mtab]], [[hurd/translator/mtab/discussion]]. + + <teythoon> should I go for an iterator like interface instead? + <braunr> depends + <braunr> how many do you need ? + <braunr> you could go for a variable sized array too + <braunr> have a look at what already exists + <teythoon> records, maybe 10-15, depends on many fs translators are running + <braunr> a variable sized array is ok if the size isn't too big (and when i + say too big, i mean hundreds of MiB) + <braunr> an iterator is ok too if there aren't too many items + <braunr> you may want to combine both (i think that's what proc does) + <braunr> be aware that the maximum size of a message is limited to 512 MiB + <teythoon> yeah I saw the array[] of stuff stuff, but array[] of string_t + does not work, I guess b/c string_t is also an array + <teythoon> how would I send an array of variable length strings? + <braunr> i'm not sure you can + <braunr> or maybe out of line + <teythoon> somehow I expected mig to serialize arbitrary data structures, + maybe it's to old for that? + <teythoon> yeah, I read about uot of line, but that seems overkill + <braunr> it is old yes + <braunr> and not very user friendly in the end + <braunr> let me check + <teythoon> we could stuff json into mig... + <braunr> see proc_getallpids for example + <braunr> we could get rid of low level serialization altogether :p + <teythoon> hah, exactly what I was looking at + <braunr> (which is what i'll do in x15) + <braunr> type pidarray_t = array[] of pid_t; + <teythoon> but that is trivial b/c its array[] of pid_t + <braunr> and always have the server writing guide near you + <teythoon> yes + <braunr> well, make one big string and an array of lengths :p + <teythoon> thought about that and said to myself, there must be a better + way that I haven't found yet + <braunr> or one big string filled with real null-terminated c strings that + you keep parsing until you ate all input bytes + <braunr> i'm almost certain there isn't + <braunr> type string_t = c_string[1024]; /* XXX */ + <teythoon> yes + <braunr> even that isn't really variable sized + <teythoon> you think anyone would object to me putting a json encoder in + /hurd/init? it is probably better than me at serializing stuff... + <braunr> try with mig anyway + <braunr> the less dependencies we have for core stuff, the simpler it is + <braunr> but i agree, mig is painful + <teythoon> would it be too hacky if I abused the argz functions? they do + exactly what I'd need + + +## IRC, freenode, #hurd, 2013-06-26 + + <teythoon> there is https://code.google.com/p/protobuf-c/ and it has a rpc + mechanism and I believe one could plug arbitrary transports easily + <braunr> please don't think about it + <braunr> we really don't want to add another layer of serialization + <braunr> it's better to completely redesign mach ipc anyway + <braunr> and there is a project for that :p + <teythoon> ive seen x15 + <teythoon> just food for thought + <braunr> i've studied google protocol buffers + <braunr> and fyi, no, it wouldn't be easy to plug arbitrary transports on + top of mach + <braunr> there is a lot of knowledge about mach ports in mig + +[[hurd/translator/mtab]], [[hurd/translator/mtab/discussion]]. + + <teythoon> but again I face the challenge of serializing a arbitrary sized + list of arbitrary sized strings + <braunr> yes + <teythoon> list of ports is easier ;) but I think its worthwile + <teythoon> so what about abusing argz* for this? you think it's too bad a + hack? + <braunr> no since it's in glibc + <teythoon> awesome :) + <braunr> but i don't remember the details well and i'm not sure the way you + use it is safe + <teythoon> yeah, I might have got the details wrong, I hadn't had the + chance to test it ;) + + <braunr> about this dynamic size problem + <braunr> a "simple" varying size array should do + <braunr> you can easily put all your strings in there + <teythoon> seperated by 0? + <braunr> yes + <teythoon> that's exactly what the argz stuff does + <braunr> you'll get the size of the array anyway, and consume it until + there is no byte left + <braunr> good + <braunr> but be careful with this too + <braunr> since translators can be run by users, they somtimes can't be + trusted + <braunr> and even a translator running as root may behave badly + <braunr> so careful with parsing + <teythoon> noted diff --git a/open_issues/mig_stub_functions.mdwn b/open_issues/mig_stub_functions.mdwn new file mode 100644 index 00000000..24a582b1 --- /dev/null +++ b/open_issues/mig_stub_functions.mdwn @@ -0,0 +1,41 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_mig]] + +[[!toc]] + + +# RPC Stubs Implemented by Hand + +## IRC, freenode, #hurd, 2013-07-28 + + <teythoon> why is libfshelp/start-translator-long.c doing the fsys_startup + rpcs by hand instead of using the mig generated stubs? + + +## IRC, freenode, #hurd, 2013-07-29 + + <teythoon> btw, anyone knows why libfshelp/start-translator-long.c + implements the fsys_startup rpc by hand? + <braunr> teythoon: no idea + <teythoon> maybe b/c of the need to specify a timeout? can one do that with + the mig stubs? + <braunr> yes + <braunr> select used to be implemented that way + + +# Generate the Request and Reply Routines from the Synchronous Routines + +## IRC, freenode, #hurd, 2013-09-19 + + <teythoon> btw, is there any reason why mig couldn't generate the request + and reply routines from the synchronous routines? + <braunr> i guess it could diff --git a/open_issues/mondriaan_memory_protection.mdwn b/open_issues/mondriaan_memory_protection.mdwn new file mode 100644 index 00000000..2c7b9ba1 --- /dev/null +++ b/open_issues/mondriaan_memory_protection.mdwn @@ -0,0 +1,85 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +<http://scale.eecs.berkeley.edu/mondriaan/>. + + +# IRC, freenode, #hurd, 2013-07-02 + + <xscript> in any case, what I wanted to check is if current hurd support + PIE + <xscript`> I just saw samuel posted some fixes to have PIE working in hurd + <xscript`> are those included in the official image? + <youpi> sure + <youpi> it's just a trivial fixup in some address calculation code + <xscript> youpi: nice + <xscript> and does anyone know how complex would it be to implement some + hackish support to have non-overlapping virtual addresses for + applications supporting PIE? + <braunr> not too difficult + <xscript> really? I didn't expect such an answer XD + <xscript> I'd like to have something similar to a SASOS + <xscript> (single address space os) + <braunr> ? + <braunr> you mean an sasos on top of mach ? + <xscript> yes, but only for a few apps I want to evaluate + <braunr> i see + <xscript> the optimal would be to have all of hurd's servers on that mode + <braunr> you'l probably need to implement a small allocator but other than + that it shouldn't be too hard, yes + <braunr> uh ?? + <xscript> but running on 32 bits can be a problem here + <braunr> and not hurdish at all + <xscript> what's not hurdish? + <braunr> we do want address space separation + <xscript> well, you can have multiple address spaces (page tables), but + without overlapping addresses between them + <xscript> that's exactly what I'm looking for + <braunr> sorry i don't see what you mean + <braunr> if you run several servers in the same address space, they can + corrupt each other + <braunr> we don't want that + <braunr> it's that simple + <xscript> yes, sorry, I didn't explain myself + <xscript> I want a separate address space on each server + <xscript> but I want all memory allocations to be on addresses unique to + the whole OS + <braunr> that still doesn't make sense + <xscript> well, it will still be secure + <xscript> but I know it does not make sense per se + <xscript> I want to do some experiments with a simulator + <braunr> why do you want them non overlapping if they're separate ? + <xscript> well, in my simulator I wouldn't need to change the page tables, + protection is provided through other means + <braunr> segmentation ? + <xscript> that's one possibility + <xscript> (small address spaces) + <braunr> what do you have in mind ? + <braunr> it wouldn't be on top of mach anyway then + <braunr> mach implements paging + <xscript> what I'm simulating is something of the likes of Mondriaan + (http://www.cs.utexas.edu/~witchel/pubs/mmp-asplos2002.pdf) + <xscript> paging is ok for me + <braunr> 19:06 < xscript> well, in my simulator I wouldn't need to change + the page tables, protection is provided through other means + <braunr> it didn't sound so + <xscript> I meant switching page tables (cr3, etc) + <braunr> mach does that + <xscript> I know, I know, I can just ignore that part for the moment + <braunr> ok + <xscript> for now, I'd like to morph hurd into a SASOS using one page table + per process + <xscript> I just wanted to know how hard that would be, without starting + with a full dive into the code + <xscript> there are other options (OSes, microkernels), but none of them + provides as many readily-available applications as hurd + <xscript> I suppose MINIX would also be easy to modify, but there's less + apps there, and I also would like to tamper with MIG + <xscript> I just wonder how hard it would be to modify MIG diff --git a/open_issues/nightly_builds.mdwn b/open_issues/nightly_builds.mdwn index 167e7375..96567685 100644 --- a/open_issues/nightly_builds.mdwn +++ b/open_issues/nightly_builds.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010, 2011, 2012 Free Software Foundation, +[[!meta copyright="Copyright © 2010, 2011, 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -25,8 +25,13 @@ Resources: * <http://hudson-ci.org/>, <http://jenkins-ci.org/> + * [[!message-id "201308251648.38010.holger@layer-acht.org"]] + * <http://buildbot.net/> + * [LAVA (Linaro Automated Validation + Architecture)](http://lava.readthedocs.org/) + --- See also [[nightly_builds_deb_packages]]. diff --git a/open_issues/nptl.mdwn b/open_issues/nptl.mdwn index 9ff5fb51..3c84bfb0 100644 --- a/open_issues/nptl.mdwn +++ b/open_issues/nptl.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,7 +10,10 @@ License|/fdl]]."]]"""]] [[!tag open_issue_libpthread open_issue_glibc]] -IRC, #hurd, 2010-07-31 +[[!toc]] + + +# IRC, freenode, #hurd, 2010-07-31 <tschwinge> Other question: how difficult is a NPTL port? Futexes and some kernel interfaces for scheduling stuff etc. -- what else? <youpi> actually NPTL doesn't _require_ futexes @@ -26,6 +29,21 @@ IRC, #hurd, 2010-07-31 <tschwinge> ... and even less so the interfavce that actual applications are using. <tschwinge> We'd need to evaluate which benefits NPTL would bring. + +# IRC, freenode, #hurd, 2013-08-05 + + <gnu_srs> Hi, looks like kfreebsd are now using an NPTL-based pthread + library: FBTL, http://lists.debian.org/debian-bsd/2013/07/msg00060.html + <gnu_srs> Anything of interest for porting to Hurd? See also + http://lists.debian.org/debian-hurd/2013/08/msg00000.html + <azeem> Petr could've been more verbose in his announcements + <pinotree> and there's + http://www.gnu.org/software/hurd/open_issues/nptl.html in our wiki + <azeem> well, it seems to work fine for kFreeBSD: + http://lists.debian.org/debian-bsd/2013/07/msg00134.html + <azeem> and http://lists.debian.org/debian-bsd/2013/07/msg00138.html + + --- # Resources diff --git a/open_issues/open_symlink.mdwn b/open_issues/open_symlink.mdwn index 20e4a4fe..663bfcbd 100644 --- a/open_issues/open_symlink.mdwn +++ b/open_issues/open_symlink.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,9 +10,21 @@ License|/fdl]]."]]"""]] [[!tag open_issue_glibc]] + # IRC, freenode, #hurd, 2012-01-02 <pinotree> hm, is it a known issue that open("somesymlink", O_RDONLY | O_NOFOLLOW) does not fail with ELOOP? <youpi> pinotree: iirc there is code for it, maybe not the same behavior as on linux + + +## IRC, OFTC, #debian-hurd, 2013-05-08 + + <pinotree> the hurd issue is that O_NOFOLLOW seems broken on symlinks, and + thus open(symlink, O_NOFOLLOW) doesn't fail with ELOOP + <youpi> I don't really see why it should fail + <youpi> since NOFOLLOW says not to follow the symlink + <pinotree> yeah, but you cannot open a symlink + <youpi> ah right ok + <youpi> interesting :) diff --git a/open_issues/profiling.mdwn b/open_issues/profiling.mdwn index 26e6c97c..545edcf6 100644 --- a/open_issues/profiling.mdwn +++ b/open_issues/profiling.mdwn @@ -9,10 +9,14 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] +[[!meta title="Profiling, Tracing"]] + *Profiling* ([[!wikipedia Profiling_(computer_programming) desc="Wikipedia article"]]) is a tool for tracing where CPU time is spent. This is usually done for [[performance analysis|performance]] reasons. + * [[hurd/debugging/rpctrace]] + * [[gprof]] Should be working, but some issues have been reported, regarding GCC spec @@ -33,3 +37,104 @@ done for [[performance analysis|performance]] reasons. * [[SystemTap]] * ... or some other Linux thing. + + +# IRC, freenode, #hurd, 2013-06-17 + + <congzhang> is that possible we develop rpc msg analyse tool? make it clear + view system at different level? + <congzhang> hurd was dynamic system, how can we just read log line by line + <kilobug> congzhang: well, you can use rpctrace and then analyze the logs, + but rpctrace is quite intrusive and will slow down things (like strace or + similar) + <kilobug> congzhang: I don't know if a low-overhead solution could be made + or not + <congzhang> that's the problem + <congzhang> when real system run, the msg cross different server, and then + the debug action should not intrusive the process itself + <congzhang> we observe the system and analyse os + <congzhang> when rms choose microkernel, it's expect to accelerate the + progress, but not + <congzhang> microkernel make debug a litter hard + <kilobug> well, it's not limited to microkernels, debugging/tracing is + intrusive and slow things down, it's an universal law of compsci + <kilobug> no, it makes debugging easier + <congzhang> I don't think so + <kilobug> you can gdb the various services (like ext2fs or pfinet) more + easily + <kilobug> and rpctrace isn't any worse than strace + <congzhang> how easy when debug lpc + <kilobug> lpc ? + <congzhang> because cross context + <congzhang> classic function call + <congzhang> when find the bug source, I don't care performance, I wan't to + know it's right or wrong by design, If it work as I expect + <congzhang> I optimize it latter + <congzhang> I have an idea, but don't know weather it's usefull or not + <braunr> rpctrace is a lot less instrusive than ptrace based tools + <braunr> congzhang: debugging is not made hard by the design choice, but by + implementation details + <braunr> as a simple counter example, someone often cited usb development + on l3 being made a lot easier than on a monolithic kernel + <congzhang> Collect the trace information first, and then layout the msg by + graph, when something wrong, I focus the trouble rpc, and found what + happen around + <braunr> "by graph" ? + <congzhang> yes + <congzhang> braunr: directed graph or something similar + <braunr> and not caring about performance when debugging is actually stupid + <braunr> i've seen it on many occasions, people not being able to use + debugging tools because they were far too inefficient and slow + <braunr> why a graph ? + <braunr> what you want is the complete trace, taking into account cross + address space boundaries + <congzhang> yes + <braunr> well it's linear + <braunr> switching server + <congzhang> by independent process view it's linear + <congzhang> it's linear on cpu's view too + <congzhang> yes, I need complete trace, and dynamic control at microkernel + level + <congzhang> os, if server crash, and then I know what's other doing, from + the graph + <congzhang> graph needn't to be one, if the are not connect together, time + sort them + <congzhang> when hurd was complete ok, some tools may be help too + <braunr> i don't get what you want on that graph + <congzhang> sorry, I need a context + <congzhang> like uml sequence diagram, I need what happen one by one + <congzhang> from server's view and from the function's view + <braunr> that's still linear + <braunr> so please stop using the word graph + <braunr> you want a trace + <braunr> a simple call trace + <congzhang> yes, and a tool + <braunr> with some work gdb could do it + <congzhang> you mean under some microkernel infrastructure help + <congzhang> ? + <braunr> if needed + <congzhang> braunr: will that be easy? + <braunr> not too hard + <braunr> i've had this idea for a long time actually + <braunr> another reason i insist on migrating threads (or rather, binding + server and client threads) + <congzhang> braunr: that's great + <braunr> the current problem we have when using gdb is that we don't know + which server thread is handling the request of which client + <braunr> we can guess it + <braunr> but it's not always obvious + <congzhang> I read the talk, know some of your idea + <congzhang> make things happen like classic kernel, just from function + ,sure:) + <braunr> that's it + <congzhang> I think you and other do a lot of work to improve the mach and + hurd, buT we lack the design document and the diagram, one diagram was + great than one thousand words + <braunr> diagrams are made after the prototypes that prove they're doable + <braunr> i'm not a researcher + <braunr> and we have little time + <braunr> the prototype is the true spec + <congzhang> that's why i wan't cllector the trace info and show, you can + know what happen and how happen, maybe just suitable for newbie, hope + more young hack like it + <braunr> once it's done, everything else is just sugar candy around it diff --git a/open_issues/pthread_atfork.mdwn b/open_issues/pthread_atfork.mdwn index ac724cf0..1b656f05 100644 --- a/open_issues/pthread_atfork.mdwn +++ b/open_issues/pthread_atfork.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -8,6 +8,13 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!tag open_issue_libpthread]] +[[!tag open_issue_glibc open_issue_libpthread]] -pthread_atfork is not actually implemented, making some programs fail. Code can probably be borrowed from nptl/sysdeps/unix/sysv/linux/register-atfork.c +`pthread_atfork` is not actually implemented, making some programs fail. Code +can probably be borrowed from `nptl/sysdeps/unix/sysv/linux/register-atfork.c`. + + +# IRC, OFTC, #debian-hurd, 2013-08-21 + + <pinotree> SRCDIR/opal/mca/memory/linux/arena.c:387: warning: warning: + pthread_atfork is not implemented and will always fail diff --git a/open_issues/resource_management_problems.mdwn b/open_issues/resource_management_problems.mdwn index 8f752d61..daf97954 100644 --- a/open_issues/resource_management_problems.mdwn +++ b/open_issues/resource_management_problems.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, +[[!meta copyright="Copyright © 2008, 2009, 2010, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -61,7 +61,8 @@ This is, of course, non-trivial to implement, and also requires changing the SPLICE_F_GIFT flag](http://www.kernel.org/doc/man-pages/online/pages/man2/vmsplice.2.html#DESCRIPTION).) -IRC, freenode, #hurd, 2011-07-31 + +## IRC, freenode, #hurd, 2011-07-31 < braunr> one of the biggest problems on the hurd is that, when a client makes a call, kernel (and other) resources are allocated on behalf of the @@ -75,6 +76,20 @@ IRC, freenode, #hurd, 2011-07-31 attempts) +## IRC, freenode, #hurd, 2013-08-13 + +In context of <https://teythoon.cryptobitch.de/posts/my-worst-week-yet/>. + + <braunr> teythoon: actually, thread migration isn't required for resource + accounting + +[[Mach_migrating_threads]]. + + <teythoon> braunr: but it solves it for free, doesn't it? + <braunr> teythoon: no + <braunr> it's really more complicated than that + + # Further Examples * [[hurd/critique]] @@ -83,4 +98,34 @@ IRC, freenode, #hurd, 2011-07-31 * [[translators_set_up_by_untrusted_users]], and [[pagers]] - * [[configure max command line length]] + * [[configure_max_command_line_length]] + + +## [[hurd/translator/exec]] server + +### IRC, freenode, #hurd, 2013-08-05 + + <teythoon> unzipping stuff in the exec server enables a dos on filesystem + translators + <teythoon> https://teythoon.cryptobitch.de/gsoc/heap/hello-1g.bz2 is + /hurd/hello padded with a gig of zeros, compressed with bzip2 + <teythoon> if set as an passive translator, it stalls other requests to the + filesystem, at least it does if ext2fs is used + <braunr> teythoon: ? + <braunr> teythoon: what's the dos here ? + <teythoon> I can prevent you from doing anything with the root filesystem + <teythoon> I'm kind of surprised myself, maybe a lock is held during the + exec of the translator? + <teythoon> the filesystem the hello-1g.bz2 translator is bound to is + affected + <braunr> teythoon: i don't understand + <braunr> have you tried starting something from another file system ? + <braunr> the lock may simply be in the exec server itself + <teythoon> no, starting other things works fine + <teythoon> but on the other hand, a find / is stalled + <braunr> :/ + <braunr> *sigh* + <teythoon> don't worry + <teythoon> there is a solution :p + <braunr> :) + <teythoon> and it only requires deleting code diff --git a/open_issues/robustness.mdwn b/open_issues/robustness.mdwn index 1f8aa0c6..a6b0dbfb 100644 --- a/open_issues/robustness.mdwn +++ b/open_issues/robustness.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,6 +10,7 @@ License|/fdl]]."]]"""]] [[!tag open_issue_documentation open_issue_hurd]] +[[!toc]] # IRC, freenode, #hurd, 2011-11-18 @@ -32,7 +33,9 @@ License|/fdl]]."]]"""]] <etenil> ah yeah I thought so :) -# IRC, freenode, #hurd, 2011-11-19 +# Reincarnation Server + +## IRC, freenode, #hurd, 2011-11-19 <chromaticwt> will hurd ever have the equivalent of a rs server?, is that even possible with hurd? @@ -127,3 +130,40 @@ License|/fdl]]."]]"""]] <spiderweb> neat, thanks <braunr> actually it's not that old at all <braunr> around 2007 + + +## IRC, freenode, #hurd, 2013-08-26 + + < teythoon> I came across some paper about process reincarnation and + created a little prototype a while back: + < teythoon> http://darnassus.sceen.net/gitweb/teythoon/reincarnation.git/ + < teythoon> and I looked into restarting the exec server in case it + dies. the exec server is an easy target since it has no state of its own + < teythoon> the only problem is that there is no exec server around to + start a new one + < youpi> teythoon: there could be another exec server only used to + (re)start the exec server + < youpi> that other exec server could even be restarted by the normal exec + server + < pinotree> what about a watchdog server? + < teythoon> youpi: yes, I had the same idea, i actually patched /hurd/init + to do that, it's just not yet working + < pinotree> make it watch other servers (exec included), and make exec + watch the watchdog only + < teythoon> pinotree: look at my prototype, there is a watchdog server + < braunr> teythoon: what's the point of reincarnation without persistence ? + < teythoon> braunr: there is no point in reincarnation w/o persistence of + course + < teythoon> my prototype does a limited form of persistence + < teythoon> the point was to see whether I can mitm a translator and + restart it on demand and to gain more insight into the whole translator + mechanism + < braunr> teythoon: ok + < teythoon> braunr: see the readme, it retains state across reincarnations + < braunr> teythoon: how ? + < teythoon> braunr: the server can store a checkpoint using the + reincarnation_checkpoint procedure + < teythoon> + http://darnassus.sceen.net/gitweb/teythoon/reincarnation.git/blame/HEAD:/reincarnation.defshttp://darnassus.sceen.net/gitweb/teythoon/reincarnation.git/blame/HEAD:/reincarnation.defs + < teythoon> uh >,< sorry, pasted twice + < braunr> oh ok diff --git a/open_issues/secure_file_descriptor_handling.mdwn b/open_issues/secure_file_descriptor_handling.mdwn index 45e983a7..16c8c85c 100644 --- a/open_issues/secure_file_descriptor_handling.mdwn +++ b/open_issues/secure_file_descriptor_handling.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -16,7 +17,10 @@ on this, posted patches to [[mailing_lists/libc-alpha]]. This works needs to be resumed and finished. ---- +Add tests from Linux kernel commit messages for `t/dup3` et al. + +Validate completeness according to <https://wiki.freebsd.org/AtomicCloseOnExec> +or a similar list. In <http://lwn.net/Articles/417421/> an interesting point is made: *you [may] want some [[unix/file_descriptor]] to still be open if 'exec' fails, but you diff --git a/open_issues/sendmsg_scm_creds.mdwn b/open_issues/sendmsg_scm_creds.mdwn index cf0103df..d4a6126e 100644 --- a/open_issues/sendmsg_scm_creds.mdwn +++ b/open_issues/sendmsg_scm_creds.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010, 2011, 2012 Free Software Foundation, +[[!meta copyright="Copyright © 2010, 2011, 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -11,7 +11,8 @@ License|/fdl]]."]]"""]] [[!tag open_issue_glibc]] -IRC, unknown channel, unknown date. + +# IRC, unknown channel, unknown date <pinotree> Credentials: s_uid 1000, c_uid 1000, c_gid 100, c_pid 2722 <pinotree> 2722: Credentials: s_uid 1000, c_uid 1000, c_gid 100, c_pid 2724 @@ -91,10 +92,80 @@ IRC, unknown channel, unknown date. <pinotree> yep <youpi> ok, good :) -/!\ IRC, freenode, #hurd, 2011-08-11 + +## IRC, freenode, #hurd, 2011-08-11 < pinotree> (but that patch is lame) + +## IRC, freenode, #hurd, 2013-05-09 + + <gnu_srs> youpi: Since you are online tonight, which authentication + callbacks to be used for SCM_CREDS calls. + <gnu_srs> I have working code and need to add this to make things + complete. The auth server, lib* or where? + <youpi> I don't understand the question + <gnu_srs> authentication callbacks like for SCM_RIGHTS, see + <gnu_srs> + http://www.gnu.org/software/hurd/open_issues/sendmsg_scm_creds.html + <youpi> I still don't understand: what are you trying to do actually? + <gnu_srs> solving the SCM_CREDS propbems with e.g. dbus. + <youpi> so what is the relation with pinotree's patch on the page above? + <youpi> (I have no idea of the current status of all that) + <gnu_srs> his patch was not merged, right? have to shut down, sorry, bbl, + gn8 + <pinotree> that patch was not merged since it is not in the correct place + <youpi> as I said, I have no idea about the status + <pinotree> youpi: basically, it boils down to knowing, when executing the + code implementing an rpc, who requested that rpc (pid, uid, gid) + <youpi> i.e. getting information about the reply port for instance? + <youpi> well that might be somehow faked + <youpi> (by perhaps giving another task's port as reply port) + <pinotree> for example (which would be the code path for SCM_CREDS), when + you call call the socket sendmsg(), pflocal would know who did that rpc + and fill the auxilliary data) + <pinotree> s,)$,, + <pinotree> youpi: yes, i know about this faking issue, iirc also antrik + mentioned quite some time ago + <youpi> ok + <pinotree> that's one of the (imho) two issues of this + <pinotree> my hurd-foo is not enough to know whether there are solutions to + the problem above + + +### IRC, freenode, #hurd, 2013-05-14 + + <gnu_srs> Hi, regarding SCM_CREDS, I have some working code in + sendmsg.c. Now I need to make a callback to authenticate the pid, uid, + etc + <gnu_srs> Where to hook call that into pflocal? + <gnu_srs> the auth server? + <gnu_srs> maybe _io_restrict_auth is the correct call to use (same as for + SCM_RIGHTS)? + + +### IRC, freenode, #hurd, 2013-05-17 + + <gnu_srs> I'm working on the scm credentials right now to enable (via dbus) + more X window managers to work properly. + <gnu_srs> seems to be rather tricky:-( + <pochu> gnu_srs: I guess you also need SCM_CREDS, right? + <gnu_srs> hi pochu, that's what I'm working on, extending your SCM_RIGHTS + work to SCM_CREDS + <pinotree> that's what i did as proof, years ago? + <gnu_srs> it would be good to know which server calls to make, I'll be back + with proposals of functions to use. + <pinotree> there was a talk, years ago when i started with this, and few + days ago too + <pinotree> every methods has its own drawbacks, and basically so far it + seems that in every method the sender identity can be faked somehow + <gnu_srs> pinotree: Yes of course your patch was perfect, but it seemed + like people wanted a server acknowledgement too. + <pinotree> no, my patch was not perfect at all + <pinotree> if it was, it would have been cleaned up and sent few years ago + already + + --- See also [[dbus]], [[pflocal_socket_credentials_for_local_sockets]] and diff --git a/open_issues/some_todo_list.mdwn b/open_issues/some_todo_list.mdwn index 80592abf..48c2944d 100644 --- a/open_issues/some_todo_list.mdwn +++ b/open_issues/some_todo_list.mdwn @@ -42,6 +42,21 @@ From Marcus, 2002: * Translators * Does settrans -g work? -fg? * Does fsysopts work? Does setting options with fsysopts work? + + IRC, freenode, #hurd, 2013-05-23: + + <gnu_srs> fsysopts /servers/socket/2 works by /1 gives Operation not + supported. + + [[!taglink open_issue_hurd]]. + + <braunr> ah right, some servers don't implement that + <braunr> work around this by using showtrans + <braunr> fsysopts asks the server itself how it's running, usually giving + its command name and options + <braunr> showtrans asks the parent how it starts a passive translator + attached to the node + <gnu_srs> Yes showtrans works :), thanks. * Does stat() work on all translated nodes and give proper data? * What about chown, chmod (some translators should pass this through to the underlying node, esp in /dev!) * Does statfs give correct data? diff --git a/open_issues/systemd.mdwn b/open_issues/systemd.mdwn index c23f887f..d00b3d8a 100644 --- a/open_issues/systemd.mdwn +++ b/open_issues/systemd.mdwn @@ -102,6 +102,939 @@ Likely there's also some other porting needed. <braunr> just assume you can't use systemd on anything else than linux +## IRC, OFTC, #debian-hurd, 2013-08-12 + + <azeem> huh, Lennert Poettering just mentioned the Hurd in his systmd talk + <azeem> well, in the context of you IPC in Unix sucks and kdbus + <azeem> s/you/how/ + <pinotree> QED + <pinotree> what did you expect? :) + <azeem> I didn't quite get it, but he seemed to imply the Hurd was a step + in the right direction over Unix + <azeem> (which is obvious, but it wasn't obvious he had that opinion) + + +## IRC, OFTC, #debian-hurd, 2013-08-13 + + <azeem> so cgroups seems to be most prominent thing the systemd people + think the Hurd lacks + <tschwinge> azeem: In 2010, I came to the same conclusion, + <http://www.gnu.org/software/hurd/open_issues/systemd.html>. ;-) + <azeem> heh + <tschwinge> I don't think of any show-stopper for implementing that -- just + someone to do it. + <youpi> azeem: which part of cgroups, like being able to kill a cgroup? + <youpi> it shouldn't be very hard to implement what systemd needs + <azeem> probably also the resource allocation etc. + <azeem> the questions are I guess (i) do the cgroups semantics make sense + from our POV and/or do we accept that cgroups is the "standard" now and + (ii) should systemd require concrete implementations or just the concept + in a more abstract sense + <teythoon> being the first non Linux OS that runs systemd would be a nice + showcase of Hurds flexibility + <azeem> maybe upstart is less trouble + <pinotree> azeem: possibly + <azeem> teythoon: can you just include upstart in your GSOC? kthxbye + <pinotree> at least libnih (the library with base utilities and such used + by upstart) required a working file monitor (and the current + implementation kind of exposes a fd) and certain semantics for waitid + <pinotree> libnih/upstart have "just" the issue of being under CLA... + <azeem> pinotree: yeah, true + <azeem> I suggested "startup" as a name for a fork + <pinotree> imho there would be no strict need to fork + <teythoon> azeem: but upstart is a lot less interesting. last time I used + it it wasn't even possible to disable services in a clean way + <pochu> pinotree: is that still so now that Scott works for google? + <pinotree> pochu: yeah, since it's a Canonical CLA, not rally something + tied to a person + <pinotree> (iirc) + <pochu> sure, but scott is the maintainer... + <pochu> shrug + <azeem> nah, scott left upstart + <azeem> AFAIK + <azeem> at least James Hunt gave a talk earlier with Steve Langasek and + introduced himself as the upstart maintainer + <azeem> also I heard in the hallway track that the upstart people are + somewhat interested in BSD/Hurd support as they see it as a selling point + against systemd + <pinotree> pochu: it's just like FSF CLA for GNU projects: even if the + maintainers/contributors change altogether, copyright assignment is still + FSF + <azeem> but their accents were kinda annoying/hard to follow so I didn't + follow their talk closesly to see whether they brought it up + <azeem> pinotree: well, it's not + <pochu> azeem: looking at https://code.launchpad.net/libnih, I'm not sure + libnih has a maintainer anymore... + <azeem> pinotree: first off, you're not signing over the copyright with + their CLA, just giving them the right to relicense + <azeem> pinotree: but more importantaly, the FSF announced in a legally + binding way that they will not take things non-free + <azeem> anyway, I'll talk to the upstart guys about libnih + + +## IRC, OFTC, #debian-hurd, 2013-08-15 + + <azeem> btw, I talked to vorlon about upstart and the Hurd + <azeem> so the situation with libnih is that it is basically + feature-complete, but still maintained by Scott + <azeem> upstart is leveraging it heavily + <azeem> and Scott was (back in the days) against patches for porting + <azeem> for upstart proper, Steve said he would happily take porting + patches + + +## IRC, freenode, #hurd, 2013-08-26 + + < youpi> teythoon: I tend to agree with mbanck + < youpi> although another thing worth considering would be adding something + similar to control groups + < youpi> AIUI, it's one of the features that systemd really requires + < braunr> uhg, cgroups already + < braunr> youpi: where is that discussion ? + < youpi> it was a private mail + < braunr> oh ok + < teythoon> right, so about upstart + < teythoon> to be blunt, I do not like upstart, though my experience with + it is limited and outdated + < braunr> that was quick :) + < braunr> i assume this follows your private discussion with youpi and + mbank ? + < teythoon> I used it on a like three years old ubuntu and back then it + couldn't do stufft hat even sysvinit could do + < teythoon> there was not much discussion, mbank suggested that I could + work on upstart + < teythoon> b/c it might be easier to support than systemd + < teythoon> which might be very well true, then again what's the benefit of + having upstart? I'm really curious, I should perhaps read up on its + features + < pinotree> event-based, etc + < youpi> it is also about avoiding being pushed out just because we don't + support it? + < teythoon> yes, but otoh systemd can do amazing things, the featurelist of + upstart reads rather mondane in comparison + < youpi> I don't really have an opinion over either, apart from portability + of the code + < braunr> teythoon: the system requirements for systemd would take much + time to implement in comparison to what we already have + < braunr> i still have maksym's work on last year gsoc on my list + < braunr> waiting to push in the various libpager related patches first + < teythoon> so you guys think it's worthwile to port upstart? + < braunr> no idea + < braunr> teythoon: on another subject + < azeem_> teythoon: I like systemd more, but the hallway track at Debconf + seemed to imply most people like Upstart better except for the CLA + < azeem_> which I totally forgot to address + < youpi> CLA ? + < azeem_> contributor license agreement + < braunr> since you've now done very good progress, is your work available + in the form of ready-to-test debian packages ? + < teythoon> braunr: it is + < teythoon> braunr: http://teythoon.cryptobitch.de/gsoc/heap/debian/ + < braunr> i remember urls in some of your mails + < braunr> ah thanks + < braunr> "cryptobitch" hum :) + < azeem_> in any case, everbody assumed either Upstart or Systemd are way + ahead of systemvinit + < braunr> sysvinit is really ancient :) + < azeem_> apart from the non-event-driven fundamental issue, a lot of + people critized that the failure rate at writing correct init-scripts + appears to be too high + < azeem_> one of the questions brought up was whether it makes sense to + continue to ship/support systemvinit once a switch is made to + systemd/upstart for the Linux archs + < azeem_> systemvinit scripts might bitrot + < azeem_> but anyway, I don't see a switch happen anytime soon + < teythoon> well, did upstart gain the capability of disabling a service + yet? + < azeem_> teythoon: no idea, but apparently: + http://askubuntu.com/questions/19320/recommended-way-to-enable-disable-services/20347#20347 + < teythoon> azeem_: then there is hope yet ;) + < azeem_> the main selling point of Upstart is that it shipped in several + LTS releases and is proven technology (and honestly, I don't read a lot + of complaints online about it) + < azeem_> (I don't agree that SystemD is unproven, but that is what the + Upstart guys implied) + < teythoon> am I the only one that thinks that upstart is rather + unimpressive? + * azeem_ doesn't have an opinion on it + < azeem> teythoon: + http://penta.debconf.org/dc13_schedule/events/1027.en.html has slides and + the video + < azeem> teythoon: eh, appears the link to the slides is broken, but they + are here: + http://people.canonical.com/~jhunt/presentations/debconf13/upstart-debconf-2013.pdf + < braunr> teythoon: actually, from the presentation, i'd tend to like + upstart + < braunr> dependency, parallelism and even runlevel compatibility flows + naturally from the event based model + < braunr> sysv compatibility is a great feature + < braunr> it does look simple + < braunr> i admit it's "unimpressive" but do we want an overkill init + system ? + < braunr> teythoon: what makes you not like it ? + < azeem> Lennart critized that upstart doesn't generate events, just + listens to them + < azeem> (which is a feature, not a bug to some) + < braunr> azeem: ah yes, that could be a lack + < azeem> braunr: http://penta.debconf.org/dc13_schedule/events/983.en.html + was the corresponding SystemD talk by Lennart, though he hasn't posted + slides yet I think + < teythoon> braunr: well, last time I used it it was impossible to cleanly + disable a service + < teythoon> also ubuntu makes such big claims about software they develop, + and when you read up on them it turns out that most of the advertised + functionality will be implemented in the near future + < teythoon> then they ship software as early as possible only to say later + that is has proven itself for so many years + < teythoon> and tbh I hate to be the one that helped port upstart to hurd + (and maybe kfreebsd as a byproduct) and later debian choses upstart over + systemd b/c it is available for all debian kernels + < kilobug> teythoon: ubuntu has a tendency to ship software too early when + it's not fully mature/stable, but that doesn't say anything about the + software itself + < pinotree> teythoon: note the same is sometimes done on fedora for young + technologies (eg systemd) + < azeem> teythoon: heh, fair enough + < p2-mate> braunr: I would prefer if my init doesn't use ptrace :P + < teythoon> p2-mate: does upstart use ptrace? + < p2-mate> teythoon: yes + < teythoon> well, then I guess there won't be an upstart for Hurd for some + time, no? + < kilobug> p2-mate: why does it use ptrace for ? + < p2-mate> kilobug: to find out if a daemon forked + < kilobug> hum I see + < azeem> p2-mate: the question is whether there's a Hurdish way to + accomplish the same + < p2-mate> + http://bazaar.launchpad.net/~upstart-devel/upstart/trunk/view/head:/init/job_process.c + < p2-mate> see job_process_trace_new :) + < kilobug> azeem: it doesn't seem too complicated to me to have a way to + get proc notify upstart of forks + < p2-mate> azeem: that's a good question. there is a linuxish way to do + that using cgroups + < azeem> right, there's a blueprint suggesting cgroups for Upstart here: + https://blueprints.launchpad.net/ubuntu/+spec/foundations-q-upstart-overcome-ptrace-limitations + < teythoon> yes, someone should create a init system that uses cgroups for + tracking child processes >,< + < teythoon> kilobug: not sure it is that easy. who enforces that proc_child + is used for a new process? isn't it possible to just create a new mach + task that has no ties to the parent process? + < teythoon> azeem: what do you mean by "upstart does not generate events"? + there are "emits X" lines in upstart service descrpitions, surely that + generates event X? + < azeem> I think the critique is that this (and those upstart-foo-bridges) + are bolted on, while SystemD just takes over your systems and "knows" + about them first-hand + < azeem> but as I said, I'm not the expert on this + < teythoon> uh, in order to install upstart one has to remove sysvinit + ("yes i am sure...") and it fails to bring up the network on booting the + machine + < teythoon> also, both systemd and upstart depend on dbus, so no cookie for + us unless that is fixed first, right? + < pinotree> true + < teythoon> well, what do you want me to do for the next four weeks? + < youpi> ideally you could make both upstart and systemd work on hurd-i"86 + < pinotree> both in 4 weeks? + < youpi> so hurd-i386 doesn't become the nasty guy that makes people tend + for one or the other + < youpi> I said "ideally" + < youpi> I don't really have any idea how much work is required by either + of the two + < youpi> I'd tend to think the important thing to implement is something + similar to control groups, so both upstart (which is supposed to use them + someday) and systemd can be happy about it + < teythoon> looks like upstarts functionality depending on ptrace is not + required, but can be enabled on a per service base + < teythoon> so a upstart port that just lacks this might be possible + < teythoon> youpi: the main feature of cgroups is that a process cannot + escape its group, no? i'm not sure how this could be implemented atop of + mach in a secure and robust way + < teythoon> b/c any process can just create mach tasks + < youpi> maybe we need to add a feature in mach itself, yes + < teythoon> ok, implementing cgroups sounds fun, I could do that + < youpi> azeem: are you ok with that direction? + < azeem> well, in general yes; however, AIUI, cgroups is being redesigned + upstream, no? + < youpi> that's why I said "something like cgroups" + < azeem> ah, ok + < youpi> we can do something simple enough to avoid design quesetions, and + that would still be enough for upstart & systemd + < azeem> + (http://www.linux.com/news/featured-blogs/200-libby-clark/733595-all-about-the-linux-kernel-cgroups-redesign) + btw + < braunr> p2-mate: upstart uses ptrace ? + < p2-mate> yes + < youpi> teythoon: and making a real survey of what needs to be fixed for + upstart & systemd + < p2-mate> see my link posted earlier + < braunr> ah already answered + < braunr> grmbl + < braunr> it's a simple alternative to cgroups though + < braunr> teythoon: dbus isn't a proble + < braunr> problem + < braunr> it's not that hard to fix + < youpi> well, it hasn't been fixed for a long time now :) + < braunr> we're being slow, that's all + < braunr> and interested by other things + < gg0> 12:58 < teythoon> btw, who is this heroxbd fellow and why has he + suddenly taken interest in so many debian gsoc projects? + < gg0> http://lists.debian.org/debian-hurd/2013/05/msg00133.html + < gg0> i notice nobody mentioned openrc + < pinotree> he's the debian student working on integrating openrc + < gg0> pinotree: no, the student is Bill Wang, Benda as he says is a + co-mentor + https://wiki.debian.org/SummerOfCode2013/Projects#OpenRC_init_system_in_Debian + < pinotree> whatever, it's still the openrc gsoc + < azeem> well, they wanted to look at it WRT the Hurd, did they follow-up + on this? + < gg0> btw wouldn't having openrc on hurd be interesting too? + < pinotree> imho not really + < gg0> no idea whether Bill is also trying to figure out what to do, + probably not + < azeem> somebody could ping that thread you mentioned above to see whether + they looked at the Hurd and/or need help/advice + < gg0> azeem: yeah somebody who could provide such help/advice. like.. you? + for instance + * gg0 can just paste urls + < azeem> they should just follow-up on-list + + +## IRC, freenode, #hurd, 2013-08-28 + + <teythoon> anyone knows a user of cgroups that is not systemd? so far I + found libcg, that looks like a promising first target to port first, + though not surprisingly it is also somewhat linux specific + <taylanub> teythoon: OpenRC optionally uses cgroups IIRC. + <taylanub> Not mandatory because unlike systemd it actually tries (at all) + to be portable. + + +## IRC, freenode, #hurd, 2013-09-02 + + <teythoon> braunr: I plan to patch gnumach so that the mach tasks keep a + reference to the task that created them and to make that information + available + <teythoon> braunr: is such a change acceptable? + <braunr> teythoon: what for ? + <teythoon> braunr: well, the parent relation is currently only implemented + in the Hurd, but w/o this information tracked by the kernel I don't see + how I can prevent malicious/misbehaving applications to break out of + cgroups + <teythoon> also I think this will enable us to fix the issue with tracking + which tasks belong to which subhurd in the long term + <braunr> ah cgroups + <braunr> yes cgroups should partly be implemented in the kernel ... + <braunr> teythoon: that doesn't surprise me + <braunr> i mean, i think it's ok + <braunr> the kernel should implement tasks and threads as closely as the + hurd (or a unix-like system) needs it + <teythoon> braunr: ok, cool + <teythoon> braunr: I made some rather small and straight forward changes to + gnumach, but it isn't doing what I thought it would do :/ + <teythoon> braunr: http://paste.debian.net/33717/ + <braunr> you added a field to task_basic_info + <braunr> thereby breaking the ABI + <teythoon> braunr: my small test program says: my task port is 1(pid 13) + created by task -527895648; my parent task is 31(pid 1) + <teythoon> braunr: no, it is not. I appended a field and these structures + are designed to be extendable + <braunr> hm + <braunr> ok + <braunr> although i'm not so sure + <braunr> there are macros defining the info size, depending on what you ask + <braunr> you may as well get garbage + <braunr> have you checked that ? + <teythoon> i initialized my struct to zero before calling mach + <braunr> teythoon: can you put some hardcoded value, just to make sure data + is correctly exported ? + <teythoon> braunr: right, good idea + <teythoon> braunr: my task port is 1(pid 13) created by task 3; my parent + task is 31(pid 1) -- so yes, hardcoding 3 works + <braunr> ok + <teythoon> braunr: also I gathered evidence that the convert_task_to_port + thing works, b/c first I did not have the task_reference call just before + that so the reference count was lowered (convert... consumes a reference) + and the parent task was destroyed + <teythoon> braunr: I must admit I'm a little lost. I tried to return a + reference to task rather than task->parent_task, but that didn't work + either + <teythoon> braunr: I feel like I'm missing something here + <teythoon> maybe I should get aquainted with the kernel debugger + <teythoon> err, the kernel debugger is not accepting any symbol names, even + though the binary is not stripped o_O + <teythoon> err, neither the kdb nor gdb attached to qemu translates + addresses to symbols, gdb at least translates symbols to addresses when + setting break points + <teythoon> how did anyone ever debug a kernel problem under these + conditions? + <braunr> teythoon: i'll have a look at it when i have some time + + +## IRC, freenode, #hurd, 2013-09-03 + + <teythoon> :/ I believe the startup_notify interface is ill designed... an + translator can defer the system shutdown indefinitely + <braunr> it can + <teythoon> that's bad + <braunr> yes + <braunr> the hurd has a general tendency to trust its "no mutual trust + required" principle + <braunr> to rely on it a bit too much + <teythoon> well, at least it's a privileged operation to request this kind + of notification, no? + <braunr> why ? + <braunr> teythoon: it normally is used mostly by privileged servers + <braunr> but i don't think there is any check on the recipient + <teythoon> braunr: b/c getting the port to /hurd/init is done via + proc_getmsgport + <braunr> teythoon: ? + <teythoon> braunr: well, in order to get the notifications one needs the + msgport of /hurd/init and getting that requires root privileges + <braunr> teythoon: oh ok then + <braunr> teythoon: what's bad with it then ? + <teythoon> braunr: even if those translators are somewhat trusted, they can + (and do) contain bugs and stall the shutdown + <teythoon> I think this even happened to me once, I think it was the pfinet + translator + <braunr> teythoon: how do you want it to behave instead ? + <teythoon> braunr: well, /hurd/init notifies the processes sequentially, + that seems suboptimal, better to send async notifications to all of them + and then to collect all the answers + <teythoon> braunr: if one fails to answer within a rather large time frame + (say 5 minutes) shutdown anyway + <braunr> i agree with async notifications but + <braunr> i don't agree with the timeout + <teythoon> for reference, a (voluntary) timeout of 1 minute is hardcoded in + /hurd/init + <braunr> the timeout should be a parameter + <braunr> it's common on large machines to have looong shutdown delays + <teythoon> of the notification? + <braunr> the answer means "ok i'm done you can shutdown" + <braunr> well this can take long + <braunr> most often, administrators simply prefer to trust their program is + ok and won't take longer than it needs to, even if it's long + <teythoon> and not answering at all causes the shutdown / reboot to fail + making the system hang + <braunr> i know + <teythoon> in a state where it is not easily reached if you do not have + access to it + <braunr> but since it only concerns essential servers, it should befine + <braunr> essential servers are expected to behave well + <teythoon> it concerns servers that have requested a shutdown notification + <braunr> ok so no essential but system servers + <teythoon> essential servers are only exec, proc, / + <teythoon> yes + <braunr> the same applies + <pinotree> init and auth too, no? + <teythoon> yes + <braunr> you expect root not to hang himself + <teythoon> I do expect all software to contain bugs + <braunr> yes but you also expect them to provide a minimum level of + reliability + <braunr> otherwise you can just throw it all away + <teythoon> no, not really + <braunr> well + <teythoon> I know, that's my dilemma basically ;) + <braunr> if you don't trust your file system, you make frequent backups + <braunr> if you don't trust your shutdown code, you're ready to pull the + plug manually + <braunr> (or set a watchdog or whatever) + <braunr> what i mean is + <braunr> we should NEVER interfere with a program that is actually doing + its job just because it seems too long + <braunr> timeouts are almost never the best solution + <braunr> they're used only when necessary + <braunr> e.g. across networks + <braunr> it's much much much worse to interrupt a proper shutdown process + because it "seems too long" than just trust it behaves well 99999%%%% of + the time + <braunr> in particular because this case deals with proper data flushing, + which is an extremely important use case + <teythoon> it's hard/theoretically impossible to distinguish between taking + long and doing nothing + <braunr> it's impossible + <braunr> agreed + <braunr> => trust + <braunr> if you don't trust, you run real time stuff + <braunr> and you don't flush data on disk + <teythoon> ^^ + <braunr> (which makes a lot of computer uses impossible as well) + <teythoon> there are only 2 people I trust, and the other one is not + /hurd/pfinet + <braunr> if this shutdown procedure is confined to the TCB, it's fine to + trust it goes well + <teythoon> tcb? + <braunr> trusted computing base + <braunr> http://en.wikipedia.org/wiki/Trusted_computing_base + * teythoon shudders + <teythoon> "trust" is used way to much these days + <teythoon> and I do not like the linux 2.0 ip stack to be part of our TCB + <braunr> basically, on a multiserver system like the hurd, the tcb is every + server on the path to getting a service done from a client + <braunr> then make it not request to be notified + <braunr> or make two classes of notifications + <braunr> because unprivileged file systems should be notified too + <teythoon> indeed + <teythoon> by the way, we should have a hurdish libnotify or something for + this kind of notifications + <braunr> but in any case, it should really be policy + <braunr> we should ... :) + <teythoon> ^^ + + +## IRC, freenode, #hurd, 2013-09-04 + + <teythoon> braunr: btw, I now believe that no server that requested + shutdown notifications can stall the shutdown for more than 1 minute + *unless* its message queue is full + <teythoon> so any fs should better sync within that timeframe + <braunr> where is this 1 min defined ? + <teythoon> init/init.c search for 60000 + <braunr> ew + <teythoon> did I just find the fs corruption bug everyone was looking for? + <braunr> no + <braunr> what corruption bug ? + <teythoon> not sure, I thought there was still some issues left with + unclean filesystems every now and then + <teythoon> *causing + <braunr> yes but we know the reasons + <teythoon> ah + <braunr> involving some of the funniest names i've seen in computer + terminology : + <braunr> writeback causing "message floods", which in turn create "thread + storms" in the servers receiving them + <teythoon> ^^ it's usually the other way around, storms causing floods >,, + <braunr> teythoon: :) + <braunr> let's say it's a bottom-up approach + <teythoon> then the fix is easy, compile mach with -DMIGRATING_THREADS :) + <braunr> teythoon: what ? + <teythoon> well, that would solve the flood/storm issue, no? + <braunr> no + <braunr> the real solution is proper throttling + <braunr> which can stem from synchronous rpc (which is the real property we + want from migrating threads) + <braunr> but the mach writeback interface is async + <braunr> :p + + +## IRC, freenode, #hurd, 2013-09-05 + + <braunr> teythoon: oh right, forgot about your port issue + <teythoon> don't worry, I figured by now that this must be a pointer + <teythoon> and I'm probably missing some magic that transforms this into a + name for the receiver + <teythoon> (though I "found" this function by looking at the mig + transformation for ports) + <braunr> i was wondering why you called the convert function manually + <braunr> instead of simply returning the task + <braunr> and let mig do the magic + <teythoon> b/c then I would have to add another ipc call, no? + <braunr> let me see the basic info call again + <braunr> my problem with this code is that it doesn't take into account the + ipc space of the current task + <braunr> which means you probably really return the ipc port + <braunr> the internal kernel address of the struct + <braunr> indeed, ipc_port_t convert_task_to_port(task) + <braunr> i'd personally make a new rpc instead of adding it to basic info + <braunr> basic info doesn't create rights + <braunr> what you want to achieve does + <braunr> you may want to make it a special port + <braunr> i.e. a port created at task creation time + <teythoon> y? + <braunr> it also means you need to handle task destruction and reparent + <teythoon> yes, I thought about that + <braunr> see + http://www.gnu.org/software/hurd/gnumach-doc/Task-Special-Ports.html#Task-Special-Ports + <braunr> for now you may simply turn the right into a dead name when the + parent dies + <braunr> although adding a call and letting mig do it is simpler + <braunr> mig handles reference counting, users just need to task_deallocate + once done + <teythoon> o_O mig does reference counting of port rights? + <braunr> mig/mach_msg + <teythoon> is there anything it *doesn't* do? + <braunr> i told you, it's a very complicated messaging interface + <braunr> coffee ? + <braunr> fast ? + <teythoon> ^^ + <braunr> mig knows about copy_send/move_send/etc... + <braunr> so even if it doesn't do reference counting explicitely, it does + take care of that + <teythoon> true + <braunr> in addition, the magic conversions are intended to both translate + names into actual structs, and add a temporary reference at the same time + <braunr> teythoon: everything clear now ? :) + <teythoon> braunr: no, especially not why you suggested to create a special + port. but this will have to wait for tomorrow ;) + + +## IRC, OFTC, #debian-hurd, 2013-09-06 + + <vorlon> teythoon: hi there + <vorlon> so I've been following your blog entries about cgroups on + hurd... very impressive :) + <vorlon> but I think there's a misunderstanding about upstart and + cgroups... your "conjecture" in + https://teythoon.cryptobitch.de/posts/what-will-i-do-next-cgroupfs-o/ is + incorrect + <vorlon> cgroups does not give us the interfaces that upstart uses to + define service readiness; adding support for cgroups is interesting to + upstart for purposes of resource partitioning, but there's no way to + replace ptrace with cgroups for what we're doing + <teythoon> vorlon: hi and thanks for the fish :) + <teythoon> vorlon: what is it exactly that upstart is doing with ptrace + then? + <teythoon> .,oO( your nick makes me suspicious for some reason... ;) + <teythoon> service readiness, what does that mean exactly? + <vorlon> teythoon: so upstart uses ptrace primarily for determining service + readiness. The idea is that traditionally, you know an init script is + "done" when it returns control to the parent process, which happens when + the service process has backgrounded/daemonized; this happens when the + parent process exits + <vorlon> in practice, however, many daemons do this badly + <vorlon> so upstart tries to compensate, by not just detecting that the + parent process has exited, but that the subprocess has exited + <vorlon> (for the case where the upstart job declares 'expect daemon') + <vorlon> cgroups, TTBOMK, will let you ask "what processes are part of this + group" and possibly even "what process is the leader for this group", but + doesn't really give you a way to detect "the lead process for this group + has changed twice" + <vorlon> now, it's *better* in an upstart/systemd world for services to + *not* daemonize and instead stay running in the foreground, but then + there's the question of how you know the service is "ready" before moving + on to starting other services that depend on it + <vorlon> systemd's answer to this is socket-based activation, which we + don't really endorse for upstart for a variety of reasons + <teythoon> hm, okay + <teythoon> so upstart does this only if expect daemon is declared in the + service description? + <vorlon> (in part because I've seen security issues when playing with the + systemd implementation on Fedora, which Lennart assures me are + corner-cases specific to cups, but I haven't had a chance to test yet + whether he's right) + <teythoon> and it is not used to track children, but only to observe the + daemonizing process? + <vorlon> yes + <teythoon> and it then detaches from the processes? + <vorlon> yes + <vorlon> once it knows the service is "ready", upstart doesn't care about + tracking it; it'll receive SIGCHLD when the lead process dies, and that's + all it needs to know + <teythoon> ok, so I misunderstood the purpose of the ptracing, thanks for + clarifying this + <vorlon> my pleasure :) + <vorlon> I realize that doesn't really help with the problem of hurd not + having ptrace + <teythoon> no, but thanks anyway + <vorlon> fwiw, the alternative upstart recommends for detecting service + readiness is for the process to raise SIGSTOP when it's ready + <vorlon> doesn't require ptracing, doesn't require socket-based activation + definitions; does require the service to run in a different mode than + usual where it will raise the signal at the correct time + <teythoon> right, but that requires patching it, same as the socket + activation stuff of systemd + <vorlon> (this is upstart's 'expect stop') + <vorlon> yes + <vorlon> though at DebConf, there were some evil ideas floating around + about doing this with an LD_PRELOAD or similar ;) + <vorlon> (overriding 'daemonize') + <vorlon> er, 'daemon()' + <teythoon> ^^ + <vorlon> and hey, what's suspicious about my /nick? vorlons are always + trustworthy + <vorlon> ;) + <teythoon> sure they are + <teythoon> but could this functionality be reasonably #ifdef'ed out for a + proof of concept port? + <vorlon> hmm, you would need to implement some kind of replacement... if + you added cgroups support to upstart as an alternative + <vorlon> that could work + <vorlon> i.e., you would need upstart to know when the service has exited; + if you aren't using ptrace, you don't know the "lead pid" to watch for, + so you need some other mechanism --> cgroups + <vorlon> and even then, what do you do for a service like openssh, which + explicitly wants to leave child processes behind when it restarts? + <teythoon> right... + <vorlon> oh, I was hoping you knew the answer to this question ;) Since + AFAICS, openssh has no native support for cgroups + <teythoon> >,< I don't, but I'll think about what you've said... gotta go, + catch what's left of the summer ;) + <teythoon> fwiw I consider fork/exec/the whole daemonizing stuff fubar... + <teythoon> see you around :) + <vorlon> later :) + + +## IRC, OFTC, #debian-hurd, 2013-09-07 + + <teythoon> vorlon: I thought about upstarts use of ptrace for observing the + daemonizing process and getting hold of the child + <teythoon> vorlon: what if cgroup(f)s would guarantee that the order of + processes listed in x/tasks is the same they were added in? + <teythoon> vorlon: that way, the first process in the list would be the + daemonized child after the original process died, no? + <vorlon> teythoon: that doesn't tell you how many times the "lead" process + has changed, however + <vorlon> you need synchronous notifications of the forks in order to know + that, which currently we only get via ptrace + + +## IRC, OFTC, #debian-hurd, 2013-09-08 + + <teythoon> vorlon: ok, but why do the notifications have to be synchronous? + does that imply that the processes need to be stopped until upstart does + something? + <vorlon> teythoon: well, s/synchronous/reliable/ + <vorlon> you're right that it doesn't need to be synchronous; but it can't + just be upstart polling the status of the cgroup + <vorlon> because processes may have come and gone in the meantime + <teythoon> vorlon: ok, cool, b/c the notifications of process changes I'm + hoping to introduce into the proc server for my cgroupfs do carry exactly + this kind of information + <vorlon> cool + <vorlon> are you discussing an API for this with the Linux cgroups + maintainers? + <teythoon> otoh it would be somewhat "interesting" to get upstart to use + this b/c of the way the mach message handling is usually implemented + <vorlon> :) + <teythoon> no, I meant in order for me to be able to implement cgroupfs I + had to create these kind of notifications, it's not an addition to the + cgroups api + <teythoon> is upstart multithreaded? + <vorlon> no + <vorlon> threads are evil ;) + <teythoon> ^^ I mostly agree + <vorlon> it uses a very nice event loop, leveraging signalfd among other + things + <teythoon> uh oh, signalfd sounds rather Linuxish + <pinotree> it is + <vorlon> I think xnox mentioned when he was investigating it that kfreebsd + now also supports it + <vorlon> but yeah, AFAIK it's not POSIX + <pinotree> it isn't, yes + <vorlon> but it darn well should be + <vorlon> :) + <vorlon> it's the best improvement to signal handling in a long time + <teythoon> systemd also uses signalfd + <teythoon> umm, it seems I was wrong about Hurd not having ptrace, the wiki + suggests that we do have it + <pinotree> FSVO "have" + <teythoon> ^^ + <xnox> vorlon: teythoon: so ok kFreeBSD/FreeBSD ideally I'd be using + EVFILT_PROC from kevent which allows to receive events & track: exit, + fork, exec, track (follow across fork) + <xnox> upstart also uses waitid() + <xnox> so a ptrace/waitid should be sufficient to track processes, if Hurd + has them. + + +## IRC, freenode, #hurd, 2013-09-09 + + <youpi> teythoon: yes, the shutdown notifications do stall the process + <youpi> but no more than a minute, or so + <youpi> teythoon: btw, did you end up understanding the odd thing in + fshelp_start_translator_long? + <youpi> I haven't had the time to have a look + <teythoon> youpi: what odd thing? the thing about being implemented by hand + instead of using the mig stub? + <youpi> the thing about the port being passed twice + <youpi> XXX this looks wrong to me, please have a look + <youpi> in the mach_port_request_notification call + <teythoon> ah, that was alright, yes + <youpi> ok + <youpi> so I can drop it from my TODO :) + <teythoon> this is done on the control port so that a translator is + notified if the "parent" translator dies + <teythoon> was that in fshelp_start_translator_long though? I thought that + was somewhere else + <youpi> that's what the patch file says + <youpi> +++ b/libfshelp/start-translator-long.c + <youpi> @@ -293,6 +293,7 @@ fshelp_start_translator_long (fshelp_open_fn_t + underlying_open_fn, + <youpi> + /* XXX this looks wrong to me, bootstrap is used twice as + argument... */ + <youpi> bootstrap, + MACH_NOTIFY_NO_SENDERS, 0, + <teythoon> right + <teythoon> I remember that when I got a better grip of the idea of + notifications I figured that this was indeed okay + <teythoon> I'll have a quick look though + <youpi> ok + <teythoon> ah, I remember, this notifies the parent translator if the child + dies, right + <teythoon> and it is a NO_SENDERS notification, so it is perfectly valid to + use the same port twice, as we only hold a receive right + + +## IRC, freenode, #hurd, 2013-09-10 + + <teythoon> braunr: are pthreads mapped 1:1 to mach threads? + <braunr> teythoon: yes + <teythoon> I'm reading the Linux cgroups "documentation" and it talks about + tasks (Linux threads) and thread group IDs (Linux processes) and I'm + wondering how to map this accurately onto Hurd concepts... + <teythoon> apparently on Linux there are PIDs/TIDs that can be used more or + less interchangeably from userspace applications + <teythoon> the Linux kernel however knows only PIDs, and each thread has + its own, and those threads belonging to the same (userspace) PID have the + same thread group id + <teythoon> aiui on Mach threads belong to a Mach task, and there is no + global unique identifier exposed for threads, right? + <teythoon> braunr: ^ + <tschwinge> teythoon: There is its thread port, which in combination with + its task port should make it unique? (I might be missing context.) + <tschwinge> Eh, no. The task port's name will only locally be unique. + * tschwinge confused himself. + <teythoon> tschwinge, braunr: well, the proc server could of course create + TIDs for threads the same way it creates PIDs for tasks, but that should + probably wait until this is really needed + <teythoon> for the most part, the tasks and cgroup.procs files contain the + same information on Linux, and not differentiating between the two just + means that cgroupfs is not able to put threads into cgroups, just + processes + <teythoon> that might be enough for now + + +## IRC, freenode, #hurd, 2013-09-11 + + <teythoon> ugh, some of the half-backed Linux interfaces will be a real + pain in the ass to support + <teythoon> they do stuff like write(2)ing file descriptors encoded as + decimal numbers for notifications :-/ + <braunr> teythoon: for cgroup ? + <teythoon> braunr: yes, they have this eventfd based notification mechanism + <teythoon> braunr: but I fear that this is a more general problem + <braunr> do we need eventfd ? + <teythoon> I mean passing FDs around is okay, we can do this just fine with + ports too, but encoding numbers as an ascii string and passing that + around is just not a nice interface + <braunr> so what ? + <teythoon> it's not a designed interface, it's one people came up with b/c + it was easy to implement + <braunr> if it's meant for compatibility, that's ok + <teythoon> how would you implement this then? as a special case in the + write(2) implementation in the libc? that sounds horrible but I do hardly + see another way + <teythoon> ok, some more context: the cgroup documentation says + <teythoon> write "<event_fd> <control_fd> <args>" to cgroup.event_control. + <teythoon> where event_fd is the eventfd the notification should be sent to + <pinotree> theorically they could have used sendmsg + a custom payload + <teythoon> control_fd is an fd to the pseudo file one wants notifications + for + <teythoon> yes, they could have, that would have been nicer to implement + <teythoon> but this... + + +## IRC, freenode, #hurd, 2013-09-12 + + <teythoon> ugh, gnumachs build system drives me crazy %-/ + <pinotree> oh there's worse than that + <teythoon> I added a new .defs file, did as Makerules.mig.am told me to do, + but it still does not create the stubs I need + <braunr> teythoon: gnumach doesn't + <braunr> teythoon: glibc does + <braunr> well, gnumach only creates the stubs it needs + <braunr> teythoon: you should perhaps simply use gnumach.defs + <teythoon> braunr: sure it does, e.g. vm/memory_object_default.user.c + <braunr> teythoon: what are you trying to add ? + <teythoon> braunr: I was trying to add a notification mechanism for new + tasks + <teythoon> b/c now the proc server has to query all task ports to discover + newly created tasks, this seems wasteful + <teythoon> also if the proc server could be notified on task creation, the + parent task is still around, so the notification can carry a reference to + it + <teythoon> that way gnumach wouldn't have to track the relationship, which + would create all kind of interesting questions, like whether tasks would + have to be reparented if the parent dies + <braunr> teythoon: notifications aren't that simple either + <teythoon> y not? + <braunr> 1/ who is permitted to receive them + <braunr> 2/ should we contain them to hurd systems ? (e.g. should a subhurd + receive notifications concerning tasks in other hurd systems ?) + <teythoon> that's easy imho. 1/ a single process that has a host_priv + handle is able to register for the notifications once + <braunr> what are the requirements so cgroups work as expected concerning + tasks ? + <braunr> teythoon: a single ? + <teythoon> i.e. the first proc server that starts + <braunr> then how will subhurd proc servers work ? + <teythoon> 2/ subhurds get the notifications from the first proc server, + and only those that are "for them" + <braunr> ok + <braunr> i tend to agree + <braunr> this removes the ability to debug the main hurd from a subhurd + <teythoon> this way the subhurds proc server doesn't even have to have the + host_priv porsts + <teythoon> yes, but I see that as a feature tbh + <braunr> me too + <braunr> and we can still debug the subhurd from the main + <teythoon> it still works the other way around, so it's still... + <teythoon> yes + <braunr> what would you include in the notification ? + <teythoon> a reference to the new task (proc needs that anyway) adn one to + the parent task (so proc knows what the parent process is and/or for + which subhurd it is) + <braunr> ok + <braunr> 17:21 < braunr> what are the requirements so cgroups work as + expected concerning tasks ? + <braunr> IOW, why is the parental relation needed ? + <braunr> (i don't know much about the details of cgroup) + <teythoon> well, currently we rely on proc_child to build this relation + <teythoon> but any task can use task_create w/o proc_child + <teythoon> until one claims a newly created task with proc_child, its + parent is pid 1 + <braunr> that's about the hurd + <braunr> i'm rather asking about cgroups + <teythoon> ah + <teythoon> the child process has to end up in the same cgroup as the parent + <braunr> does a cgroup include its own pid namespace ? + <teythoon> not quite sure what you mean, but I'd say no + <teythoon> do you mean pid namespace as in the Linux sense of that phrase? + <teythoon> cgroups group processes(threads) into groups + <teythoon> on Linux, you can then attach controllers to that, so that + e.g. scheduling decisions or resource restrictions can be applied to + groups + <teythoon> braunr: http://paste.debian.net/38950/ + <braunr> teythoon: ok so a cgroup is merely a group of processes supervised + by a controller + <braunr> for resource accounting/scheudling + <braunr> teythoon: where does dev_pager.c do the same ? + <teythoon> braunr: yes. w/o such controllers cgroups can still be used for + subprocess tracking + <teythoon> braunr: well, dev_pager.c uses mig generated stubs from + memory_object_reply.defs + <braunr> ah memory_object_reply ok + <braunr> teythoon: have you tried adding it to EXTRA_DIST ? + <braunr> although i don't expect it will change much + <braunr> teythoon: hum, you're not actually creating client stubs + <braunr> create a kern/task_notify.cli file + <braunr> as it's done with device/memory_object_reply.cli + <braunr> see #define KERNEL_USER 1 + <teythoon> braunr: right, thanks :) + + +## IRC, freenode, #hurd, 2013-09-13 + + <teythoon> hm, my notification system for newly created tasks kinda works + <teythoon> as in I get notified when a new task is created + <teythoon> but the ports for the new task and the parent that are carried + in the notification are both MACH_PORT_DEAD + <teythoon> do I have to add a reference manually before sending it? + <teythoon> that would make sense, the mig magic transformation function for + task_t consumes a reference iirc + <braunr> ah yes + <braunr> that reference counting stuff is some hell + <teythoon> braunr: ah, there's more though, the mig transformations are + only done in the server stub, not in the client, so I still have to + convert_task_to_port myself afaics + <teythoon> awesome, it works :) + <braunr> :) + <teythoon> ugh, the proc_child stuff is embedded deep into libc and signal + handling stuff... + <teythoon> "improving" the child_proc stuff with my shiny new notifications + wrecks havoc on the system + + # Required Interfaces In the thread starting diff --git a/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn b/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn index cf41550d..7159551d 100644 --- a/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn +++ b/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -16,26 +16,37 @@ License|/fdl]]."]]"""]] IRC, unknown channel, unknown date: - <youpi> azeem, marcus: ext2fs.static: thread-cancel.c:55: hurd_thread_cancel: Assertion '! __spin_lock_locked (&ss->critical_section_lock)' failed + <youpi> azeem, marcus: ext2fs.static: thread-cancel.c:55: + hurd_thread_cancel: Assertion '! __spin_lock_locked + (&ss->critical_section_lock)' failed <youpi> I actually don't understand this assertion <youpi> it's just before __spin_lock (&ss->critical_section_lock); <youpi> why should one check that a lock is free before taking it ? <youpi> just the same in hurdexec.c - <youpi> (no, ss is not our own sigstate, so it's not safe to assume no other path can take it) + <youpi> (no, ss is not our own sigstate, so it's not safe to assume no + other path can take it) <youpi> there's another one in sysdeps/mach/hurd/spawni.c <youpi> and jmp-unwind.c - <antrik> youpi: why do you think it's nonsense?... the fact that we take the lock (so we can't be interrupted) doesn't mean we are willing to wait for others to release the lock... maybe the code path should never be reached while others have a lock, or something + <antrik> youpi: why do you think it's nonsense?... the fact that we take + the lock (so we can't be interrupted) doesn't mean we are willing to wait + for others to release the lock... maybe the code path should never be + reached while others have a lock, or something <youpi> then it's useless to take the lock - <youpi> "we take the lock (so we can't be interrupted)": no, it's not _our_ lock here, it's the lock of the thread we want to cancel - <antrik> what exactly is cancelling a thread?... (sorry, I don't really have experience with thread programming) + <youpi> "we take the lock (so we can't be interrupted)": no, it's not _our_ + lock here, it's the lock of the thread we want to cancel + <antrik> what exactly is cancelling a thread?... (sorry, I don't really + have experience with thread programming) <youpi> ~= killing it - <antrik> well, we take the lock so nobody can mess with the thread while we are cancelling it, no?... + <antrik> well, we take the lock so nobody can mess with the thread while we + are cancelling it, no?... <youpi> yes <youpi> that is fine - <youpi> but checking that the lock is free before taking it doesn't make sense + <youpi> but checking that the lock is free before taking it doesn't make + sense <youpi> why nobody should be able to take the lock ? - <youpi> and if nobody is, why do we take it ? (since nobody would be able to take it) - <antrik> well, maybe after taking the lock, we do some action that might result in others trying to take it... + <youpi> and if nobody is, why do we take it ? (since nobody would be able + to take it) + <antrik> well, maybe after taking the lock, we do some action that might + result in others trying to take it... <youpi> nope: look at the code :) <youpi> or maybe the cancel_hook, but I really doubt it - diff --git a/open_issues/time.mdwn b/open_issues/time.mdwn index becb88b0..367db872 100644 --- a/open_issues/time.mdwn +++ b/open_issues/time.mdwn @@ -11,6 +11,11 @@ License|/fdl]]."]]"""]] [[!tag open_issue_porting]] +[[!toc]] + + +# `time` + Neither the `time` executable from the GNU time package work completely correctly, nor does the GNU Bash built-in one. @@ -56,20 +61,20 @@ As above; also here all the running time should be attributed to *user* time. This is probably a [[!taglink open_issue_gnumach]]. -# 2011-09-02 +## 2011-09-02 Might want to revisit this, and take Xen [[!tag open_issue_xen]] into account -- I believe flubber has already been Xenified at that time. -## IRC, freenode, #hurd, 2011-09-02 +### IRC, freenode, #hurd, 2011-09-02 While testing some [[performance/IPC_virtual_copy]] performance issues: <tschwinge> And I can confirm that with dd if=/dev/zero of=/dev/null bs=4k running, a parallel sleep 10 takes about 20 s (on strauss). -# 2013-03-30/31 +## 2013-03-30/31 Investigating time's `configure`, a difference of the output between Linux and Hurd shows: @@ -81,3 +86,754 @@ This causes a different code path in `resuse.c` to be used; such code path does not get a define for `HZ`, which is then defined with a fallback value of 60. [[!debbug 704283]] has been filed with a fix for this no-wait3 case. + + +# `times` + +## guile + +### IRC, freenode, #hurd, 2013-08-21 + + <nalaginrut> does guile2 on hurd fixed? times issue + <teythoon> nalaginrut: does not look good + <teythoon> scheme@(guile-user)> (times) + <teythoon> $1 = #(0 0 0 0 0) + <nalaginrut> well, seems not a fixed version, if there's fixed version + <nalaginrut> since it's not Guile's bug, I can do nothing for it + <teythoon> ah + <nalaginrut> in spite of this, Guile2 works I think + <nalaginrut> all tests passed but 2 fail + <nalaginrut> one of the failure is version shows "UNKNOWN" which is + trivials + <teythoon> well, did you try to fix the times issue in Hurd? + <nalaginrut> I didn't , I have to get more familiar with hurd first + <nalaginrut> I'm playing hurd these days + <teythoon> :) + <nalaginrut> anyway, I think times issue is beyond my ability at present + <nalaginrut> ;-P + <teythoon> times is implemented in the glibc, in sysdeps/mach/hurd/times.c + <teythoon> don't say that before you had a look + <nalaginrut> yes, you're right + <nalaginrut> but I think times has something to do with the kernel time + mechanism, dunno if it's related to the issue + <nalaginrut> how did you get the times.c under hurd? + <nalaginrut> apt-get source glibc? + <teythoon> well, I'd clone git://sourceware.org/git/glibc.git + <teythoon> and yes, the kernel is involved + <teythoon> task_info is used to obtain the actual values + <teythoon> + http://www.gnu.org/software/hurd/gnumach-doc/Task-Information.html + <teythoon> I'd guess that something fails, but the times(2) interface is + not able to communicate the exact failure + <nalaginrut> maybe it's not proper to get src from upstream git? since it's + OK under Linux which uses it too + <nalaginrut> but apt-get source glibc has nothing + <teythoon> so I would copy the times(2) implementation from the libc so + that you can modify it and run it as a standalone program + <teythoon> well, the libc has system dependent stuff, times(2) on Linux is + different from the Hurd version + <teythoon> it has to be + <nalaginrut> alright, I got what you mean ;-) + <teythoon> and the debian libc is built from the eglibc sources, so the + source package is called eglibc iirc + <nalaginrut> ah~I'll try + <teythoon> have you tried to rpctrace your times test program? the small c + snippet you posted the other day? + <nalaginrut> I haven't build all the tools & debug environment on my hurd + ;-( + <teythoon> what tools? + <nalaginrut> well, I don't even have git on it, and I'm installing but + speed is slow, I'm looking for a new mirror + <teythoon> ah well, no need to do all this on the Hurd directly + <teythoon> building the libc takes like ages anyway + <nalaginrut> oops ;-) + <nalaginrut> I'll take your advice to concentrate on times.c only + <teythoon> oh well, it might be difficult after all, not sure though + <teythoon> times sends two task_info messages, once with TASK_BASIC_INFO, + once with TASK_THREAD_TIMES_INFO + <teythoon> here is the relevant rpctrace of your test program: + <teythoon> task131(pid14726)->task_info (1 10) = 0 {0 25 153427968 643072 0 + 0 0 0 1377065590 570000} + <teythoon> task131(pid14726)->task_info (3 4) = 0 {0 0 0 10000} + <teythoon> ok, I don't know enough about that to be honest, but + TASK_THREAD_TIMES_INFO behaves funny + <teythoon> I put a sleep(1) into your test program, and if I rpctrace it, + it behaves differently o_O + * nalaginrut is reading task-information page to get what it could be + <nalaginrut> maybe I have to do the same steps under Linux to find some + clue + <teythoon> no, this is Mach specific, there is no such thing on Linux + <teythoon> on Linux, times(2) is a system call + <teythoon> on Hurd, times is a function implemented in the libc that + behaves roughly the same way + <nalaginrut> OK~so different + <teythoon> look at struct task_basic_info and struct task_thread_times_info + in the task-information page for the meaning of the values in the + rpctrace + <teythoon> yes, very + <braunr> nalaginrut: you may want to try a patch i did but which is still + waiting to be merged in glibc + <nalaginrut> braunr: ah~thanks for did it ;-) + <nalaginrut> can I have the link? + <braunr> i'm getting it + <braunr> teythoon: funny things happen with rpctrace, that's expected + <braunr> keep in mind rpctrace doesn't behave like ptrace at all + <braunr> it acts as a proxy + <braunr> nalaginrut: + http://git.savannah.gnu.org/cgit/hurd/glibc.git/commit/?h=rbraun/getclktck_100_hz&id=90404d6d1aa01f6ce1557841f5a675bb6a30f508 + <braunr> nalaginrut: you need to add it to the debian eglibc patch list, + rebuild the packages, and install the resulting .debs + <braunr> if you have trouble doing it, i'll make packages when i have time + <nalaginrut> braunr: I think your test result is expected? ;-) + <braunr> what test result ? + <nalaginrut> times test under that patch + <braunr> yes + <braunr> but i have no idea if it will work + <braunr> my patch fixes a mismatch between glibc and the procfs server + <braunr> nothing more + <braunr> it may help, it may not, that's what i'd like to know + <nalaginrut> hah~thanks for that + <nalaginrut> I get source from apt-get, then manually modified the files, + no much code ;-) + <nalaginrut> compiling + <nalaginrut> there is no cpuinfo in /proc? + <teythoon> no + <nalaginrut> a feature need to be done? or there's another way for that? + <teythoon> well, it hasn't been implemented + <teythoon> do you need that? what for? + <nalaginrut> compiling error, I realized I should use gcc-4.7 + <pinotree> how are you building? + <nalaginrut> I just happened to play proc while compiling, and found + there's no + <nalaginrut> cxa_finalize.c:48:1: error: tcbhead_t has no member + named multiple_threads + <nalaginrut> I changed to gcc-4.7 + <pinotree> just edit the sources, and then dpkg-buildpackage -nc -us -uc + <pinotree> that will rebuild the debian package as it would be in a debian + build, making sure all the build dependencies are there, etc + <pinotree> doing it different than that is just wrong™ + <nalaginrut> ok, doing + <pinotree> were you really doing ./configure etc yourself? + <nalaginrut> well, I can't wait till it's done, I'll let it compile and + check it out tomorrow + <nalaginrut> I used configure, yes ;-P + <pinotree> not good + <nalaginrut> I have to go, thanks for help guys + + +### IRC, freenode, #hurd, 2013-08-22 + + < nalaginrut> eglibc was done by dpkg-buildpackage, then how to install it? + (sorry I'm a brand new debian users) + < nalaginrut> oh~I found it + < nalaginrut> yes, (times) returns reasonable result ;-) + * nalaginrut is trying 'make check' + < nalaginrut> unfortunately, it can't pass the test though, I'm researching + it, anyway, we made first step + < nalaginrut> for Hurd internal-time-units-per-second will be 1000 + < nalaginrut> , but the elapsed time is far larger than (* 2 + internal-time-units-per-second) + < nalaginrut> I think the different of two returned clocks after 1 second + should be the TIME_UNITS_PER_SECOND, in principle + < nalaginrut> but I'm not sure if it's elibc or Guile bug + < nalaginrut> dunno, maybe clock tick should be 1000? + < nalaginrut> well, I'll try clock per second as 1000 + < braunr> nalaginrut: clock tick (or actually, the obsolete notion of a + clock tick in userspace) should be 100 + < braunr> nalaginrut: how did you come with 1000 ? + < nalaginrut> braunr: Guile set TIME_UNITS_PER_SECOND to 1000 when there's + no 8bytes size and doesn't define HAVE_CLOCK_GETTIME + < nalaginrut> #if SCM_SIZEOF_LONG >= 8 && defined HAVE_CLOCK_GETTIME + < nalaginrut> #define TIME_UNITS_PER_SECOND 1000000000 + < nalaginrut> #else + < nalaginrut> #define TIME_UNITS_PER_SECOND 1000 + < nalaginrut> #endif + < nalaginrut> and the test for 'times' used time-units-per-second + < pinotree> what has sizeof(long) have to do with time units per second? + < nalaginrut> dunno, maybe the representation of time? + < nalaginrut> the test failed since the difference between two clocks after + 1sec is too large + < nalaginrut> and for the test context, it should small than 2 times of + units-per-second + < nalaginrut> should be smaller + < nalaginrut> sorry for bad English + < pinotree> aren't you basically looking for clock_getres? + < nalaginrut> pinotree: I don't understand what you mean + < pinotree> + http://pubs.opengroup.org/onlinepubs/9699919799/functions/clock_getres.html + < nalaginrut> I wonder if there's a standard CLK_PER_SEC for Hurd + < nalaginrut> or it can be modified as wish + < pinotree> why do you need it? + < nalaginrut> the difference is 10,000,000, which can never less than + 2*clock_per_second + < nalaginrut> pinotree: I don't need it, but I want to know if there's a + standard value + < braunr> nalaginrut: ok so, this is entirely a guile thing + < braunr> nalaginrut: did you test with my patch ? + < nalaginrut> braunr: yes, 'times' works fine + < braunr> but even with that, a tets fails ? + < braunr> test* + < nalaginrut> well, I can't say works fine, the proper description is "now + it has reasonable result" + < braunr> youpi: could you bring + http://darnassus.sceen.net/gitweb/savannah_mirror/glibc.git/commit/90404d6d1aa01f6ce1557841f5a675bb6a30f508 + into debian glibc btw ? + < nalaginrut> braunr: it failed the test since the clock run too fast, but + it should be smaller than 2*clk-per-sec + < braunr> i don't get that + < braunr> can you show the code that checks the condition ? + < nalaginrut> braunr: http://pastebin.com/sG3QxnPt + < braunr> * 0.5 internal-time-units-per-second ? + < nalaginrut> for C users, it's just like + a=times(...);sleep(1);b=times(...); then time-units-per-sec/2 <= (b-a) <= + time-units-per-sec*2 + < braunr> ah ok + < nalaginrut> the test passes when it's true + < braunr> so basically, it says sleep(1) sleeps for more than 2 seconds + < braunr> can you check the actual value ? + < braunr> b-a + < nalaginrut> hold on for minutes + < nalaginrut> it's 10,000,000 + < nalaginrut> for clk-per-sec=1000,000,000, it's OK + < nalaginrut> but for 100 or 1000, it's too small + < braunr> let's forget 100 + < braunr> guile uses 1000 + < nalaginrut> OK + < braunr> but i still don't get why + < nalaginrut> so I asked if there's standard value, or it can be ajustified + < nalaginrut> adjusted + < braunr> ok so, times are expressed in clock ticks + < braunr> are you sure you're using a patched glibc ? + < nalaginrut> yes I used your patch, and the 'times' get reasonable result + < braunr> then + < braunr> 11:28 < nalaginrut> it's 10,000,000 + < braunr> doesn't make sense + < nalaginrut> hmm + < braunr> anhd i don't understand the test + < braunr> what's tms:clock new ? + < nalaginrut> it's actually the return value of 'times' + < nalaginrut> Guile wrap the clock_t and tms to a vector, then we can get + all the thing in a row + < nalaginrut> 'new' is a variable which was gotten after 1 sec + < braunr> let's see what this does exactly + < nalaginrut> equal to "new = times(...)" + < nalaginrut> 'tms' equal to (clock_t (struct tms)) + < nalaginrut> we have to pass in the struct pointer to get the struct + values filled, but for Guile we don't use pointer, times actually returns + two things: clock_t and struct tms + < nalaginrut> and Guile returns them as a vector in a row, that's it + < braunr> nalaginrut: test this please: + http://darnassus.sceen.net/~rbraun/test.c + < braunr> i don't have a patched libc here + < braunr> i'll build one right now + < nalaginrut> clock ticks: 1000000 + < braunr> and this seems reasonable to you ? + < braunr> anyway, i think the guile test is bugged + < nalaginrut> no, the reasonable is not for this + < braunr> does it ever get the clock tick value from sysconf() ? + < nalaginrut> I say reasonable since it's always 0 both for clock and tms, + before apply your patch + < braunr> uh no + < braunr> i have the same value, without my patch + < nalaginrut> so I said "I can't say it works fine" + < braunr> either the test is wrong because it doesn't use sysconf() + < nalaginrut> anyway, I don't think times should return "all zero" + < braunr> or the clock values have already been ocnverted + < braunr> but it doesn't + < braunr> you did something wrong + < nalaginrut> with your patch it doesn't + < braunr> without neither + < braunr> 11:43 < braunr> i have the same value, without my patch + < nalaginrut> well, it's too strange + < braunr> check how the test actually gets the clock values + < braunr> also, are your running in vbox ? + < braunr> you* + < nalaginrut> no ,it's physical machine + < braunr> oh + < braunr> nice + < braunr> note that vbox has timing issues + < nalaginrut> I thought I should give you some info of CPU, but there's no + /proc/cpuinfo + < braunr> shouldn't be needed + < nalaginrut> OK + < braunr> run my test again with an unpatched glibc + < braunr> just to make sure it produces the same result + < braunr> and + < nalaginrut> so the clock-per-sec is machine independent for Hurd I think + < braunr> 11:46 < braunr> check how the test actually gets the clock values + < nalaginrut> since it's implemented in userland + < braunr> clock-per-sec is always system dependent + < braunr> All times reported are in clock ticks. + < braunr> The number of clock ticks per second can be obtained + using: + < braunr> sysconf(_SC_CLK_TCK); + < braunr> 11:46 < braunr> check how the test actually gets the clock values + < braunr> to see if they're converted before reaching the test code or not + * nalaginrut is building eglibc + < braunr> building ? + < braunr> what for ? + < nalaginrut> I modified it to 1000, now it's useless + < braunr> we want it to 100 either way + < nalaginrut> and how to reinstall eglibc under debian? + < braunr> it's obsolete, procfs already uses 100, and 100 is low enough to + avoid overflows in practically all cases + < braunr> aptitude install libc0.3=<version> + < nalaginrut> OK + < braunr> aptitude show -v libc0.3 + < braunr> for the list of available versions + < nalaginrut> out of topic, what's the meaning of the code in + quantize_timeval ? + < nalaginrut> tv->tv_usec = ((tv->tv_usec + (quantum - 1)) / quantum) * + quantum; + < nalaginrut> I can't understand this line + < braunr> scaling and rounding i guess + < nalaginrut> hmm...but quantum seems always set to 1? + < nalaginrut> 100/__getclktck() + < braunr> ah right + < braunr> old crap from the past + < nalaginrut> and clk-tck is 100 + < braunr> the author probably anticipated clk_ticks could vary + < braunr> in practice it doesn't, and that's why it's been made obsolete + < nalaginrut> I wonder if it could be vary + < braunr> no + < nalaginrut> alright + < nalaginrut> why not just assign it to 1? + < braunr> 11:55 < braunr> old crap from the past + < braunr> the hurd is 20 years old + < braunr> like linux + < nalaginrut> oh~ + < braunr> but with a lot less maintenance + < nalaginrut> braunr: well, I tried the original eglibc, your test was + clock ticks: 1000000 + < nalaginrut> but in Guile, (times) ==> (0 0 0 0 0) + < nalaginrut> the reasonable result maybe: #(4491527510000000 80000000 0 0 + 0) + < braunr> 11:46 < braunr> check how the test actually gets the clock values + < braunr> ah, he left + + +### IRC, freenode, #hurd, 2013-08-23 + + < braunr> nalaginrut: times() doesn't seem to be affected by my patch at + all + < nalaginrut> braunr: but it did in my machine + < nalaginrut> well, I think you mean it doesn't affect your C test code + < braunr> i'm almost sure something was wrong in your test + < braunr> keep using the official debian glibc package + < nalaginrut> I don't think it's test issue, since every time (times) + return zero, the test can never get correct result + < braunr> times doesn't return 0 + < braunr> for sleep(1), i always have the right result, except in + microseconds + < nalaginrut> times in Guile always return #(0 0 0 0 0) + < braunr> (microseconds is the native mach time unit) + < braunr> well, guile does something wrong + < nalaginrut> after sleep 1, it's 0 again, so it's none sense + < braunr> 11:46 < braunr> check how the test actually gets the clock values + < braunr> not on my system + < nalaginrut> but (times) returns reasonable result after applied your + patch + < braunr> that's not normal, since times isn't affected by my patch + < nalaginrut> oops + < braunr> you need to look for what happens in guile between the times() + call and the #(0 0 0 0 0) values + < nalaginrut> well, I tried many times between patch or non-patch, I think + there's no mistake + < nalaginrut> I read the 'times' code in Guile, there's nothing strange, + just call 'times' and put all the result to a vector + < braunr> which means there is no conversion + < braunr> in which case the test is plain wrong since there MUST also be a + call to sysconf() + < braunr> to obtain the right clock ticks value + < braunr> is your box reachable with ssh ? + < nalaginrut> oh~wait, seems there's a quotient operation, I'm checking + < nalaginrut> factor = scm_quotient (scm_from_long (TIME_UNITS_PER_SECOND), + < nalaginrut> scm_from_long (ticks_per_second)); + < braunr> iirc, TIME_UNITS_PER_SECOND is hardcoded + < nalaginrut> unless factor is zero + < nalaginrut> yes, it's hardcoded + < braunr> that's completely non portable and wrong + < nalaginrut> you suggest to call sysconf? + < braunr> yes + < braunr> but i don't have the code in mind + < braunr> what is ticks_per_second ? + < nalaginrut> OK, that's one issue, we have to find why times return 0 + < braunr> 14:14 < braunr> is your box reachable with ssh ? + < braunr> i'd like to make sure times returns 0 at your side + < braunr> because it doesn't at mine + < nalaginrut> no + < braunr> until i can reproduce, i can't consider there is a problem + < nalaginrut> I think it's unreachable for outer space + < nalaginrut> well, if you want to reproduce, just get guile src of debian + < braunr> guile 2.0 ? + < nalaginrut> yes, apt-get source guile-2.0 + < nalaginrut> I'm checking ticks_per_second + < braunr> got the source, how do i test + < braunr> ? + < nalaginrut> you have to build it, and run ./meta/guile, then you don't + have to install it + < nalaginrut> and try (times) + < braunr> aw libgc + < nalaginrut> the reasonable result should be #(4313401920000000 110000000 + 20000000 0 0) or something alike + < nalaginrut> but #(0 0 0 0 0) in each time is not reasonable apparently + < nalaginrut> maybe you need apt-get build-dep guile-2.0? + < braunr> already done + < nalaginrut> building Guile2 may take very long time + < nalaginrut> about 30 minutes in my old machine + < braunr> then it should take just a few minutes on mine + < nalaginrut> alright it's not very long, I've spent 8 hours for gcc in LFS + < braunr> 8 hours ? + < braunr> takes 5-10 minutes on a common machine .. + < nalaginrut> but it's Celeron566 at that time... + < braunr> ah, that old + < nalaginrut> include bootstrap, so very long + < braunr> nalaginrut: i got the test failure from the build procedure, how + do i run it manually ? + < nalaginrut> braunr: ./meta/guile -L test-suite + test-suite/tests/time.test + < nalaginrut> braunr: or make check for all + < braunr> put a print after the schedule() and before the return nil; in + runtime_mstart, since that's the body of new threads + < nlightnfotis> unfortunately, I can't confirm this with goroutines + running; the assertion failure aborts before I can get anything useful + < braunr> you can + < braunr> make sure there is a \n in the message, since stdout is line + buffered by default + < braunr> if you don't reach that code, it means threads don't exit + < braunr> at least goroutine threads + < braunr> btw, where is the main thread running ? + < nlightnfotis> I just checked there is a \n at the end. + < nlightnfotis> "<braunr> btw, where is the main thread running " could you + elaborate a little bit on this? + < braunr> what does main() after initializing the runtime ? + < braunr> +do + < nlightnfotis> the runtime main or the process's main? + < braunr> the process + < braunr> nlightnfotis: what we're interested in is knowing whether main() + exits or not + < nlightnfotis> braunr: I can see there are about 4 functions of interest: + runtime_main (the main goroutine, and I can imagine 1st thread) + < nlightnfotis> main_init (I don't know what it does, will check this out + now) + < nlightnfotis> main_main (not sure about this one either) + < nlightnfotis> and runtime_exit (0) + < braunr> i can see that too + < braunr> i'm asking about main() + < nlightnfotis> which seems to be the function that terminates the main + thread + < nlightnfotis> <braunr> nlightnfotis: what we're interested in is knowing + whether main() exits or not --> my theory is runtime_exit (0) exits the + process' main. Seeing as at various times go programs echo $? == 0. + < nlightnfotis> let me research that a little bit + < nlightnfotis> braunr: that will require a bit more studying. main_main() + and main_init() are both expanded to assembly tags if I understand it + correctly. + < nlightnfotis> main.main and __go_init_main respectively. + < braunr> why are you looking from there instead of looking from main() ? + < nlightnfotis> are we not looking out if main exits? + < braunr> we are + < braunr> so why look at main_main ? + < braunr> or anything else than main ? + < nlightnfotis> these are called inside runtime_main and I figured out they + might have a clue + < braunr> runtime_main != main + < braunr> (except if there is aliasing) + < nlightnfotis> there is still the possibility that runtime_main is the + main function and that runtime_exit(0) exits it. + < braunr> there is no doubt that main is main + < braunr> (almost) + < nlightnfotis> and I just found out that there is no main in assembly + produced from go. Only main.main + < braunr> check the elf headers for the entry point then + < nlightnfotis> braunr: I went through the headers, and found the process' + main. You can find it in <gcc_root>/libgo/runtime/go-main.c + < nlightnfotis> it seems very strange though: It creates a new thread, then + aborts? + < braunr> nlightnfotis: see :) + < braunr> nlightnfotis: add traces there + < nlightnfotis> braunr: can you look into that piece of code to check out + something I don't understand? + < nlightnfotis> braunr: I can not seem able to find __go_go 's definition + < nlightnfotis> only a declaration in runtime.h + < braunr> + https://github.com/NlightNFotis/gcc/blob/master/libgo/runtime/proc.c, + line 1552 + < nlightnfotis> gee thanx. For a strange kind of fashion, I was looking for + it in runtime.c + < braunr> use git grep + < braunr> or tags/cscope + < nlightnfotis> braunr: yep! runtime_exit does seem to terminate a go + process that was not otherwise abnormally terminated. + < braunr> ? + < braunr> is it called or not ? + < braunr> runtime_exit is a macro on exit() + < braunr> so we already know what it does + < nlightnfotis> it is called + < braunr> ok + < braunr> that's not normal :) + < nlightnfotis> for a simple program + < braunr> uh ? + < nlightnfotis> for one that has a go routine + < braunr> but + < nlightnfotis> it doesn't + < nlightnfotis> it's expected + < braunr> ok + < braunr> that makes sense + < braunr> well, trace + < braunr> keep tracing + < braunr> for example in main() + < braunr> is runtime_mstart() actually reached ? + < nlightnfotis> yeah main and runtime_main were my next two targets + < braunr> good + < nlightnfotis> and now I followed your advice and it does compiler much + faster + < braunr> so, it looks like the main thread just becomes a mere kernel + thread + < braunr> running runtime_mstart() and fetching goroutines as needed + < braunr> after your traces, i'd suggest running a small go test program, + with one simple goroutine (doesn't crash right ?) + < braunr> and trace context switching + < braunr> but after the traces + < braunr> one important trace is to understand why runtime_exit gets called + < nlightnfotis> it does crash even with 1 goroutine + < braunr> oh + < braunr> when doesn't it crash ? + < nlightnfotis> when it has 0 goroutines + < nlightnfotis> it works as expected + < nlightnfotis> but anything involving goroutines crashes + < nlightnfotis> and goroutines are very important; everything in the + standard library involves goroutines + < braunr> ok + < braunr> doesn't change what i suggested, good + < braunr> 1/ find out why runtime_exit gets called + < braunr> 2/ trace context switching with 1 goroutine + < nlightnfotis> on it. + < braunr> in all cases, make all your goroutines (including the main one) + *not* return + < braunr> so that you don't deal with goroutine destruction yet + < nlightnfotis> runtime_mstart in main doesn't to be run at all. So the + path is __go_go and then return from it. + < nlightnfotis> *doesn't seem + + +### IRC, freenode, #hurd, 2013-08-26 + + < braunr> youpi: my glibc clock patch looks incomplete btw + < youpi> which one? + < youpi> ah, the ticks one? + < braunr> yes + < braunr> it doesn't change the values returned by times + < braunr> as a side effect, the load average bumps to 2+ on an idle machine + + +### IRC, freenode, #hurd, 2013-08-27 + + < nalaginrut> braunr: have you tried Guile2 on your machine? ;-) + < braunr> nalaginrut: no + < braunr> nalaginrut: but i saw the code actually does use sysconf() + < nalaginrut> braunr: yes, for ticks_per_second + < braunr> i had to look myself to find it out, you didn't say it, despite + me asking multiple times + < braunr> it won't make debugging easier ;p + < braunr> nalaginrut: also, the return value of times is actually *never* + used + < braunr> i don't know why you've been talking about it so much + < nalaginrut> braunr: I'm sorry, it's first time to look stime.c for me + < braunr> the interesting function is get_internal_run_time_times() + < nalaginrut> what do you mean about "the return value of times is actually + *never* used"? in which context? + < braunr> see get_internal_run_time_times + < braunr> struct tms time_buffer; + < braunr> times(&time_buffer); + < braunr> return ... + < braunr> and yes, the user and system time reported in struct tms are 0 + < braunr> let's see what posix has to say about it + < pinotree> it says it will return (clock_t)-1 for errors, but no standard + errors are defined yet + < nalaginrut> but I don't think get_internal_run_time_times has something + to do with scm_times + < braunr> well, i don't see any other call to times() + < braunr> i've asked you repeatedly to look for how guile fetches the data + < braunr> i think it's done in get_internal_run_time_times + < braunr> what makes you think otherwise ? + < braunr> our times() seems to behave fine, other than the units of the + return value + < nalaginrut> I don't understand what do you mean? + get_internal_run_time_times is unrelated to scm_times which is actually + "times" in Scheme code + < braunr> ok + < nalaginrut> I think we're talking about "times" activity, right? + < braunr> ok so result is a vector + < braunr> with the return value and the four values in struct tms + < nalaginrut> yes + < braunr> and what looks interesting is + < braunr> factor = scm_quotient (scm_from_long (TIME_UNITS_PER_SECOND), + scm_from_long (ticks_per_second)); + < braunr> SCM_SIMPLE_VECTOR_SET (result, 0, scm_product (scm_from_long + (rv), factor)); + < braunr> TIME_UNITS_PER_SECOND is 1000 + < nalaginrut> yes, it means (clock_t * + (TIME_UNITS_PER_SECOND/ticks_per_second)), though I've no idea why it + does this + < braunr> normalizing values i guess + < nalaginrut> I wonder if the factor should be 1, just guessing + < braunr> let's see what our clock tick really is + < braunr> 1000000 on an unmodified libc + < braunr> 100 with my patch + < nalaginrut> so what's the problem? + < nalaginrut> all the values were multiplied by ticks, it's fair for the + subtraction + < nalaginrut> I think the problem is clock is too large for the difference + between utime and utime(sleep 1) + < nalaginrut> oops, is too small + < nalaginrut> sorry, I confused, + < nalaginrut> the problem is the difference of clock is too large for + 2*internal-time-units-per-second + < nalaginrut> and actually, internal-time-units-per-second is + SCM_TIME_UNITS_PER_SECOND + < nalaginrut> but without your patch, 'times' would return zeros all the + time, which is never meet the condition: SCM_TIME_UNITS_PER_SECOND/2 <= + (clock2 - clock1) + < nalaginrut> well, maybe your point is + TIME_UNITS_PER_SECOND/ticks_per_second is too small without your patch, + which causes the scm_to_long cast give a 0 value + < nalaginrut> s/cast/casting + < nalaginrut> when ticks_per_second is 100, the factor would be 10, which + seems to be reasonable + < nalaginrut> s/scm_to_long/scm_from_long + < nalaginrut> well, I have to checkout this + < nalaginrut> OK, let me reconstruct the point: ticks_per_second so too + large that makes the factor becomes zero + < nalaginrut> but decrease ticks_per_second to 100 causes the clock become + too large than TIME_UNITS_PER_SECOND + < braunr> 10:59 < nalaginrut> but without your patch, 'times' would return + zeros all the time, which is never meet the condition: + SCM_TIME_UNITS_PER_SECOND/2 <= (clock2 - clock1) + < braunr> until you prove me otherwise, this is plain wrong + < braunr> times() never returned me 0 + < braunr> so let's see, this gives us a factor of 1000 / 1000000 + < braunr> so the problem is factor being 0 + < braunr> that's why *guile* times returns 0 + < braunr> with my patch it should return 10 + < nalaginrut> braunr: I'm sorry I mean "stime" in Scheme returns zeros + < nalaginrut> yes, I think the problem is factor + < nalaginrut> the factor + < braunr> now why doesn't my patch fix it all ? + < braunr> ah yes, rv is still in microseconds + < braunr> that's what i've been telling youpi recently, my patch is + incomplete + < braunr> i'll cook a quick fix, give me a few minutes please + < nalaginrut> but it fixed something ;-) + < braunr> well, guile makes a stupid assumption here + < braunr> so it's not really a fix + < nalaginrut> braunr: should I ask some info about TIME_UNITS_PER_SECOND + from Guile community? + < nalaginrut> or it doesn't help + < braunr> what do you want to ask them ? + < nalaginrut> since I don't know how this value was chosen + < nalaginrut> dunno, I'll ask if you need it + < nalaginrut> I just think maybe you need this info + < braunr> well + < braunr> my plan is to align the hurd on what other archs do + < braunr> i.e. set clk_tck to 100 + < braunr> in which case this won't be a problem any more + < braunr> now you could warn them about the protability issue + < braunr> i'm not sure if they would care though + < nalaginrut> the warning is useful for the future + < nalaginrut> and it's not hard to make a change I think, for a constant, + but it depends on the maintainers + < braunr> it's not that simple + < braunr> time related things can easily overflow in the future + < nalaginrut> alright + < braunr> refer to the 2038 end-of-the-world bug + < nalaginrut> so how can I describe the warning/suggestion to them? + < braunr> i'm not sure + < braunr> tell them the TIME_UNITS_PER_SECOND isn't appropriate for larger + values of clk_tck + < braunr> dammit, microseconds are hardcoded everywhere in + sysdeps/mach/hurd ... >( + < braunr> nalaginrut: my new patch seems to fix the problem + < braunr> nalaginrut: i've built debian packages with which you can + directly test + < braunr> nalaginrut: deb http://ftp.sceen.net/debian-hurd-i386 + experimental/ + < braunr> Totals for this test run: + < braunr> passes: 38605 + < braunr> failures: 0 + < braunr> unexpected passes: 0 + < braunr> expected failures: 7 + < braunr> unresolved test cases: 578 + < braunr> untested test cases: 1 + < braunr> unsupported test cases: 10 + < braunr> errors: 0 + < braunr> PASS: check-guile + < braunr> ============= + < braunr> 1 test passed + < braunr> ============= + < braunr> :) + < braunr> youpi: the branch i added to glibc contains a working patch for + clock_t in centiseconds + < youpi> k + + +### IRC, freenode, #hurd, 2013-08-28 + + <nalaginrut> braunr: well, looks great! I'll try it soon~ + <nalaginrut> braunr: BTW, where is the patch/ + <mark_weaver> braunr: what was needed to get guile working on the hurd? + <mark_weaver> well, if the fix wasn't to guile, I don't need the details. + <braunr> 04:53 < nalaginrut> braunr: BTW, where is the patch/ + <braunr> there is hardly anyone here at 5am + <braunr> nalaginrut: + http://git.savannah.gnu.org/cgit/hurd/glibc.git/log/?h=rbraun/clock_t_centiseconds + <nalaginrut> braunr: thanks for that, but why not use a constant for 100? + <braunr> nalaginrut: i don't know where to define it + <braunr> it's glibc, you don't define new stuff mindlessly + <youpi> braunr: about your centiseconds patch, did you run the libc + testsuite with it? + <mark_weaver> it does seem a shame to reduce the resolution of the timers + from microseconds to centiseconds. I wonder if that could be avoided. + <youpi> by fixing all applications which assume centiseconds + <mark_weaver> *nod* well, if there's such a problem in Guile, I'd be glad + to fix that. + <braunr> youpi: no + <mark_weaver> I see that there's a macro CLOCKS_PER_SEC that programs + should consult. + <youpi> braunr: ok, I'll do then + <braunr> mark_weaver: why is it a shame ? + <braunr> it's not clock or timer resolution + <youpi> it's clock_t resolution + <braunr> it's an obsolete api to measure average cpu usage + <braunr> having such a big value on the other hand reduces the cpu usage + durations + <mark_weaver> braunr: good point :) I confess to being mostly ignorant of + these APIs. + <mark_weaver> Though Guile should still consult CLOCKS_PER_SEC instead of + assuming centiseconds. If it's making an improper assumption, I'd like + to know so I can fix it. + <braunr> the improper assumption is that there are less than 1000 clock + ticks per second + <mark_weaver> do you know off-hand of some code in Guile that is making + improper assumptions? + <braunr> yes + <braunr> let me find it + <mark_weaver> thanks + <braunr> factor = scm_quotient (scm_from_long (TIME_UNITS_PER_SECOND), + <braunr> scm_from_long (ticks_per_second)); + <braunr> it seems guile attempts to normalize all times values to + TIME_UNITS_PER_SECOND + <braunr> while i think it would be better off using ticks_per_second (clock + ticks as provided by sysconf()) + <braunr> attempting to normalize here causes factor to become 0 if + TIME_UNITS_PER_SECOND < ticks_per_second + <mark_weaver> ah, I see. + <mark_weaver> I'll take care of it. thanks for the pointer! + <youpi> braunr: I've commited the centisecond patch to debian's glibc + + +### IRC, freenode, #hurd, 2013-08-29 + + <nalaginrut> braunr: Guile2 works smoothly now, let me try something cool + with it + <braunr> nalaginrut: nice diff --git a/open_issues/tmux.mdwn b/open_issues/tmux.mdwn new file mode 100644 index 00000000..f71d13e1 --- /dev/null +++ b/open_issues/tmux.mdwn @@ -0,0 +1,24 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_porting]] + +# IRC, freenode, #hurd, 2013-08-01 + + <braunr> teythoon: can you stop tmux on darnassus please ? + <braunr> i'd like to check something + <teythoon> done + <braunr> tmux makes load average grow to 5 without any visible activity :/ + <braunr> can't reproduce it with my instances though + <braunr> anyway, that's minor + <teythoon> I used tmux before and never encountered that + <teythoon> sometimes tmux would hang on attaching or detaching though, but + overall I had less problems with tmux than with screen + <teythoon> ah, I tried to start tmux on darnassus and now it hangs diff --git a/open_issues/translate_fd_or_port_to_file_name.mdwn b/open_issues/translate_fd_or_port_to_file_name.mdwn index fda4b811..98fe0cfc 100644 --- a/open_issues/translate_fd_or_port_to_file_name.mdwn +++ b/open_issues/translate_fd_or_port_to_file_name.mdwn @@ -105,6 +105,57 @@ License|/fdl]]."]]"""]] <tschwinge> Ah, for /proc/*/maps, right. I've been thinking more globally. +## task_get_name, task_set_name RPCs + +[[!message-id "518AA5B0.6030409@verizon.net"]] + + +## IRC, freenode, #hurd, 2013-05-10 + + <youpi> tschwinge's suggestion to put names on ports instead of tasks would + be useful too + <braunr> do you get task ports as easily as you get tasks in kdb ? + <youpi> there is task->itk_self & such + <youpi> or itk_space + <youpi> I don't remember which one is used by userspace + <braunr> i mean + <braunr> when you use the debugger, can you easily find its ports ? + <braunr> the task ports i mean + <braunr> or thread ports or whatever + <youpi> once you have a task, it's a matter of getting the itk_self port + <youpi> s/port/field member/ + <braunr> so the debugger provides you with the addresses of the structs + <braunr> right ? + <youpi> yes, that's what we have already + <braunr> then ok + <braunr> bddebian: do that :p + <braunr> hehe + <youpi> see show all thread + <braunr> (haven't used kdb in a long time) + <bddebian> So, adding a name to ports like I did with tasks? + <braunr> remove what you did for tasks + <braunr> move it to ports + <braunr> it's very similar + <braunr> but hm + <braunr> i'm not sure where the RPC would be + <braunr> this RPC would exist for *all* ports + <braunr> or only for kernel objects if added to gnumach.defs + <youpi> it's just about moving the char array field to another structure + <youpi> and plugging that + <bddebian> But mach_task_self is a syscal, it looks like itk_self is just a + pointer to an ipc_port ? + <braunr> so ? + <braunr> you take that pointer and you get the port + <braunr> just like vm_map gets a struct vm_map from a task + <bddebian> So I am just adding ipc_port_name to the ipc_port struct in this + case? + <braunr> yes + <braunr> actually + <braunr> don't do anything just yet + <braunr> we need to sort a few details out first + <braunr> see bug-hurd + + # IRC, freenode, #hurd, 2011-07-13 A related issue: @@ -134,4 +185,8 @@ A related issue: Also see email thread starting at [[!message-id "20110714082216.GA8335@sceen.net"]]. -Justus: Once [these patches](http://lists.gnu.org/archive/html/bug-hurd/2013-07/msg00262.html) are merged there is a way to map from ports to file names, at least for lib{disk,net}fs, one would only have to make this information available somehow. +Justus: Once [[!message-id desc="these patches" +"1375178364-19917-4-git-send-email-4winter@informatik.uni-hamburg.de"]] are +merged, there will be a way to map from ports to file names, at least for +libdiskfs and libnetfs, one would only have to make this information available +somehow. diff --git a/open_issues/translator_stdout_stderr.mdwn b/open_issues/translator_stdout_stderr.mdwn index 14ea1c6d..89efd4e1 100644 --- a/open_issues/translator_stdout_stderr.mdwn +++ b/open_issues/translator_stdout_stderr.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation, -Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011, 2013 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -9,25 +9,69 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!tag open_issue_hurd]] +[[!toc]] -There have been several discussions and proposals already, about adding a -suitable logging mechanism to translators, for example. +# "Weird Issue" -Decide / implement / fix that (all?) running (passive?) translators' output -should show up on the (Mach / Hurd) console / syslog. +## IRC, freenode, #hurd, 2013-07-01 +[[!tag open_issue_hurd]] -[[!taglink open_issue_documentation]]: [[!message-id -"87oepj1wql.fsf@becket.becket.net"]] + <teythoon> oh, btw, there is this weird issue that I cannot figure out + <teythoon> I noticed that there is no newline printed by /hurd/init after + printing " proc" and " auth" + <teythoon> but there *is* a printf("\n"); fflush(stdout); in there + <teythoon> it's just not working + <pinotree> iirc a newline is supposed to be printed after all the essential + servers have been started + <pinotree> that one + <teythoon> yes + <teythoon> but this doesn't happen + <teythoon> for whatever reason printf("foo"); yields no output + <braunr> how are proc and auth printed ? + <teythoon> neither does printf("%s", "foo"); + <teythoon> using printf + <teythoon> but printf("%i fooo", 4); works + <youpi> uh + <braunr> ?? + <youpi> and does printf("loooooooooong line") worker? + <teythoon> no + <youpi> uh + <youpi> -er + <teythoon> and yes, the code is always fflushing stdout + <youpi> perhaps trying to put a sleep(1); to check for timing issues? + <teythoon> yes, I suspect something like that + <teythoon> b/c *sometimes* my hurd just freezes at this point + <braunr> ??? + <teythoon> after printing proc auth and not printing the newline + <braunr> such horror stories . + <braunr> .. + <teythoon> and I *think* that putting more printfs there for testing + purposes makes this worse, but I'm not sure about this + <braunr> in case you need to debug using printf, there is the mach_print + system call + <braunr> (in -dbg kernels) +[[microkernel/mach/gnumach/interface/syscall/mach_print]]. -[[!taglink open_issue_documentation]]: Neal once had written an email on this -topic. + <teythoon> what's wrong with using printf? + <braunr> you need to write the little assembly call yourself, where you + intend to use it, because it's not exported by glibc + <braunr> printf is an RPC + <braunr> teythoon: RPCs are complicated stuff + <braunr> in particular in core glibc parts like signal handling + <youpi> and printf goes through the console translator + <braunr> also, if you don't yet have a console server available, it comes + in handy + <youpi> better direct output directly to the kernel -IRC, freenode, #hurd, 2011-11-06 +# `stderr` buffered + +## IRC, freenode, #hurd, 2011-11-06 + +[[!tag open_issue_hurd]] <youpi> about CLI_DEBUG, you can use #define CLI_DEBUG(fmt, ...) { fprintf(stderr, fmt, ## __VA_ARGS__); fflush(stderr); } @@ -40,7 +84,24 @@ IRC, freenode, #hurd, 2011-11-06 <tschwinge> That sounds wrong. -IRC, freenode, #hurd, 2011-11-23 +# Logging + +[[!tag open_issue_hurd]] + +There have been several discussions and proposals already, about adding a +suitable logging mechanism to translators, for example. + +Decide / implement / fix that (all?) running (passive?) translators' output +should show up on the (Mach / Hurd) console / syslog. + +[[!taglink open_issue_documentation]]: [[!message-id +"87oepj1wql.fsf@becket.becket.net"]] + +[[!taglink open_issue_documentation]]: Neal once had written an email on this +topic. + + +## IRC, freenode, #hurd, 2011-11-23 <braunr> we'd need a special logging task for hurd servers <pinotree> if syslog would work, that could be used optionally diff --git a/open_issues/user-space_device_drivers.mdwn b/open_issues/user-space_device_drivers.mdwn index 8cde8281..d6c33d30 100644 --- a/open_issues/user-space_device_drivers.mdwn +++ b/open_issues/user-space_device_drivers.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2009, 2011, 2012 Free Software Foundation, +[[!meta copyright="Copyright © 2009, 2011, 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -154,12 +154,102 @@ A similar problem is described in < braunr> s/disk/storage/ -### IRC, freenode, #hurd, 2012-04-25 +#### IRC, freenode, #hurd, 2012-04-25 <youpi> btw, remember the initrd thing? <youpi> I just came across task.c in libstore/ :) +#### IRC, freenode, #hurd, 2013-06-24 + + <youpi> we added a new initrd command to gnumach, to expose a new mach + device, which ext2fs can open and unzip + <youpi> we consider replacing that with simply putting the data in a dead + process + <youpi> s/process/task + <youpi> and let ext2fs read data from the task, and kill it when done + <teythoon> ok + <youpi> alternatively, tmps would work with an initial .tar.gz payload + <youpi> that would be best for memory usage + <youpi> tmpfs* + <teythoon> can't we replace the initrd concept with sub/neighbourhood? + <youpi> setting up tmpfs with an initial payload could be done with a + bootstrap subhurd + <teythoon> yes + <youpi> but it seems to me that having tmpfs being able to have an initial + payload is interesting + <teythoon> is there any advantage of the tmpfs translator prefilled with a + tarball over ext2fs with copy & bunzip? + <youpi> memory usage + <youpi> ext2fs with copy&bunzip takes memory for zeroes + <youpi> and we have to forecast how much data might be stored + <youpi> (if writable) + <teythoon> ah sure + <teythoon> but why would it have to be in the tmpfs translator? I why not + start the translator and have tar extract stuff there? + <teythoon> with the livecd I had trouble replacing the root translator, but + when using subhurds that shouldn't be a prwoblem at all + <youpi> I don't have a real opinion on this + <youpi> except that people don't usually like initrd :) + <braunr> 12:43 < teythoon> but why would it have to be in the tmpfs + translator? I why not start the translator and have tar extract stuff + there? + <braunr> that sounds an awful lot like an initramfs + <teythoon> yes, exactly, without actually having an initramfs of course + <braunr> yep + <braunr> i actually prefer that way too + <teythoon> a system on a r/o isofs cannot do much, but it can do this + <braunr> on the other hand, i wouldn't spend much time on a virtio disk + driver for now + <braunr> the hurd as it is can't boot on a device that isn't managed by the + kernel + <braunr> we'd need to change the boot protocol + +[[virtio]]. + + +#### IRC, freenode, #hurd, 2013-06-28 + + <teythoon> I'm tempted to redo a livecd, simpler and without the initrd + hack that youpi used for d-i + <braunr> initrd hack ? + <braunr> you mean more a la initramfs then ? + <teythoon> no, I thought about using a r/o isofs translator, but instead of + fixing that one up with a r/w overlay and lot's of firmlinks like I used + to, it would just start an ext2fs translator with copy on an image stored + on the iso and start a subhurd + <braunr> why a subhurd ? + <teythoon> neighbourhurd even + <teythoon> b/c back in the days I had trouble replacing / + <braunr> yes, that's hard + <teythoon> subhurd would take of that for free + <braunr> are you sure ? + <teythoon> somewhat + <braunr> i'm not, but this requires thorough thinking + <braunr> and i'm not there yet + <teythoon> y would it not? + <teythoon> just start a subhurd and let that one take over the console and + let the user and d-i play nicely in that environment + <teythoon> no hacks involved + <braunr> because it would require sharing things between the two system + instances, and that's not easy + <teythoon> no but the bootstrap system does nothing after launching the + subhurd + <teythoon> I mean yes, technically true, but why would it be hard to share + with someone who does nothing? + <braunr> the context isn't well defined enough to clearly state anything + <braunr> if you don't use the resources of the first hurd, that's ok + <braunr> otherwise, it may be easy or not, i don't know yet + <teythoon> you think it's worth a shot and see what issues crop up? + <braunr> sure + <braunr> definitely + <teythoon> it doesn't sound complicated at all + <braunr> it's easy enough to the point we see something goes wrong or works + completely + <braunr> so worth testin + <teythoon> cool :) + + ### IRC, freenode, #hurd, 2012-07-17 <bddebian> OK, here is a stupid question I have always had. If you move @@ -634,3 +724,8 @@ A similar problem is described in * <http://ertos.nicta.com.au/research/drivers/uldd/> * <http://gelato.unsw.edu.au/IA64wiki/UserLevelDrivers> + + * [Running applications on the Xen + Hypervisor](http://blog.netbsd.org/tnf/entry/running_applications_on_the_xen), + Antti Kantee, 2013-09-17. [The Anykernel and Rump + Kernels](http://www.netbsd.org/docs/rump/). diff --git a/open_issues/virtio.mdwn b/open_issues/virtio.mdwn new file mode 100644 index 00000000..8298cbfe --- /dev/null +++ b/open_issues/virtio.mdwn @@ -0,0 +1,208 @@ +[[!meta copyright="Copyright © 2010, 2011, 2012, 2013 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd open_issue_gnumach]] + + +# IRC, freenode, #hurd, 2012-07-01 + +In context of [[DDE]]. + + <braunr> hm, i haven't looked but, does someone know if virtio is included + in netdde ? + <youpi> braunr: nope, there's an underlying virtio layer needed before + + +# IRC, freenode, #hurd, 2013-07-24 + + <teythoon> btw, I'd love to see libvirt support in hurd + <teythoon> I tried to hack up a dde based net translator + <teythoon> afaics they are very much like any other pci device, so the + infrastructure should be there + <teythoon> if anything I expect the libvirt stuff to be more easily + portable + <youpi> what do you mean by "a dde based net translator" ? + <youpi> ah, you mean virtio support in netdde ? + <teythoon> yes + <teythoon> virtio net is present in the kernel version we use for the dde + drivers + <teythoon> so I just copied the dde driver over, but I had no luck + compiling it + <youpi> ok, but what would be the benefice over e1000 & co? + <teythoon> any of the dde drivers btw + <teythoon> youpi: less overhead + <youpi> e1000 is already low overhead actually + <youpi> there are less and less differences in strategies for driving a + real board, and a virtual one + <youpi> we are seeing shared memory request buffer, dma, etc. in real + boards + <youpi> which ends up being almost exactly what virtio does :) + <youpi> ahci, for instance, really looks extremely like a virtio interface + <youpi> (I know, it's a disk, but that's the same idea, and I do know what + I'm talking about here :) ) + <teythoon> that would actually be my next wish, a virtio disk driver, and + virt9p ;) + <braunr> on the other hand, i wouldn't spend much time on a virtio disk + driver for now + <braunr> the hurd as it is can't boot on a device that isn't managed by the + kernel + <braunr> we'd need to change the boot protocol + <teythoon> ok, I wasn't planning to, just wanted to see if I can easily + hack up the virtio-net translator + <braunr> well, as youpi pointed, there is little benefit to that as well + <braunr> but if that's what you find fun, help yourself :) + <teythoon> I didn't know that, I assumed there was some value to the virtio + stuff + <braunr> there is + <braunr> but relatively to other improvements, it's low + + +# IRC, freenode, #hurd, 2013-09-14 + + <rekado> I'm slowly beginning to understand the virtio driver framework + after reading Rusty's virtio paper and the Linux sources of a few virtio + drivers. + <rekado> Has anyone started working on virtio drivers yet? + <youpi> rekado: nobody has worked on virtio drivers, as I know of + <rekado> youpi: I'm still having a hard time figuring out where virtio + would fit in in the hurd. + <rekado> I'm afraid I don't understand how drivers in the hurd work at all. + Will part of this have to be implemented in Mach? + <youpi> rekado: it could be implemented either as a Mach driver, or as a + userland driver + <youpi> better try the second alternative + <youpi> i.e. as a translator + <youpi> sitting on e.g. /dev/eth0 or /dev/hd0 + + +## IRC, freenode, #hurd, 2013-09-18 + + <rekado> To get started with virtio I'd like to write a simple driver for + the entropy device which appears as a PCI device when running qemu with + -device virtio-rng-pci . + <braunr> why entropy ? + <rekado> because it's the easiest. + <braunr> is it ? + <braunr> the driver itself may be, but integrating it within the system + probably isn't + <rekado> It uses the virtio framework but only really consists of a + read-only buffer virtqueue + <braunr> you're likely to want something that can be part of an already + existing subsystem like networking + <rekado> All the driver has to do is push empty buffers onto the queue and + pass the data it receives back from the host device to the client + <rekado> The thing about existing subsystems is: I don't really understand + them enough. + <rekado> I understand virtio, though. + <braunr> but isn't your goal understanding at least one ? + <rekado> yes. + <braunr> then i suggest working on virtio-net + <braunr> and making it work in netdde + <rekado> But to write a virtio driver for network I must first understand + how to actually talk to the host virtio driver/device. + <braunr> rekado: why ? + <rekado> There is still a knowledge gap between what I know about virtio + and what I have learned about the Hurd/Mach. + <braunr> are you trying to learn about virtio or the hurd ? + <rekado> both, because I'd like to write virtio drivers for the hurd. + <braunr> hm no + <rekado> with virtio drivers pass buffers to queues and then notify the + host. + <braunr> you may want it, but it's not what's best for the project + <rekado> oh. + <braunr> what's best is reusing existing drivers + <braunr> we're much too far from having enough manpower to maintain our own + <rekado> you mean porting the linux virtio drivers? + <braunr> there already is a virtio-net driver in linux 2.6 + <braunr> so yes, reuse it + <braunr> the only thing which might be worth it is a gnumach in-kernel + driver for virtio block devices + <braunr> because currently, we need our boot devices to be supported by the + kernel itself ... + <rekado> when I boot the hurd with qemu and the entropy device I see it as + an unknown PCI device in the output of lspci. + <braunr> that's just the lspci database which doesn't know it + <rekado> Well, does this mean that I could actually talk to the device + already? E.g., through libpciaccess? + <rekado> I'm asking because I don't understand how exactly devices "appear" + on the Hurd. + <braunr> it's one of the most difficult topic currently + <braunr> you probably can talk to the device, yes + <braunr> but there are issues with pci arbitration + * rekado takes notes: "pci arbitration" + <rekado> so, this is about coordinating bus access, right? + <braunr> yes + <braunr> i'm not a pci expert so i can't tell you much more + <rekado> heh, okay. + <rekado> what kind of "issues with pci arbitration" are you referring to, + though? + <rekado> Is this due to something that Mach isn't doing? + <braunr> ideally, mach doesn't know about pci + <braunr> the fact we still need in-kernel drivers for pci devices is a big + problem + <braunr> we may need something like a pci server in userspace + <braunr> on l4 system it's called an io server + <rekado> How do in-kernel drivers avoid these issues? + <braunr> they don't + <rekado> Or rather: why is it they don't have these issues? + <braunr> they do + <rekado> oh. + <braunr> we had it when youpi added the sata driver + <braunr> so currently, all drivers need to avoid sharing common interrupts + for example + <braunr> again, since i'm not an expert about pci, i don't know more about + the details + <Hooligan0> pci arbitrations are made by hardware ... no ? + <braunr> Hooligan0: i don't know + <braunr> i'm not merely talking about bus mastering here + <braunr> simply preventing drivers from mapping the same physical memory + should be enforced somewhere + <braunr> i'm not sure it is + <braunr> same for irq sharing + <Hooligan0> braunr : is the support for boot devices into the kernel is + really needed if a loader put servers into the memory before starting + mach ? + <braunr> Hooligan0: there is a chicken-and-egg problem during boot, + whatever the solution + <braunr> obviously, we can preload from memory, but then you really want + your root file system to use a disk + <braunr> Hooligan0: the problem with preloading from memory is that you + want the root file system to use a real device + <braunr> the same way / refers to one on unix + <braunr> so you have an actual, persistent hierarchy from which the system + can be initialized and translators started + <braunr> you also want to share as much as possible between the early + programs and the others + <braunr> so for example, both the disk driver and the root file system + should be able to use the same libc instance + <braunr> this requires a "switch root" mechanism that needs to be well + defined and robust + <braunr> otherwise we'd just build our drivers and root fs statically + <braunr> (which is currently done with rootfs actually) + <braunr> and this isn't something we're comfortable with + <braunr> so for now, in-kernel drivers + <Hooligan0> humm ... disk driver and libc ... i see + <Hooligan0> in other way ... disk drivers can use only a little number of + lib* functions ; so with a static version, a bit of memory is lots + <Hooligan0> s/lots/lost + <Hooligan0> and maybe the driver can be hot-replaced after boot (ok ok, + it's more simple to say than to write) + + +<a name="KVM"></a> +# Virtio Drivers for KVM + +In context of [[hurd/running/cloud]], *OpenStack*. + +Ideally they would be userland. That means getting documentation about how +virtio works, and implement it. The hurdish part is mostly about exposing the +driver interface. The [[hurd/translator/devnode]] translator can be used as a +skeleton. diff --git a/open_issues/virtualization/fakeroot.mdwn b/open_issues/virtualization/fakeroot.mdwn index f4739776..f9dd4756 100644 --- a/open_issues/virtualization/fakeroot.mdwn +++ b/open_issues/virtualization/fakeroot.mdwn @@ -22,3 +22,46 @@ License|/fdl]]."]]"""]] <youpi> btw, I believe our fakeroot-hurd is close to working actually <youpi> it's just a argv[0] issue supposed to be fixed by exec_file_name but apparently not fixed in that case, for some reason + + +## IRC, freenode, #hurd, 2013-08-26 + + < teythoon> also I looked into the fakeroot issue, aiui the problem is that + scripts are not handled correctly, right? + < teythoon> the exec server fails to locate the scripts file name, and so + it hands the file_t to the interpreter process and passes /dev/fds/3 as + script name + < teythoon> afaics that breaks e.g. python + < youpi> yes + < youpi> pinotree's exec_file_name is supposed to fix that, but for some + reason it doesn't work here + < pinotree> it was pochu's, not mine + < youpi> ah, right + < teythoon> ah I see, I was wondering about that + < pochu> it was working for a long time, wasn't it? + < pochu> and only stopped working recently + < youpi> did it completely stop? + < youpi> I have indeed seen odd issues + < youpi> I haven't actually checked whether it has completely stopped + working + < youpi> probably worth looking there first + < pinotree> gtk+3.0 fails, but other stuff like glib2.0 and gtester-using + stuff works + < teythoon> huh? I created tests like "#!/bin/sh\necho $0" and that says + /dev/fd..., and a python script doing the same doesn't even run, so how + can it work for a package build? + < youpi> it works for me in plain bash + < youpi> #!/bin/sh + < youpi> echo $0 + < youpi> € $PWD/test.sh + < youpi> /home/samy/test.sh + < teythoon> it does !? + < youpi> yes + < youpi> not in fakeroot-hurd however, as we said + < teythoon> well, obviously it works when not being run under + fakeroot-hurd, yes + < youpi> ok, so we weren't talking about the same thing + < youpi> a mere shell script doesn't work in fakeroot-hurd indeed + < youpi> that's why we still use fakeroot-sysv + < teythoon> right + < youpi> err, -tcp diff --git a/open_issues/virtualization/networking.mdwn b/open_issues/virtualization/networking.mdwn index 7a6474a1..f8bda063 100644 --- a/open_issues/virtualization/networking.mdwn +++ b/open_issues/virtualization/networking.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -28,3 +28,73 @@ Collection about stuff that is relevant for *virtualization* and *networking*. [[hurd/translator/pfinet]] by setting environment variables. Project is now part of [[Virtual_Square_View-OS]]. + + +# OpenVPN + +[[community/meetings/GHM2013]]. + + +## IRC, freenode, #hurd, 2013-08-23 + + <youpi> good news + <youpi> with a couple small patches, openvpn does work as joe user + + +## IRC, freenode, #hurd, 2013-08-30 + + <youpi> it's really cool that openvpn ended up working completely the day + before :) + + +## IRC, freenode, #hurd, 2013-09-03 + + <_d3f> Hey guys, how did you get openvpn working on the Hurd? just curious + as I saw it in the GHM video + <_d3f> no one here who has a clue how to get *vpn working on the Hurd? + <braunr> _d3f: youpi did it + <braunr> i don't know the details + <_d3f> okay, I will question him when I see him around, thx. Do you know if + it was a lot of work to get the tun device working? Because I would like + to use tinc on the Hurd. + <braunr> _d3f: a bit but not that much either + <_d3f> braunr: well, okay. Do you know if the source of his 'port' is + online, I haven't found it :/ + <braunr> it should be soon + + +## IRC, freenode, #hurd, 2013-09-04 + + <_d3f> youpi: you are the guy who has brought openvpn to the hurd, right? I + would like to know how you got the tun/tap thing working as I would like + to use tinc on it. :) + <youpi> _d3f: essentially no modification of openvpn iirc + <youpi> just tell it to open the tun node created by pfinet + <youpi> and read/write it + <youpi> i.e. the existing generic code in place in openvpn + <_d3f> I will have a look at it, somekind tinc builds with the linux + specific device.c but I wasn't able to exchange keys. I will have a look + at the device handling again and try to get the pfinet tun node used. + + +## IRC, freenode, #hurd, 2013-09-07 + + <d3f> anyone here knows how /dev/net is handled on the hurd? Programs using + it say it's not a directory. I tried creating one and setting a netdde + translator for a tun device in it, but this may be wrong as it doesn't + work + <teythoon> d3f: what does /dev/net do? + <teythoon> ah, its tun/tap stuff... + <d3f> on my gnu/linux it includes a tun device + <teythoon> right + <d3f> I am still reading about the Hurd and try to understand /hurd/netdde + and devnode but by now I am quite sure I will need those to set a tun + networktranslator on /dev/net/tun? + <teythoon> hm, I don't think netdde or devnode will be of any help + <teythoon> afaiui devnode makes mach devices available in the hurdish way, + i.e. available for lookup in the filesystem + <teythoon> d3f: ping youpi if he shows up, he hacked up openvpn to work on + the hurd + <d3f> yeah I know, I talked to him as I am tring to get tinc working on the + Hurd (tinc builds by now). I will give him a shot about creating the + "tun" device |