diff options
Diffstat (limited to 'open_issues')
40 files changed, 4629 insertions, 108 deletions
diff --git a/open_issues/anatomy_of_a_hurd_system.mdwn b/open_issues/anatomy_of_a_hurd_system.mdwn index 75a62535..ba72b00f 100644 --- a/open_issues/anatomy_of_a_hurd_system.mdwn +++ b/open_issues/anatomy_of_a_hurd_system.mdwn @@ -660,3 +660,146 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l <braunr> but as youpi said, it still requires work <braunr> and nobody's working on it <braunr> you may want to check l4 fiasco.oc though + + +# System Personality + +## IRC, freenode, #hurd, 2013-07-29 + + <teythoon> over the past few days I gained a new understanding of the Hurd + <braunr> teythoon: really ? :) + <tschwinge> teythoon: That it's a complex and distributed system? ;-) + <tschwinge> And at the same time a really simple one? + <tschwinge> ;-D + <teythoon> it's just a bunch of mach programs and some do communicate and + behave in a way a posix system would, but that is more a convention than + anything else + <teythoon> tschwinge: yes, kind of simple and complex :) + <braunr> the right terminology is "system personality" + <braunr> 11:03 < teythoon> over the past few days I gained a new + understanding of the Hurd + <braunr> teythoon: still no answer on that :) + <teythoon> braunr: ah, I spent lot's of time with the core servers and + early bootstrapping and now I gained the feeling that I've seen the Hurd + for what it really is for the first time + + +# RPC Interfaces + +## IRC, freenode, #hurd, 2013-09-03 + + <rekado> I'm a little confused by the hurd and incubator git repos. + <rekado> DDE is only found in the dde branch in incubator, but not in the + hurd repo. + <rekado> Does this mean that DDE is not ready for master yet? + <braunr> yes + <rekado> If DDE is not yet used in the hurd (except in the dde branch in + the incubator repo), does pfinet use some custom glue code to use the + Linux drivers? + <braunr> this has nothing to do with pfinet + <braunr> pfinet is the networking stack, netdde are the networking drivers + <braunr> the interface between them doesn't change, whether drivers are in + kernel or not + <rekado> I see + + +# IRC, freenode, #hurd, 2013-09-20 + + <giuscri> HI there, I have no previous knowledge about OS's. I'm trying to + undestand the structure of the Hurd and the comparison between, say, + Linux way of managing stuff ... + <giuscri> for instance, I read: "Unlike other popular kernel software, the + Hurd has an object-oriented structure that allows it to evolve without + compromising its design." + <giuscri> that means that while for adding feature to the Linux-kernel you + have to add some stuff `inside` a procedure, whilst in the Hurd kernel + you can just, in principle at least, add an object and making the kernel + using it?... + <giuscri> Am I making stuff too simple? + <giuscri> Thanks + <braunr> not exactly + <braunr> unix historically has a "file-oriented" structure + <braunr> the hurd allows servers to implement whatever type they want, + through the ability to create custom interfaces + <braunr> custom interfaces means custom calls, custom semantics, custom + methods on objects + <braunr> you're not restricted to the set of file interfaces (open, seek, + read, write, select, close, etc..) that unix normally provides + <giuscri> braunr: uhm ...some example? + <braunr> see processes for example + <braunr> see + http://darnassus.sceen.net/gitweb/savannah_mirror/hurd.git/tree/HEAD:/hurd + <braunr> this is the collection of interfaces the hurd provides + <braunr> most of them map to unix calls, because gnu aims at posix + compatibility too + <braunr> some are internal, like processes + <braunr> or authentication + <braunr> but most importantly, you're not restricted to that, you can add + your own interfaces + <braunr> on a unix, you'd need new system calls + <braunr> or worse, extending through the catch-all ioctl call + <giuscri> braunr: mhn ...sorry, not getting that. + <braunr> what part ? + <kilobug> ioctl has become such a mess :s + <giuscri> braunr: when you say that Unix is `file-oriented` you're + referring to the fact that sending/receiving data to/from the kernel is + designed like sending/receiving data to/from a file ...? + <braunr> not merely sending/receiving + <braunr> note how formatted your way of thinking is + <braunr> you directly think in terms of sending/receiving (i.e. read and + write) + <giuscri> braunr: (yes) + <braunr> that's why unix is file oriented, access to objects is done that + way + <braunr> on the hurd, the file interface is one interface + <braunr> there is nothing preventing you from implementing services with a + different interface + <braunr> as a real world example, people interested in low latency + profesionnal audio usually dislike send/recv + <braunr> see + http://lac.linuxaudio.org/2003/zkm/slides/paul_davis-jack/unix.html for + example + <kilobug> braunr: how big and messy ioctl has become is a good proof that + the Unix way, while powerful, does have its limits + <braunr> giuscri: keep in mind the main goal of the hurd is extensibility + without special privileges + <giuscri> braunr: privileges? + <braunr> root + <giuscri> braunr: what's wrong with privileges? + <braunr> they allow malicious/buggy stuff to happne + <braunr> and have dramatic effects + <giuscri> braunr: you're obviously *not* referring to the fact that once + one have the root privileges could change some critical-data + <giuscri> ? + <braunr> i'm referring to why privilege separation exists in the first + place + <braunr> if you have unprivileged users, that's because you don't want them + to mess things up + <braunr> on unix, extending the system requires privileges, giving those + who do it the ability to destroy everything + <giuscri> braunr: yes, I think the same + <braunr> the hurd is designed to allow unprivileged users to extend their + part of the system, and to some extent share that with other users + <braunr> although work still remains to completely achieve that + <giuscri> braunr: mhn ...that's the `server`-layer between the + single-application and kernel ? + <braunr> the multi-server based approach not only allows that, but + mitigates damage even when privileged servers misbehave + <braunr> one aspect of it yes + <braunr> but as i was just saying, even root servers can't mess things too + much + <braunr> for example, our old (sometimes buggy) networking stack can be + restarted when it behaves wrong + <braunr> the only side effect being some applications (ssh and exim come to + mind) which need to be restarted too because they don't expect the + network stack to be restarted + <giuscri> braunr: ...instead? + <braunr> ? + <kilobug> giuscri: on Linux, if the network stack crash/freezes, you don't + have any other option than rebooting the system - usually with a nice + "kernel pani" + <kilobug> giuscri: and you may even get filesystem corruption "for free" in + the bundle + <braunr> and hoping it didn't corrupt something important like file system + caches before being flushed + <giuscri> kilobug, braunr : mhn, ook diff --git a/open_issues/arm_port.mdwn b/open_issues/arm_port.mdwn index b07df939..ebbad1a4 100644 --- a/open_issues/arm_port.mdwn +++ b/open_issues/arm_port.mdwn @@ -273,3 +273,56 @@ architecture. <slpz> braunr: OK, thanks. I'm interested on it, and didn't want to duplicate efforts. <braunr> little addition: it may have started, but we don't know about it + + +# IRC, freenode, #hurd, 2013-09-18 + + <Hooligan0> as i understand ; on startup, vm_resident.c functions configure + the whole available memory ; but at this point the system does not split + space for kernel and space for future apps + <Hooligan0> when pages are tagged to be used by userspace ? + <braunr> Hooligan0: at page fault time + <braunr> the split is completely virtual, vm_resident deals with physical + memory only + <Hooligan0> braunr: do you think it's possible to change (at least) + pmap_steal_memory to mark somes pages as kernel-reserved ? + <braunr> why do you want to reserve memory ? + <braunr> and which memory ? + <Hooligan0> braunr: first because on my mmu i have two entry points ; so i + want to set kernel pages into a dedicated space that never change on + context switch (for best cache performance) + <Hooligan0> braunr: and second, because i want to use larger pages into + kernel (1MB) to reduce mmu work + <braunr> vm_resident isn't well suited for large pages :( + <braunr> i don't see the effect of context switch on kernel pages + <Hooligan0> at many times, context switch flush caches + <braunr> ah you want something like global pages on x86 ? + <Hooligan0> yes, something like + <braunr> how is it done on arm ? + <Hooligan0> virtual memory is split into two parts depending on msb bits + <Hooligan0> for example 3G/1G + <Hooligan0> MMU will use two pages tables depending on vaddr (hi-side or + low-side) + <braunr> hi is kernel, low is user ? + <Hooligan0> so, for the moment i've put mach at 0xC0000000 -> 0xFFFFFFFF ; + and want to use 0x00000000 -> 0xBFFFFFFF for userspace + <Hooligan0> yes + <braunr> ok, that's what is done for x86 too + <Hooligan0> 1MB pages for kernel ; and 4kB (or 64kB) pages for apps + <braunr> i suggest you give up the large page stuff + <braunr> well, you can use them for the direct physical mapping, but for + kernel objects, it's a waste + <braunr> or you can rewrite vm_resident to use something like a buddy + allocator but it's additional work + <Hooligan0> for the moment it's waste ; but with some littles changes this + allow only one level of allocation mapping ; -i think- it's better for + performances + <braunr> Hooligan0: it is, but not worth it + <Hooligan0> will you allow changes into vm_resident if i update i386 too ? + <braunr> Hooligan0: sure, as long as these are relevant and don't introduce + regressions + <Hooligan0> ok + <braunr> Hooligan0: i suggest you look at x15, since you may want to use it + as a template for your own changes + <braunr> as it was done for the slab allocator for example + <braunr> e.g. x15 already uses a buddy allocator for physical memory diff --git a/open_issues/boehm_gc.mdwn b/open_issues/boehm_gc.mdwn index 7f860bba..623dcb83 100644 --- a/open_issues/boehm_gc.mdwn +++ b/open_issues/boehm_gc.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2010, 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2012, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -433,3 +434,92 @@ restults of GNU/Linux and GNU/Hurd look very similar. <civodul> pinotree: is it a Debian-specific change, or included upstream? <pinotree> libgc using SIGUSR1/2? upstream <civodul> ok + + +### IRC, freenode, #hurd, 2013-09-03 + + <congzhang> braunr: when will libc malloc say memory corruption? + <braunr> congzhang: usually on free + <braunr> sometimes on alloc + <congzhang> and after one thread be created + <congzhang> I want to know why and how to find the source + <congzhang> does libgc work well on hurd? + <braunr> i don't think it does + <congzhang> so , why it can't? + <braunr> congzhang: what ? + <congzhang> libgc was not work on hurd + <pinotree> why? + <congzhang> I try porting dotgnu + <braunr> ah + <braunr> nested signal handling + <congzhang> one program always receive Abort signal + <pinotree> and why it should be a problem in libgc? + <congzhang> for malloc memory corruption + <braunr> libgc relies on this + <congzhang> yes + <congzhang> so, is there a workaround to make it work? + <braunr> show the error please + <congzhang> http://paste.debian.net/34416/ + <pinotree> where's libgc? + <congzhang> i compile dotgnu with enable-gc + <pinotree> so? + <congzhang> I am not sure about it + <pinotree> so why did you say earlier that libgc doesn't work? + <congzhang> because after I see one thread was created notice by gdb, it + memory corruption + <pinotree> so what? + <congzhang> maybe gabage collection happen, and gc thread start + <pinotree> that's speculation + <pinotree> you cannot debug things speculating on code you don't know + <pinotree> less speculation and more in-deep debugging, please + * congzhang I try again, to check weather thread list changing + <congzhang> sorry for this + <braunr> it simply looks like a real memory corruption (an overflow) + <congzhang> maybe PATH related problem + <pinotree> PATH? + <congzhang> yes + <braunr> PATH_MAX + <braunr> but unlikely + <congzhang> csant do path traverse + <congzhang> I fond the macro + <congzhang> found + <congzhang> #if defined(__sun__) || defined(__BEOS__) + <congzhang> #define BROKEN_DIRENT 1 + <congzhang> #endif + <congzhang> and so for hurd? + <pinotree> BROKEN_DIRENT doesn't say much about what it does + <WhiteKIBA> nope + <WhiteKIBA> whoops + <congzhang> it seems other port meet the trouble too + <pinotree> which trouble? + <congzhang> http://comments.gmane.org/gmane.comp.gnu.dotgnu.developer/3642 + <congzhang> (gdb) ptype struct dirent + <congzhang> type = struct dirent { + <congzhang> __ino_t d_ino; + <congzhang> unsigned short d_reclen; + <congzhang> unsigned char d_type; + <congzhang> unsigned char d_namlen; + <congzhang> char d_name[1]; + <congzhang> } + <congzhang> + <congzhang> d_name should be char[PATH_MAX]? + <congzhang> and + http://libjit-linear-scan-register-allocator.googlecode.com/svn/trunk/pnet/support/dir.c + <pinotree> no + <braunr> stop pasting that much + <_d3f> uhm PATH_MAX on the hurd? + <braunr> and stop saying nonsense + <congzhang> sorry, i think four line was not worth to pastbin + <pinotree> they are 8 + <congzhang> never again + <braunr> just try by defining BROKEN_DIRENT to 1 in all cases and see how + it goes + * congzhang read dir.c again + <congzhang> braunr: it does not crash this time, I do more test + + +#### IRC, freenode, #hurd, 2013-09-04 + + <congzhang> hi, I am dotgnu work on hurd, and even winforms app + <congzhang> s/am/make + <congzhang> and maybe c# hello world translate another day :) diff --git a/open_issues/clock_gettime.mdwn b/open_issues/clock_gettime.mdwn index 98454d45..65ab52df 100644 --- a/open_issues/clock_gettime.mdwn +++ b/open_issues/clock_gettime.mdwn @@ -197,4 +197,14 @@ In context of [[select]]. "atomic" update of the struct with time :) +# IRC, freenode, #hurd, 2013-09-04 + + <teythoon> do we have CLOCK_MONOTONIC ? + <braunr> teythoon: i think we do but it's actually a simple offset from + CLOCK_REALTIME .. :) + <teythoon> ah never mind, I do hate this posix time interface anyways + <braunr> really ? + <braunr> i think librt is decent + + # Candidate for [[vDSO]] code? diff --git a/open_issues/cloud.mdwn b/open_issues/cloud.mdwn new file mode 100644 index 00000000..58ed2f5b --- /dev/null +++ b/open_issues/cloud.mdwn @@ -0,0 +1,49 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +Some *cloud*y things. + +[[!toc]] + + +# [[!wikipedia OpenStack]] + +## IRC, freenode, #hurd, 2013-09-21 + + <jproulx> Hmmm, was hoping to run hurd on my kvm based openstack cloud, but + no virtio. + <jproulx> I see "Write virtio drivers for KVM. Ideally they would be + userland" is listed as a "small hack", as a sysadmin rather than an OS + hacker it doesn't sound small to me, but if there's some standard + documentation on porting drivers I could take a run at it. + <youpi> well, perhaps "small" is not the proper word + <youpi> compared to e.g. revamping disk i/o :) + <youpi> it's not something one can achieve in e.g. 1h, for instance + <youpi> it's not something straightforward either, one has to get + documentation about virtio (I don't know what exists), and get + documentation about the mach device interface (that's in the gnumach + manual, the devnode translator can be used as a skeleton) + <youpi> jproulx: openstack imposes the use of virtio drivers? that's odd + <jproulx> that's more like I'd expect. I there's enough search terms in + your response for me to see what's really involved + <jproulx> youpi it doesn't impose that but it is how mine is configured the + other thousand VMs are happier that way. + <jproulx> I can look at that side too and see if I need to have everything + use the same device settings or if I can control it per instance + <jproulx> A bit of a non-sequitur at this point but just in case someone + searches the transcripts and sees my questions about hurd on openstack, + yes it is possible to specify non-virtio devices per image, here's the + commandline to load sthibault's qemu image into openstack with devices + that work: + <jproulx> glance image-create --property hw_disk_bus=ide --property + hw_cdrom_bus=ide --property hw_vif_model=rtl8139 --disk-format raw + --container-format bare --name gnu-hurd --copy-from + http://people.debian.org/~sthibault/hurd-i386/debian-hurd.img + <youpi> jproulx: thanks, I've pushed it on the wiki diff --git a/open_issues/crash_server.mdwn b/open_issues/crash_server.mdwn index 7ed4afbf..5182df6f 100644 --- a/open_issues/crash_server.mdwn +++ b/open_issues/crash_server.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2009, 2010, 2011 Free Software Foundation, +[[!meta copyright="Copyright © 2009, 2010, 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -189,6 +189,65 @@ one... mach_msg_trap /home/tschwinge/tmp/gnumach/gnumach-1-branch-Xen-branch.build/../gnumach-1-branch-Xen-branch/ipc/mach_msg.c:1367 + +# IRC, freenode, #hurd, 2013-09-07 + + <rekado> I'm trying to investigate a crash in pfinet, so it will actually + die. I just want to know why it dies and what the value of a few + variables has been when it died. + <teythoon> have you tried to make it dump core? + <rekado> oh, good idea. + <rekado> I'll try that. + <teythoon> do you know how? + <rekado> I don't, but I think I can figure it out. + <teythoon> look into /servers + <rekado> do I just have to set CRASHSERVER=/servers/crash-dump-core and run + pfinet in that environment? + <teythoon> possibly, I've never heard of CRASHSERVER, but it's certainly + plausible ;) + <teythoon> I just link crash to crash-dump-core, that way it is permanent + and for all processes + <rekado> found it in the website contents + <rekado> gotta try that. + <rekado> hmm, I can't get pfinet to dump core; linked /servers/crash to + /servers/crash-dump-core and compiled pfinet to raise(6) at one point. + <rekado> But no core file is created. + <teythoon> :/ + <teythoon> rekado: try cd /tmp ; cat & kill -SIGILL %% to see if that dumps + core + <rekado> yes, this works. + <rekado> I replaced the original pfinet with my crashing version. + <rekado> Should it dump core to /hurd then? + <teythoon> I'm not sure about it's wd + <teythoon> hm, ok, I just did settrans -ca foo /hurd/pfinet and then killed + that pfient with SIGILL and it dumped core + <teythoon> to the directory I issued the settrans from + <rekado> So I must run it myself. I can't just replace the original binary + and have it dump core somewhere. + <teythoon> it seems that you have to use settrans -ca to start an active + translator + <teythoon> do fsysopts /servers/socket/2 to find out the cmdline of your + pfinet + <rekado> that's very helpful. + <rekado> thanks + <teythoon> then use this to restart it, e.g.: + <teythoon> settrans -afg /servers/socket/2 $(fsysopts /servers/socket/2) + <teythoon> if it dies it should dump core to you cwd + <rekado> great. Thank you very much. I had been wondering how to get the + full cmdline of pfinet. + * rekado makes a note of fsysopts + <rekado> yup, there's the core file. Nice. + <teythoon> cool 8D + <teythoon> btw, in case using gdb doesn't work out for your problem, if you + start pfinet (or any translator) this way (with -a == active), you can + write stuff to stderr + <rekado> yeah, I noticed that. The assert() call wrote to stderr. Useful. + <braunr> rekado: core dumps are another not-working-well feature of the + hurd :/ + <braunr> i recommend attaching + <tschwinge> rekado: In case that's still helpful: + <http://www.gnu.org/software/hurd/hurd/debugging/translator.html>. + --- If someone is working in this area, they may want to have a look at diff --git a/open_issues/dbus.mdwn b/open_issues/dbus.mdwn index 2f02579e..a41515a1 100644 --- a/open_issues/dbus.mdwn +++ b/open_issues/dbus.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2012, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -14,15 +15,17 @@ The dbus problems are due to missing scm credentials [[sendmsg_scm_creds]] and s [[pflocal_socket_credentials_for_local_sockets]]. There was also a problem with short timeout in [[select]], but that has been solved in Debian by setting a minimum timeout of 1ms. ---- +[[!toc]] -IRC, freenode, #hurd, 2011-11-26: + +# IRC, freenode, #hurd, 2011-11-26 <antrik> BTW, how much effort is necessary to fix dbus? <pinotree> basically, have pflocal know who's the sender (pid/uid/gid/groups) in the socket send op -IRC, freenode, #hurd, 2011-12-16: + +# IRC, freenode, #hurd, 2011-12-16 <braunr> pinotree: what's the problem with dbus ? <pinotree> braunr: select() returning 0 changed fd's with very short (eg < @@ -53,7 +56,8 @@ IRC, freenode, #hurd, 2011-12-16: <braunr> hm i agree with neal, i don't understand why the timeout is given to the kernel as part of the mach_msg call -IRC, freenode, #hurd, 2011-12-20: + +# IRC, freenode, #hurd, 2011-12-20 <braunr> hm, i don't see any occurrence of SCM_CREDENTIALS in dbus <braunr> only SCM_RIGHTS @@ -88,3 +92,164 @@ IRC, freenode, #hurd, 2011-12-20: <pinotree> iirc roland didn't like one or more parts of it (but i could be wrong) <braunr> ok + + +# IRC, freenode, #hurd, 2013-07-17 + + <teythoon> btw pinotree, what happened to your efforts to make dbus work? + <pinotree> not much, my initial patch was just a crude hack, a better + solution requires more thinkering and work + <teythoon> yes, ive seen that + <teythoon> but that was only a tiny patch against the libc, surely there + must be more to that? + <pinotree> not really + <teythoon> and the proper fix is to patch pflocal to query the auth server + and add the credentials? + <pinotree> possibly + <teythoon> that doesn't sound to bad, did you give it a try? + <pinotree> not really, got caught in other stuff + + +# IRC, freenode, #hurd, 2013-09-02 + + <gnu_srs1> something is wrong with libc0.3 since the switch to 2.17. dbus + does not run any longer when rebuilt + <gnu_srs1> the latest build of dbus was with 2.13: libc0.3-dev: already + installed (2.13-38) + <pinotree> debug it + <gnu_srs1> Yes, I will. Maybe somebody could rebuild it and verify my + findings? + <pinotree> gnu_srs1: your finding is "doesn't work", which is generic and + does not help without investigation + <gnu_srs1> just rebuild it and: e.g. ./build-debug/bus/dbus-daemon --system + (--nofork) + <pinotree> gnu_srs1: please, debug it + <gnu_srs1> I have partially already. But maybe the problems only shows on + my box. I'll rebuild on another box before continuing debugging. + <pinotree> gnu_srs1: are you, by chance, running a libc or something else + with your scm_creds work? + <gnu_srs1> I did, but I've backed to 2.17-92 right now. + <gnu_srs1> sane problem with dbus on another box, something's fishy:-( + <gnu_srs1> braunr: any good way to find out if the dbus problems are with + libpthread? Setting a breakpoint with libc0.3-dbg installed. + <braunr> gnu_srs1: i don't know + +See [[glibc]], *Missing interfaces, amongst many more*, *`SOCK_CLOEXEC`*. + + +# IRC, freenode, #hurd, 2013-09-04 + + <gnu_srs> Hi, looks like dbus requires abstract socket namespace: #undef + HAVE_ABSTRACT_SOCKETS What's missing? + <pinotree> uh? + <pinotree> abstract unix sockets are a Linux feature, and surely it is not + mandatory for dbus + <gnu_srs> Looks like dbus exits if they are not supported: + <gnu_srs> dbus_set_error (error, DBUS_ERROR_NOT_SUPPORTED, "Operating + system does not support abstract socket namespace\n"); _dbus_close + (listen_fd, NULL); 1061 return -1; + <pinotree> that is enclosed in a if (abstract) + <pinotree> and that parameter is set to true in other places (eg + dbus/dbus-server-unix.c) only when HAVE_ABSTRACT_SOCKETS is defined + <pinotree> so no, abstract sockets are not mandatory + <gnu_srs> Well this code is executed e.g. when running emacs remotely in + X. Have to dig deeper then to see why. + <pinotree> maybe it could have to do the fact that your dbus server is + running in linux and runs by default using such sockets type + <pinotree> but yes, you need to dig better + <gnu_srs> pinotree: You are right. when running natively the problem is: + <pinotree> *drums* + <gnu_srs> Manually: Process /usr/lib/at-spi2-core/at-spi-bus-launcher + exited with status 1 + <pinotree> eh? + <gnu_srs> Error retrieving accessibility bus address: + org.freedesktop.DBus.Error.Spawn.ChildExited: ^ + <pinotree> most probably that service does not start due to the lack of + socket credentials which affects dbus + <pinotree> uninstall or disable those additional services, they are not + your problem + <gnu_srs> credentials is enabled. which services to remove? + <pinotree> dunno + + +# IRC, freenode, #hurd, 2013-09-11 + + <gnu_srs> Hi, looks like frebsd had (2008) the same problem as hurd when + sending credentials over PF_INET: + <gnu_srs> + http://lists.freebsd.org/pipermail/freebsd-hackers/2008-May/024577.html + <gnu_srs> Since the dbus code is about the same now (2013), maybe they + added support? + <gnu_srs> The next message in the thread confirms that the dbus code is + invalid, does anybody have pointers? + <pinotree> from what i've seen so far, socket credentials are done only for + local sockets (ie PF_UNIX) + <pinotree> i don't see how things like uid/gid/pid of the socket endpoint + can have anything to do with AF_INET + <pinotree> and socket credentials in dbus are used only in the [local] + socket transport, so there's no issue + + +# IRC, freenode, #hurd, 2013-09-12 + + <gnu_srs> pinotree: Yes, there is an issue with dbus and AF_INET, see + test/corrupt.c: tests /corrupt/tcp and /corrupt/byte-order/tcp:-/ + <pinotree> gnu_srs: what's wrong with those? they are just testing the + connection over a tcp socket + <pinotree> as said above, socket credentials shouldn't be used in such + cases + <gnu_srs> They are, see also test/relay.c: /relay and /limit tests:-( + <pinotree> how are they? + <pinotree> please be more specifc... + <gnu_srs> Just run the tests yourself with DBUS_VERBOSE=1 + <pinotree> you are claiming there is a problem, so please specify what is + the actual issue + <gnu_srs> DBUS_VERBOSE=1 build-debug/test/test-relay + <pinotree> you are claiming there is a problem, so please specify what is + the actual issue + <gnu_srs> same with test-corrupt + <gnu_srs> look at the verbose output: Failed to write credentials: Failed + to write credentials byte: Invalid argument + <gnu_srs> coming from pfinet since PF_INET is used. + <pinotree> check what it does on linux then + <pinotree> put an abort() at the start of the read/write socket credential + functions in dbus-sysdeps-unix.c and see whether it is triggered also on + linux + <gnu_srs> SO_PEERCRED is used for linux and LOCAL_CREDS is used for + kfreebsd, so we are on our own here:-/ + <pinotree> and linux' SO_PEERCRED works also on AF_INET sockets? i'd doubt + it + <gnu_srs> + http://stackoverflow.com/questions/10037086/so-peercred-vs-scm-credentials-why-there-are-both-of-them + <pinotree> yes, i know the difference, but please read what i asked again + <gnu_srs> I'll check to be sure... + <braunr> gnu_srs: user credentials are not supposed to be passed through an + AF_INET socket + <braunr> how hard is that to understand ? + <gnu_srs> OK, linux use send since CMSGCREDS is not defined to write + credentials. Working on how they are received. + <gnu_srs> braunr: I do understand, but the dbus code tries to do that for + Hurd:-( + <pinotree> then it should do that on linux too + <pinotree> (since the local socket credentials code is isolated in own + functions, and they are called only for the unix transport) + <gnu_srs> Happiness:-D, almost all dbus tests pass! + <gnu_srs> 17(17) dbus tests pass:) + <braunr> gnu_srs: hopefully your patch does things right + <gnu_srs> which patch + <braunr> adding credentials through unix socket + <braunr> isn't that what you're doing ? + <gnu_srs> the mail to MLs is from the stock installed packages. + <braunr> ? + <gnu_srs> the test reports are with the SCM_CREDS patches, but I stumbled + on the SCM_RIGHTS issues reported to MLs + <gnu_srs> no patches applied, just test the attached file yourself. + <braunr> so what's your work about ? + <gnu_srs> I'm working on SCM_CREDS, yes, and created patches for dbus, + glib2.0 and libc. + <gnu_srs> the mail was about some bug in the call to io_restrict_auth in + sendmsg.c: without any of my patches applied (another image) + <teythoon> gnu_srs: you have to give us more context, how are we supposed + to know how to find this sendmsg.c file? + <pinotree> (it's in glibc, but otherwise the remark is valid) + <pinotree> s/otherwise/anyway/ diff --git a/open_issues/dde.mdwn b/open_issues/dde.mdwn index 76b80211..9cb31d1c 100644 --- a/open_issues/dde.mdwn +++ b/open_issues/dde.mdwn @@ -512,6 +512,18 @@ After the microkernel devroom at [[community/meetings/FOSDEM_2013]]. <antrik> hm... good point +## IRC, freenode, #hurd, 2013-09-20 + + <braunr> i should take some time to integrate my pcap changes into the + libpcap debian package at least + <pinotree> braunr: if upstream is active, i'd say to go there directly + <braunr> the problem with that approach is that netdde is still not part of + our upstream code + <pinotree> don't understand the relation + <braunr> i don't want to send the pcap guys code for an interface that is + still not considered upstream ... + + # IRC, freenode, #hurd, 2012-08-14 <braunr> it's amazing how much code just gets reimplemented needlessly ... @@ -642,3 +654,135 @@ In context of [[libpthread]]. stuff <braunr> there is <braunr> but relatively to other improvements, it's low + + +## IRC, freenode, #hurd, 2013-09-14 + + <rekado> I'm slowly beginning to understand the virtio driver framework + after reading Rusty's virtio paper and the Linux sources of a few virtio + drivers. + <rekado> Has anyone started working on virtio drivers yet? + <youpi> rekado: nobody has worked on virtio drivers, as I know of + <rekado> youpi: I'm still having a hard time figuring out where virtio + would fit in in the hurd. + <rekado> I'm afraid I don't understand how drivers in the hurd work at all. + Will part of this have to be implemented in Mach? + <youpi> rekado: it could be implemented either as a Mach driver, or as a + userland driver + <youpi> better try the second alternative + <youpi> i.e. as a translator + <youpi> sitting on e.g. /dev/eth0 or /dev/hd0 + + +## IRC, freenode, #hurd, 2013-09-18 + + <rekado> To get started with virtio I'd like to write a simple driver for + the entropy device which appears as a PCI device when running qemu with + -device virtio-rng-pci . + <braunr> why entropy ? + <rekado> because it's the easiest. + <braunr> is it ? + <braunr> the driver itself may be, but integrating it within the system + probably isn't + <rekado> It uses the virtio framework but only really consists of a + read-only buffer virtqueue + <braunr> you're likely to want something that can be part of an already + existing subsystem like networking + <rekado> All the driver has to do is push empty buffers onto the queue and + pass the data it receives back from the host device to the client + <rekado> The thing about existing subsystems is: I don't really understand + them enough. + <rekado> I understand virtio, though. + <braunr> but isn't your goal understanding at least one ? + <rekado> yes. + <braunr> then i suggest working on virtio-net + <braunr> and making it work in netdde + <rekado> But to write a virtio driver for network I must first understand + how to actually talk to the host virtio driver/device. + <braunr> rekado: why ? + <rekado> There is still a knowledge gap between what I know about virtio + and what I have learned about the Hurd/Mach. + <braunr> are you trying to learn about virtio or the hurd ? + <rekado> both, because I'd like to write virtio drivers for the hurd. + <braunr> hm no + <rekado> with virtio drivers pass buffers to queues and then notify the + host. + <braunr> you may want it, but it's not what's best for the project + <rekado> oh. + <braunr> what's best is reusing existing drivers + <braunr> we're much too far from having enough manpower to maintain our own + <rekado> you mean porting the linux virtio drivers? + <braunr> there already is a virtio-net driver in linux 2.6 + <braunr> so yes, reuse it + <braunr> the only thing which might be worth it is a gnumach in-kernel + driver for virtio block devices + <braunr> because currently, we need our boot devices to be supported by the + kernel itself ... + <rekado> when I boot the hurd with qemu and the entropy device I see it as + an unknown PCI device in the output of lspci. + <braunr> that's just the lspci database which doesn't know it + <rekado> Well, does this mean that I could actually talk to the device + already? E.g., through libpciaccess? + <rekado> I'm asking because I don't understand how exactly devices "appear" + on the Hurd. + <braunr> it's one of the most difficult topic currently + <braunr> you probably can talk to the device, yes + <braunr> but there are issues with pci arbitration + * rekado takes notes: "pci arbitration" + <rekado> so, this is about coordinating bus access, right? + <braunr> yes + <braunr> i'm not a pci expert so i can't tell you much more + <rekado> heh, okay. + <rekado> what kind of "issues with pci arbitration" are you referring to, + though? + <rekado> Is this due to something that Mach isn't doing? + <braunr> ideally, mach doesn't know about pci + <braunr> the fact we still need in-kernel drivers for pci devices is a big + problem + <braunr> we may need something like a pci server in userspace + <braunr> on l4 system it's called an io server + <rekado> How do in-kernel drivers avoid these issues? + <braunr> they don't + <rekado> Or rather: why is it they don't have these issues? + <braunr> they do + <rekado> oh. + <braunr> we had it when youpi added the sata driver + <braunr> so currently, all drivers need to avoid sharing common interrupts + for example + <braunr> again, since i'm not an expert about pci, i don't know more about + the details + <Hooligan0> pci arbitrations are made by hardware ... no ? + <braunr> Hooligan0: i don't know + <braunr> i'm not merely talking about bus mastering here + <braunr> simply preventing drivers from mapping the same physical memory + should be enforced somewhere + <braunr> i'm not sure it is + <braunr> same for irq sharing + <Hooligan0> braunr : is the support for boot devices into the kernel is + really needed if a loader put servers into the memory before starting + mach ? + <braunr> Hooligan0: there is a chicken-and-egg problem during boot, + whatever the solution + <braunr> obviously, we can preload from memory, but then you really want + your root file system to use a disk + <braunr> Hooligan0: the problem with preloading from memory is that you + want the root file system to use a real device + <braunr> the same way / refers to one on unix + <braunr> so you have an actual, persistent hierarchy from which the system + can be initialized and translators started + <braunr> you also want to share as much as possible between the early + programs and the others + <braunr> so for example, both the disk driver and the root file system + should be able to use the same libc instance + <braunr> this requires a "switch root" mechanism that needs to be well + defined and robust + <braunr> otherwise we'd just build our drivers and root fs statically + <braunr> (which is currently done with rootfs actually) + <braunr> and this isn't something we're comfortable with + <braunr> so for now, in-kernel drivers + <Hooligan0> humm ... disk driver and libc ... i see + <Hooligan0> in other way ... disk drivers can use only a little number of + lib* functions ; so with a static version, a bit of memory is lots + <Hooligan0> s/lots/lost + <Hooligan0> and maybe the driver can be hot-replaced after boot (ok ok, + it's more simple to say than to write) diff --git a/open_issues/device_drivers_and_io_systems.mdwn b/open_issues/device_drivers_and_io_systems.mdwn index 5bda0213..085a737a 100644 --- a/open_issues/device_drivers_and_io_systems.mdwn +++ b/open_issues/device_drivers_and_io_systems.mdwn @@ -92,3 +92,9 @@ Also see [[user-space device drivers]]. * OSF Mach * Darwin + + * IRC, freenode, #hurd, 2013-08-26 + + < stargater> in haiku is a layer wraper for bsd driver + < stargater> + https://www.haiku-os.org/news/2007-05-08/haiku_getting_a_freebsd_network_driver_compatibility_layer diff --git a/open_issues/exec.mdwn b/open_issues/exec.mdwn index ff3fccf5..fe70123d 100644 --- a/open_issues/exec.mdwn +++ b/open_issues/exec.mdwn @@ -10,7 +10,10 @@ License|/fdl]]."]]"""]] [[!tag open_issue_hurd]] -IRC, unknown channel, unknown date. +[[!toc]] + + +# IRC, unknown channel, unknown date. <youpi> oh my, disabling gzip/bzip2 support makes apt preconfigure hang <youpi> support in exec* I meant @@ -18,6 +21,50 @@ IRC, unknown channel, unknown date. <youpi> now a funny bug: if I disable gzip/bzip2 support from exec <youpi> trying to run a zero-byte file hangs + +## IRC, freenode, #hurd, 2013-08-01 + + <teythoon> uh, all the non trivial exec server code has #ifdef'd BFD code + all over it and it looks like that isn't even used anymore + <teythoon> that's too bad actually, I figured out how to get the values + from BFD, not so for the other elf parser that is used instead + + +## IRC, freenode, #hurd, 2013-08-05 + + <teythoon> btw, there is a Debian bug concerning zipped executables. now + I'm not sure if I understood the problem, but gziped and bzip2ed + executables work for me + <teythoon> (not that I'm a big fan of that particular feature) + <youpi> iirc these somehow got fixed yes + <youpi> something like a previous out of bound access + <teythoon> the exec server contains lot's of code that is unused and + probably bit rot (#ifdef BFD) or otherwise ignored (#if 0) + <youpi> yes :/ + <teythoon> and there's gunzipping and bunzip2ing, which we probably don't + want anyway + <pinotree> why not? + <teythoon> we should strip all that from exec and start adding features + <teythoon> pinotree: b/c it's slow and the gain is questionable + <teythoon> it breaks mmapping the code in + <teythoon> exec/exec.c is huge (~2300 lines) and complex and it is an + essential server + <teythoon> and I wonder if the unzipping is done securely, e. g. if it's + not possible to crash exec with an maliciously compressed executable + + +## IRC, freenode, #hurd, 2013-09-12 + + <rekado> The zip code in hurd/exec/ looks really complicated; does it + really just unpack zipped files in memory (which could be replaced by + library calls) or is there something else going on? + <braunr> rekado: + http://lists.gnu.org/archive/html/bug-hurd/2013-08/msg00049.html + <rekado> braunr: interesting. Thanks. + <rekado> Does this mean that the "small hack entry" on the contributing + page to use libz and libbz2 in exec is no longer valid? + <braunr> probably + --- May want to have a look at using BFD / libiberty/simpleobject. diff --git a/open_issues/exec_leak.mdwn b/open_issues/exec_leak.mdwn deleted file mode 100644 index b58d2c81..00000000 --- a/open_issues/exec_leak.mdwn +++ /dev/null @@ -1,57 +0,0 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -[[!tag open_issue_hurd]] - - -# IRC, freenode, #hurd, 2012-08-11 - - <braunr> the exec servers seems to leak a lot - <braunr> server* - <braunr> exec now uses 109M on darnassus - <braunr> it really leaks a lot - <pinotree> only 109mb? few months ago, exec on exodar was taking more than - 200mb after few days of uptime with builds done - <braunr> i wonder how much it takes on the buildds - - -# IRC, freenode, #hurd, 2012-08-17 - - <braunr> the exec leak is tricky - <braunr> bddebian: btw, look at the TODO file in the hurd source code - <braunr> bddebian: there is a not from thomas bushnell about that - <braunr> "*** Handle dead name notifications on execserver ports. ! - <braunr> not sure it's still a todo item, but it might be worth checking - <bddebian> braunr: diskfs_execboot_class = ports_create_class (0, 0); - This is what would need to change right? It should call some cleanup - routine in the first argument? - <bddebian> Would be ideal if it could just use deadboot() from exec. - <braunr> bddebian: possible - <braunr> bddebian: hum execboot, i'm not so sure - <bddebian> Execboot is the exec task, no? - <braunr> i don't know what execboot is - <bddebian> It's from libdiskfs - <braunr> but "diskfs_execboot_class" looks like a class of ports used at - startup only - <braunr> ah - <braunr> then it's something run in the diskfs users ? - <bddebian> yes - <braunr> the leak is in exec - <braunr> if clients misbehave, it shouldn't affect that server - <bddebian> That's a different issue, this was about the TODO thing - <braunr> ah - <braunr> i don't know - <bddebian> Me either :) - <bddebian> For the leak I'm still focusing on do-bunzip2 but I am baffled - at my results.. - <braunr> ? - <bddebian> Where my counters are zero if I always increment on different - vars but wild freaking numbers if I increment on malloc and decrement on - free diff --git a/open_issues/exec_memory_leaks.mdwn b/open_issues/exec_memory_leaks.mdwn index d504c4f0..67281bdc 100644 --- a/open_issues/exec_memory_leaks.mdwn +++ b/open_issues/exec_memory_leaks.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -12,8 +12,56 @@ License|/fdl]]."]]"""]] There are is some memory leak in [[`exec`|hurd/translator/exec]]. +[[!toc]] -# I + +# IRC, freenode, #hurd, 2012-08-11 + + <braunr> the exec servers seems to leak a lot + <braunr> server* + <braunr> exec now uses 109M on darnassus + <braunr> it really leaks a lot + <pinotree> only 109mb? few months ago, exec on exodar was taking more than + 200mb after few days of uptime with builds done + <braunr> i wonder how much it takes on the buildds + + +## IRC, freenode, #hurd, 2012-08-17 + + <braunr> the exec leak is tricky + <braunr> bddebian: btw, look at the TODO file in the hurd source code + <braunr> bddebian: there is a not from thomas bushnell about that + <braunr> "*** Handle dead name notifications on execserver ports. ! + <braunr> not sure it's still a todo item, but it might be worth checking + <bddebian> braunr: diskfs_execboot_class = ports_create_class (0, 0); + This is what would need to change right? It should call some cleanup + routine in the first argument? + <bddebian> Would be ideal if it could just use deadboot() from exec. + <braunr> bddebian: possible + <braunr> bddebian: hum execboot, i'm not so sure + <bddebian> Execboot is the exec task, no? + <braunr> i don't know what execboot is + <bddebian> It's from libdiskfs + <braunr> but "diskfs_execboot_class" looks like a class of ports used at + startup only + <braunr> ah + <braunr> then it's something run in the diskfs users ? + <bddebian> yes + <braunr> the leak is in exec + <braunr> if clients misbehave, it shouldn't affect that server + <bddebian> That's a different issue, this was about the TODO thing + <braunr> ah + <braunr> i don't know + <bddebian> Me either :) + <bddebian> For the leak I'm still focusing on do-bunzip2 but I am baffled + at my results.. + <braunr> ? + <bddebian> Where my counters are zero if I always increment on different + vars but wild freaking numbers if I increment on malloc and decrement on + free + + +# 2012-11-25 After twelve hours worth of `fork/exec` ([[GCC]]'s `check-c` part of the testsuite), we got: @@ -29,7 +77,7 @@ quite noticeable. In comparison: 276 0 3 1 1 344 442M 28.2M 0.6 48:09.36 91min /hurd/ext2fs /dev/hd2s5 -# II +# 2012-12-20 After running the libtool testsuite for some time: diff --git a/open_issues/fakeroot_eagain.mdwn b/open_issues/fakeroot_eagain.mdwn index 6b684a04..168ddf7d 100644 --- a/open_issues/fakeroot_eagain.mdwn +++ b/open_issues/fakeroot_eagain.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -132,7 +132,7 @@ License|/fdl]]."]]"""]] <braunr> or rather, a lot more <braunr> (or maybe not, since it leaks only in some cases) -[[exec_leak]]. +[[exec_memory_leaks]]. <braunr> pinotree: actually, the behaviour under linux is the same with the alternative correctly set, whereas faked-tcp is restarted (if used at diff --git a/open_issues/gccgo.mdwn b/open_issues/gccgo.mdwn index 9e724b95..a3c0e1d1 100644 --- a/open_issues/gccgo.mdwn +++ b/open_issues/gccgo.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -38,6 +38,15 @@ been working on this, has some (unpublished) patches, and this was blocked on [[`getcontext`/`makecontext`/`setcontext`/`swapcontext`|glibc]]. +### IRC, freenode, #hurd, 2013-08-26 + + < gnu_srs> tschwinge: on + http://www.gnu.org/software/hurd/open_issues/gccgo.html you might change + the text, my patches are published + < gnu_srs> http://lists.gnu.org/archive/html/bug-hurd/2013-06/msg00050.html + to msg00052.html + + ## `getcontext`/`makecontext`/`setcontext`/`swapcontext` usage analysis In context of [[glibc/t/tls-threadvar]]. Looking at GCC trunk commit diff --git a/open_issues/gdb.mdwn b/open_issues/gdb.mdwn index 67a38e96..aec797ee 100644 --- a/open_issues/gdb.mdwn +++ b/open_issues/gdb.mdwn @@ -355,6 +355,18 @@ like `gdb/testsuite/boards/cc-with-tweaks.exp` would help, or setting Cannot access memory at address 0x6c62616e (gdb) testcase ../../../Ferry_Tagscherer/gdb/testsuite/gdb.base/attach-pie-noexec.exp completed in 3 seconds + IRC, freenode, #hurd, 2013-09-06: + + <gnu_srs1> How to debug a program that works in the shell but Cannot + access memory at address ... in gdb? + <tschwinge> Build it without -pie -- but that is just a guess of what + might be going on. + * tschwinge clearly has spent enough time with obscure things to be + able to make such guesses. + <gnu_srs1> tschwinge: looks like -fPIE is used. + <gnu_srs1> verified: some (all?) executables compiled with -fPIE, -fpie + and linked with -pie cannot be debugged in gdb :( + * `solib-event stop` Running ../../../Ferry_Tagscherer/gdb/testsuite/gdb.mi/mi-catch-load.exp ... @@ -540,3 +552,34 @@ like `gdb/testsuite/boards/cc-with-tweaks.exp` would help, or setting related. TODO. + + +# Open Issues + +## [[tag/open_issue_gdb]] + +## `info files` SIGSEGV + +[[!tag open_issue_gdb]] + + +### IRC, freenode, #hurd, 2013-09-07 + + <rekado> I'm trying to debug pfinet, but I'm not very familiar with gdb. + Tried to attach to the running pfinet process (built with debug symbols), + set a breakpoint and ... when I ran "info files" the process segfaulted. + <teythoon> which process segfaults, pfinet or gdb? + <rekado> gdb segfaults. + + +## Watchpoints + +[[!tag open_issue_gdb]] + + +### IRC, freenode, #hurd, 2013-09-16 + + <gnu_srs> tschwinge: Is gdb watch known to fail on hurd? It hangs for me + when logged in via ssh. + <tschwinge> gnu_srs: Don't know about GDB's watch command. Are you sure it + is hanging? diff --git a/open_issues/glibc.mdwn b/open_issues/glibc.mdwn index 31437744..b453b44f 100644 --- a/open_issues/glibc.mdwn +++ b/open_issues/glibc.mdwn @@ -281,14 +281,55 @@ Last reviewed up to the [[Git mirror's 0323d08657f111267efa47bd448fbf6cd76befe8 [[glibc_madvise_vs_static_linking]]. + IRC, OFTC, #debian-hurd, 2013-09-09: + + <gg0> does hurd MADV_DONTNEED or MADV_FREE or none? + http://sources.debian.net/src/jemalloc/3.4.0-1/include/jemalloc/jemalloc_defs.h.in#L239 + <gg0> seems it builds by defining JEMALLOC_PURGE_MADVISE_DONTNEED + but i don't know what i'm talking about, so it could build with + JEMALLOC_PURGE_MADVISE_FREE as well + + IRC, OFTC, #debian-hurd, 2013-09-10: + + <youpi> gg0: it implements none, even if it defines DONTNEED (but + not FREE) + + See also: + + gnash (0.8.11~git20130903-1) unstable; urgency=low + + * Git snapshot. + + Embedded jemalloc copy has been replaced by system one. + [...] + - Disable jemalloc on hurd and kfreebsd-*. No longer disabled upstream. + * `msync` Then define `_POSIX_MAPPED_FILES`, `_POSIX_SYNCHRONIZED_IO`. - * `sys/epoll.h` + * `epoll`, `sys/epoll.h` Used by [[wayland]], for example. + IRC, freenode, #hurd, 2013-08-08: + + <nalaginrut> is there any possible to have kquque/epoll alike + things in hurd? or there is one? + <braunr> nalaginrut: use select/poll + <nalaginrut> is it possible to implement epoll? + <braunr> it is + <braunr> we don't care enough about it to do it + <braunr> (for now) + <nalaginrut> well, since I wrote a server with Guile, and it could + take advantage of epoll, never mind, if there's no, it'll use + select automatically + <nalaginrut> but if someday someone care about it, I'll be + interested on it + <braunr> epoll is a scalability improvement over poll + <braunr> the hurd being full of scalability issues, this one is + clearly not a priority + <nalaginrut> ok + * `sys/eventfd.h` * `sys/inotify.h` @@ -390,6 +431,429 @@ Last reviewed up to the [[Git mirror's 0323d08657f111267efa47bd448fbf6cd76befe8 libgc, libsigsegv, luatex, mono, nspr, pth, ruby1.8, texlive-bin, uim, and more. + IRC, OFTC, #debian-hurd, 2013-09-08: + + <pinotree> oh, and even ruby2.0 suffers because of fixed-stack + threads + <youpi> yes, we definitely need to finish fixing it + <youpi> my current work is in our glibc repo, youpi/tls-threadvar + <pinotree> | *** makecontext: a stack at 0xbc000 with size 0x40000 + is not usable with threadvars + <pinotree> all 8 failing tests with that + <youpi> maybe we can hand-disable the use of contexts in ruby for + now? + <pinotree> gg0: ↑ :) + <gg0> after the pseudo-patch i RFCed, i don't deserve to say + anything else about that :) + <pinotree> i mean, feel free to investigate and "fix" ruby2.0 as + above :) + <gg0> eh maybe i'd just be able to hand-disable failing + thread-related _tests_ :) + <gg0> i'm still hoping some real developer picks and actually fixes + it, seems it's not enough interesting though + <azeem> 21:37 < youpi> yes, we definitely need to finish fixing it + <gg0> afaiu youpi is working on threadvars-tls migration, which + would mean fixing them all. i just meant fixing ruby, which would + mean having puppet btw + <youpi> gg0: "actually fixing" means fixing threadvars-tls + migration + <youpi> "just fixing" ruby can be done by simply disabling context + use in ruby + + IRC, OFTC, #debian-hurd, 2013-09-10: + + <gg0> this one fixes make test by disabling context and giving more + time to timing related tests http://paste.debian.net/plain/37977/ + <gg0> make test-all is another story + <youpi> gg0: AIUI, the sleep part should get fixed by the next + glibc upload, which will include the getclk patch + <youpi> but the disabling context part could be good to submit to + the debian ruby package, mentioning that this is a workaround for + now + <gg0> unfortunately still not enough, test-all still fails + <youpi> does it make the package not build? + <gg0> test-all is the second part of what we call tests + <gg0> they build and package (they produce all ruby packages), + after that they run debian/run-test-suites.bash which is make + test + make test-all + <gg0> well after or during the build doesn't matter, it's their + testsuite + <gg0> ok just failed: + <gg0> TestBug4409#test_bug4409 = Illegal instruction + <gg0> make: *** [yes-test-all] Error 132 + <gg0> what to do with Illegal instruction? + <gg0> just found 2 words that make everybody shut up :p + <pinotree> same as above: debug it + <teythoon> gg0: have you confirmed that this is reproducible? I've + once had a process die with SIGILL and it was not and I figured + it might have been a (qemu?) glitch + <gg0> seems i'm running tests which are disabled on _all_ archs, + better so + <gg0> well, this should be reproducible. i just got it on a qemu, i + could try to reproduce it on real hardware but as just said, i + was testing tests disabled by maintainer so completely useless + <teythoon> gg0: yeah, I'm running all my hurd instances on qemu/kvm + as well, I meant did you get this twice in a row? + <gg0> to be honest i got another illegal instruction months ago but + don't recall doing what + <gg0> nope not twice, i've commented it out. then run the remaining + and then found out i should not have done what i was doing + <gg0> but i could try to reproduce it + <gg0> ok now i recall i got it another one few hours ago on real + hardware, from logs: + <gg0> TestIO#test_copy_stream_socket = Illegal instruction + <gg0> teythoon: on real hardware though + <gg0> and this is the one i should debug once it finishes, still + running + + IRC, freenode, #hurd, 2013-09-11: + + <gg0_> ../sysdeps/mach/hurd/jmp-unwind.c:53: _longjmp_unwind: + Assertion `! __spin_lock_locked (&ss->critical_section_lock)' + failed. + <gg0_> and + <gg0_> ../libpthread/sysdeps/mach/pt-thread-halt.c:51: + __pthread_thread_halt: Unexpected error: (ipc/send) invalid + destination port. + <tschwinge> gg0_: Which libpthread source are these? Stock Debian + package? + <gg0_> tschwinge: everything debian, ruby rebuilt with + http://paste.debian.net/plain/38519/ which should disable + *context + + IRC, OFTC, #debian-hurd, 2013-09-11: + + <gg0_> wrt ruby, i'd propose a patch that disables *context and + comments out failed tests (a dozen). most of them are timing + related, don't always fail + <gg0_> if they failed gracefully, we could leave them enabled and + just ignoring testsuite result, but most of them block testsuite + run when fail + <gg0_> anyone against? any better idea (and intention to implement + it? :p)? + <gg0_> youpi: is disabling some tests acceptable? ^ + <youpi> it'd be good to at least know what is failing + <youpi> so as to know what impact hiding these failures will have + <youpi> remember that hiding bugs usually means getting bitten by + them even harder later :) + <gg0_> many of them use pipes + <gg0_> here the final list, see commented out ones + http://paste.debian.net/plain/38426 + <gg0_> and as said some don't always fails + <gg0_> test_copy_stream_socket uses a socket + <youpi> note that we can still at least build packages with notest + <youpi> at least to get the binaries uploaded + <youpi> disabling *context should however really be done + <youpi> and the pipe issues are concerning + <youpi> I don't remember other pipe issues + <youpi> so maybe it's a but in the ruby bindings + <gg0_> i just remember they didn't die, then something unknown + fixed it + <youpi> I see something frightening in io.c + <youpi> #if BSD_STDIO + <youpi> preserving_errno(fseeko(f, lseek(fileno(f), + (off_t)0, SEEK_CUR), SEEK_SET)); + <youpi> #endif + <youpi> this looks very much like a workaround for an odd thing in + BSD + <youpi> it happens that that gets enabled on hurd too, since + __MACH__ is defined + <youpi> you could try to drop these three lines, just to see + <youpi> this is very probably very worth investigating, at any rate + <youpi> even just test_gets_limit_extra_arg is a very simple test, + that I fail to see why it should ever fail on hurd-i386 + <youpi> starting debugging it would be a matter of putting printfs + in io.c, to check what gets called, with what parameters, etc. + <youpi> just a matter of taking the time to do it, it's not very + complex + <gg0_> youpi: are you looking at 1.8? no BSD_STDIO here + <youpi> yes, 1.8 + <gg0_> 1.9.3.448 + <gg0_> landed to sid few days ago + <youpi> ah, I have 1.87 + <youpi> +. + <gg0_> my favourites are TestIO#test_copy_stream_socket and + TestIO#test_cross_thread_close_fd -> Illegal instruction + <gg0_> TestIO#test_io_select_with_many_files sometimes Illegal + instruction, sometimes ruby1.9.1: + ../sysdeps/mach/hurd/jmp-unwind.c:53: _longjmp_unwind: Assertion + `! __spin_lock_locked (&ss->critical_section_lock)' failed. + + [[thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock]]? + + <gg0_> trying to debug illegal instruction + http://paste.debian.net/plain/38585/ + <gg0_> (yes, i'm not even good at gdbing) + <gg0_> any hint? + <gg0_> oh found out there's an intree .gdbinit, that might + complicate things + + IRC, OFTC, #debian-hurd, 2013-09-13: + + <gg0_> where should it be implemented MAP_STACK? plus, is it worth + doing it considering migration to tls, wouldn't it be useless? + <gg0_> sysdeps/mach/hurd/mmap.c i should reduce stupid questions + frequency from daily to weekly basis + + IRC, OFTC, #debian-hurd, 2013-09-14: + + <gg0_> say i managed to mmap 0x200000-aligned memory + <gg0_> now i get almost the same failed tests i get disabling + *context + <gg0_> that would mean they don't depend on threading + + IRC, freenode, #hurd, 2013-09-16: + + <gg0> i get many ../sysdeps/mach/hurd/jmp-unwind.c:53: + _longjmp_unwind: Assertion `! __spin_lock_locked + (&ss->critical_section_lock)' failed. + <gg0> by running ruby testsuite, especially during test_read* tests + http://sources.debian.net/src/ruby1.9.1/1.9.3.448-1/test/ruby/test_io.rb#L972 + <gg0> read/write operations with pipes + <braunr> gg0: that's weird + <braunr> gg0: debian glibc ? + <gg0> braunr: yep, debian 2.17-92 + <gg0> sometimes assertion above, sometimes tests in question get + stuck reading + <gg0> it would be nice reproducing it w/o ruby + <gg0> probably massive io on pipes could do the job + <gg0> also more nice finding someone who finds it interesting to + fix :p + <gg0> ruby is rebuilt with http://paste.debian.net/plain/40755/, no + *context + <gg0> pipe function in tests above creates one thread for write, + one for read + http://sources.debian.net/src/ruby1.9.1/1.9.3.448-1/test/ruby/test_io.rb#L26 + <tschwinge> gg0: About the jmp-unwind assertion failure: is it be + chance this issue: + <http://www.gnu.org/software/hurd/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.html>? + I didn't look in detail. + <braunr> tschwinge: that's what i thought too about the assertion, + which is why i find it strange + <gg0> asserting it's not locked then locking it doesn't exclude + race conditions + + IRC, OFTC, #debian-hurd, 2013-09-17: + + <gg0> youpi: i guess no one saw it anymore since + tg-thread-cancel.diff patch + <gg0> it = + http://www.gnu.org/software/hurd/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.html + <gg0> this one comes from sysdeps/mach/hurd/jmp-unwind.c:53 though + <gg0> another assertion to remove? + <youpi> gg0: it's not exactly the same: in hurd_thread_cancel we + hold no lock at all at the assertion point + <youpi> in jmp-unwind.c, we do hold a lock + <youpi> and the assertion might be actually true because all other + threads are supposed to hold the first lock before taking the + other one + <youpi> you could check for that in other places + <youpi> and maybe it's the other place which wouldhave to be fixed + <youpi> also look for documentation which would say that + + IRC, freenode, #hurd, 2013-09-17: + + <braunr_> gg0: is that what we do ?? + <gg0> braunr: well, i was looking at + http://sources.debian.net/src/eglibc/2.17-92/debian/patches/hurd-i386/tg-thread-cancel.diff + <gg0> which afaics fixes + http://www.gnu.org/software/hurd/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.html + <gg0> the one i get now is + http://sources.debian.net/src/eglibc/2.17-92/sysdeps/mach/hurd/jmp-unwind.c#L53 + <gg0> 09:12 < youpi> gg0: it's not exactly the same: in + hurd_thread_cancel we hold no lock at all at the assertion point + <gg0> 09:13 < youpi> in jmp-unwind.c, we do hold a lock + <gg0> 09:13 < youpi> and the assertion might be actually true + because all other threads are supposed to hold the first lock + before taking the other one + <braunr> gg0: that assertion is normal + <braunr> it says there is a deadlock + <braunr> ss->critical_section_lock must be taken before ss->lock + <gg0> you mean ss->lock before ss->critical_section_lock + <braunr> no + <gg0> ah ok got it + <braunr> that's a bug + <braunr> longjmp + <braunr> ugh + <braunr> you could make a pass through the various uses of those + locks and check what the intended locking protocol should be + <braunr> i inferred ss->critical_section_lock before ss->lock from + hurd_thread_cancel + <braunr> this might be wrong too but considering this function is + used a lot, i doubt it + <gg0> (no, i hadn't got it, i was looking at jmp-unwind.c where + lock is before critical_section_lock) + <gg0> could we get useful info from gdb'ing the assertion? + <tschwinge> gg0: Only if you first get an understanding why it is + happening, what you expect to happen instead/why it shall not + happen/etc. Then you can perhaps use GDB to verify that. + <gg0> i can offer an irc interface if anyone is interested, it's + ready, just to attach :) + <gg0> this is the test + http://sources.debian.net/src/ruby1.9.1/1.9.3.448-1/test/ruby/test_io.rb#L937 + <gg0> pipe function creates two threads + http://sources.debian.net/src/ruby1.9.1/1.9.3.448-1/test/ruby/test_io.rb#L26 + <gg0> Attaching to pid 15552 + <gg0> [New Thread 15552.1] + <gg0> [New Thread 15552.2] + <gg0> (gdb) + + IRC, freenode, #hurd, 2013-09-21: + + <youpi> gg0: it seems the assert (! __spin_lock_locked + (&ss->critical_section_lock)); is bogus + <youpi> but it'd be good to catch a call trace + <youpi> well, it may not be bogus, in case that lock is only ever + taken by the thread itself + <youpi> in that case, inside longjmp_unwind we're not supposed to + have it already + <youpi> ok, that's what we had tried to discuss with Roland + <youpi> it can happen when playing with thread cancelation + <braunr> youpi: the assertion isn't exactly bogus + <braunr> the lock ordering is + <youpi> braunr: which one are you talking about? + <youpi> the one in hurd_thread_cancel looks really wrong + <youpi> and some parts of the code keep the critical section lock + without ss->lock held, so I don't see how lock ordering can help + + IRC, OFTC, #debian-hurd, 2013-09-22: + + <gg0> how much does this patch suck on a scale from 1 to 10? + http://paste.debian.net/plain/44810/ + <youpi> well, the stack allocation issue will go away once I get + the threadvars away + <youpi> I'm working on it right now + <youpi> about the lib paths, it makes sense to add the gnu case, + but i386-gnu shouldn't be put in the path + <gg0> that's great + <gg0> so seems the wrong moment for what i've already done + ie. asking terceiro what he thinks about patch above :/ + <gg0> any distro-independent way to get libc.so and libm.so path? + <gg0> ruby as last resource takes them from "ldd ruby" + <pinotree> gg0: should work fine then + <gg0> well it does. but gnu doesn't have a case so it hits default + which is broken + http://bugs.ruby-lang.org/projects/ruby-trunk/repository/revisions/40235/entry/test/dl/test_base.rb + <gg0> btw even linux and kfreebsd with debian multipath have broken + cases but they don't hit default and get fixed by ldd later + <pinotree> why it is broken? are arguments passed to that script? + <gg0> i'm not sure about what propose. a broken case so it doesn't + hit default like linux and kfbsd + <gg0> yes they are :/ + <pinotree> and which ones are? who executes that script and which + arguments does it pass to it? + <gg0> other ruby scripts which have nothing to do with libc/libm + <pinotree> well, if they pass arguments which should be the paths + to libc and libm, they must be getting such paths, aren't they? + <gg0> they don't. arguments are other ruby scripts, don't know why, + maybe something else broken before + <gg0> but that would mean that before there's a smarter path + detection way, i doubt + <pinotree> then add the case for hurd, but setting both libc and + libm as nil + <pinotree> so they will be fetched again + <gg0> yep and would really ugly + <gg0> +be + <gg0> "please commit this one which wrongly sets paths." + <gg0> an alternative would be removing default case + <gg0> or pointing it out by proposing ldd in hurd case might make + them review the whole detection + <gg0> by setting correct paths like in patch above it wouldn't + break a possible hurd-amd64, it would work due to ldd + <youpi> gg0: that's why I said the patch is fine, but without the + i386-gnu part of the path + <youpi> just like it happens to be on linux & kfreebsd + <gg0> i might take ldconfig -p output + <gg0> to make it uselessly correct from start + <gg0> http://bugs.ruby-lang.org/issues/8937 + <pinotree> note thar ruby 1.8 is EOL + <pinotree> *that + <gg0> -- If you're reporting a bug in both Ruby 1.9/2.0 and Ruby + 1.8: ruby-trunk, and write like "this bug can be reproduced in + Ruby 1.8 as well." -- + <gg0> i suspect this one won't be the only one i'll file. unless + upcoming youpi's tls and braunr's thread destruction patches fix + all ruby tests + <pinotree> did you check ruby2.0 too, btw? + <gg0> switched to ruby2 few hours ago. i pointed out 2nd part of + testsuite is not enabled, probably terceiro will enable it soon + <gg0> by applying my patch above we'd completely fix current + ruby2.0 build (yes because tests are not completely enabled) + <pinotree> what you run those extra tests? + <gg0> + http://anonscm.debian.org/gitweb/?p=collab-maint/ruby1.9.1.git;a=blob;f=debian/run-test-suites.bash + <gg0> make test + make test-all + <gg0> (test-all is 2nd part) + <gg0> many are problematic. i didn't finish yet to suppress them + one-by-one. one i suppress, another one pops up + <gg0> either get stuck or well known assertion + <pinotree> check those that get stuck :) + <gg0> which kind of check? + <pinotree> "check" as in "debug" + <gg0> btw i tested puppet few days ago (with ruby1.8), it seems to + be working, at least at trasferring files from master + <gg0> don't know about any advanced usage + <pinotree> ruby 1.8 is going to die soon, so testing things against + it is not totally useful + <gg0> so you assume 1.8 is less broken than 1.9/2.0, right? + <pinotree> no + <gg0> i just can see it's been built without tests itself too + <pinotree> erm no + <gg0> well ok, if i can be wrong, i'll be wrong + <gg0> i say that after a quick check time ago, might be wrong + <pinotree> `getbuildlogs ruby1.8 last hurd-i386`, see the last + build log + <gg0> ah from pkg-kde-tools + <gg0> i hate kde :) + <pinotree> no? + <gg0> no what? + <pinotree> devscripts: /usr/bin/getbuildlog + <gg0> pkg-kde-tools: /usr/bin/pkgkde-getbuildlogs + <pinotree> which is not what i said + <gg0> wait that's what apt-file found + <gg0> maybe i should update it + <gg0> is it so recent? + <pinotree> no + <pinotree> i just added an 's' more at the end of the command, but + typing getbu<tab> could have been helpful anyway... + <gg0> yeah just got it + <gg0> my fault not to have tried to run it before looking for it + <pinotree> and btw, i don't see what hating kde has to do with + tools developed by qt/kde debian packagers + <gg0> j/k i simply don't use kde, never used and apt-file search + told me it was from pkg-kde-tools + <gg0> btw build log says "make test" fails, doesn't even start. and + its failure doesn't block the build + <pinotree> exactly + <gg0> s/make test/make test-all/ + <gg0> "make test" (aka "1st part" above) doesn't run. i guess it's + missing in packaging + + IRC, freenode, #hurd, 2013-09-22: + + <braunr> youpi: i mean the lock order where the assertion occurs is + reserved compared to the one in hurd_thread_cancel + <braunr> (and the one in hurd_thread_cancel is the same used in + hurd condition-related functions) + <youpi> "reserved" ? + <braunr> reversed + <braunr> :) + <youpi> by "the assertion occurs", you mean gg0's spot? + <braunr> yes + <youpi> well , the assertion also happens in hurd_thread_cancel + <braunr> it does oO + <braunr> i didn't see that + <youpi> but otherwise yes, it's completely bogus that we have both + locking in different orders + <youpi> could you submit the fix for jmp-unwind.c to upstream? + <braunr> what fix ? + <youpi> reversing the lock order + <braunr> ah, simply that + <youpi> (well, provided that hurd_thread_cancel is right) + <braunr> that's what i suggested to gg0 + <braunr> to check where those locks are held and determine the + right order + * `recvmmsg`/`sendmmsg` (`t/sendmmsg`) From [[!message-id "20120625233206.C000A2C06F@topped-with-meat.com"]], @@ -401,13 +865,140 @@ Last reviewed up to the [[Git mirror's 0323d08657f111267efa47bd448fbf6cd76befe8 Then perhaps the Linux fallback case should be that instead of stubs, too.* + * `SOCK_CLOEXEC` + + IRC, freenode, #hurd, 2013-09-02: + + <gnu_srs1> Do we support accept4 with the SOCK_CLOEXEC flag? + <gnu_srs1> According to the code in sysdeps/mach/hurd/accept4.c + that case is not covered + <gnu_srs1> (only O_NONBLOCK, not SOCK_NONBLOCK??)) + <pinotree> gnu_srs1: we do + <pinotree> but only for accept4, not for socket and socketpair + <gnu_srs1> pinotree: cannot find the case for O_CLOEXEC covered in + __libc_accept4() + <pinotree> gnu_srs1: no, you need SOCK_* + <gnu_srs1> The only code for accept4() is in sysdeps/mach/hurd/ and + it uses O_* for flags ? + <pinotree> flags = sock_to_o_flags (flags); + <pinotree> tried checking it? + <gnu_srs1> Aha, tks:-D + <pinotree> and you don't need an explicit case of O_CLOEXEC, since + it is handled in other ways + + [[!message-id "1378154151.21738.15.camel@G3620.my.own.domain"]]. + + IRC, freenode, #hurd, 2013-09-03: + + <gnu_srs> any ideas about the SOCK_CLOEXEC issue? + <pinotree> didn't i tell already about it? + <gnu_srs> I did not find any hurd related code in tschwinges + branches. + <pinotree> you didn't check deep then... + <gnu_srs> so why does socket/socketpair not return ENOSYS then? + <pinotree> why should it, since they are implemented? + <braunr> ... + <gnu_srs> for socket/socketpair? + <braunr> gnu_srs: enosys means no system call + <gnu_srs> s/ENOSYS/EINVAL/ + <gnu_srs> see the mail to the bug-hurd/debian-hurd ML for more info + <gnu_srs> and tschwinges reply + <pinotree> which is what i knew already? + <gnu_srs> pinotree: please reply on the mailing list on the EINVAL + vs EPROTOTYPE issue to clarify things + <pinotree> gnu_srs: + https://sourceware.org/ml/libc-alpha/2013-02/msg00092.html + <pinotree> gnu_srs: things were clear already... + <gnu_srs> pinotree: I've read that patch and still pflocal/pf.c + returns EPROTOTYPE not changed by the __socket wrapper in eglibc + <pinotree> gnu_srs: what about realizing SOCK_CLOEXEC and friends + are NOT posix? + <gnu_srs> since socket/socketpair does not return EINVAL the dbus + code has to be patched then? + <pinotree> pflocal should never ever get such flags mixed to the + protocol, so any invalid value of protocol correctly returns + EPROTOTYPE + <gnu_srs> this is the question I need answered: Which way to go? + <pinotree> all of them + <gnu_srs> ? + <pinotree> - applications should not assume that because you have + accept4 (which is not posix) then SOCK_CLOEXEC and SOCK_NONBLOCK + (flags for it) are usable to socket and socketpair + <pinotree> - glibc should (as the idea of my patch) handle + implementations providing SOCK_* but not supporting them for + socket/socketpair + <pinotree> - finally the hurd part of glibc could implement them + <gnu_srs> to conclude: should I send a bug report for dbus then? + <gnu_srs> pinotree: yes or no? + <pinotree> gnu_srs: *shrug* i wrote it above, so an *upstream* + report (not a debian one) + + IRC, freenode, #hurd, 2013-09-06: + + <gnu_srs> I've found another error code issue, now in glib2.0 (and + dbus). Are you really sure the error code + <gnu_srs> for protocol of pflocal/pf.c should be + EPROTONOSUPPORT. The code expects EINVAL for a protocol with + <gnu_srs> SOCK_CLOEXEC, which is a flag. Maybe pf.c should add + this case and return EINVAL instead of + <gnu_srs> submitting bug reports upstream. Yes, I know this is not + POSIX, but it is defined for Hurd too, + <gnu_srs> currently only supported for accept4, not socket or + socketpair. + <pinotree> gnu_srs: no, and i explained already why it is wrong + this way + <pinotree> pflocal shouldn't even get such flags + <pinotree> (pflocal or any other server implementing socket_create) + <gnu_srs> (20:19:35) pinotree: pflocal shouldn't even get such + flags + <gnu_srs> then the glibc wrapper code is missing to catch this + flag:( + <gnu_srs> youpi: ? + <pinotree> gnu_srs: because, as told many times, socket and + socketpair do not support such flags + <pinotree> given they don't do, they filter nothing + <pinotree> and no, you need to file bugs upstream, since having + SOCK_* and accept4 does not imply at all that socket and + socketpair support them + + IRC, freenode, #hurd, 2013-09-07: + + <gnu_srs> A correction from yesterdays discussion: + s/EPROTONOSUPPORT/EPROTOTYPE + + IRC, freenode, #hurd, 2013-09-10: + + <gnu_srs> for dbus2.0 I found out that the third SOCK_CLOEXEC case + needs a patch too (more working tests), + <gnu_srs> the updated patch is at http://paste.debian.net/37948/ if + you have the time, otherwise I'll do it. + <pinotree> gnu_srs: which is what i wrote in my bug report... + <gnu_srs> Yes you wrote that, but the patch is not updated yet? + <pinotree> it refers to a different socket access, recently added, + which is not used by default + <gnu_srs> I got two more tests running when adding that patch:-/ + <pinotree> tests of what? + <gnu_srs> run-test.sh and run-test-systemserver.sh:P + <pinotree> tests of what? + <pinotree> i don't have the universal knowledge of the files in all + the sources + <gnu_srs> dbus-1.6.14/test/name-test/* + + [[!message-id "523A3D6C.2030200@gmx.de"]]. + + IRC, OFTC, #debian-hurd, 2013-09-19: + + <pinotree> tschwinge: ehm, regarding the SOCK_* patches for + socket/socketpair, didn't we talk about them when i worked on + eglibc 2.17? + For specific packages: * [[octave]] * Create `t/cleanup_kernel-features.h`. - * Add tests from Linux kernel commit messages for `t/dup3` et al. + * [[Secure_file_descriptor_handling]]. * In `sysdeps/unix/sysv/linux/Makefile`, there are a bunch of `-DHAVE_SENDFILE` -- but we do have `sendfile`, too. @@ -927,6 +1518,31 @@ Last reviewed up to the [[Git mirror's 0323d08657f111267efa47bd448fbf6cd76befe8 <tschwinge> Yeah, that's known for years... :-D <tschwinge> Probably not too difficult to resolve, though. + * IRC, OFTC, #debian-hurd, 2013-08-16: + + <pinotree> http://paste.debian.net/25934/ ← _hurd_thread_sigstate calls + malloc, boom + + * `conformtest` + + IRC, OFTC, #debian-hurd, 2013-09-22: + + <youpi> btw, I noticed that glibc has a head conformance test which we + happily fail quite a bit :) + <youpi> it's not so awful, we don't have twice as many failures as + linux, but not so far + <pinotree> youpi: do you mean "header" for "head", right? + <youpi> err, where ? :) + <pinotree> <youpi> btw, I noticed that glibc has a head conformance + test which we happily fail quite a bit :) + <youpi> ah, yes + <pinotree> noticed that too + <youpi> I had a quick look at the POSIX part, some things are probably + not too hard to change (e.g. exposing pthread_kill in signal.h) + <youpi> others will by quite hard to fix (short type instead of int + type for some flock structure field) + <youpi> s/by/be/ + * Verify baseline changes, if we need any follow-up changes: * a11ec63713ea3903c482dc907a108be404191a02 @@ -1253,6 +1869,13 @@ TODO. [[!message-id "20120723195143.7F8142C0B9@topped-with-meat.com"]]. + IRC, freenode, #hurd, 2013-08-27: + + < gnu_srs> Hi, is this fixed by now? + < gnu_srs> ../hurd/hurd.h:72:5: warning: case value ‘0’ not in + enumerated type ‘error_t’ [-Wswitch] + < pinotree> nope + * baseline fd5bdc0924e0cfd1688b632068c1b26f3b0c88da..2ba92745c36eb3c3f3af0ce1b0aebd255c63a13b (or probably Samuel's mmap backport) introduces: diff --git a/open_issues/glibc/debian.mdwn b/open_issues/glibc/debian.mdwn index 9886ec98..2ef2c474 100644 --- a/open_issues/glibc/debian.mdwn +++ b/open_issues/glibc/debian.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -24,6 +24,43 @@ locale stuff. `--disable-compatible-utmp`? +## IRC, freenode, #hurd, 2013-08-28 + + <youpi> uh, the i686 profiles have much more progression than i386 + <youpi> it seems they don't actually run these + <pinotree> youpi: what do you mean with "we don't run those"? + <pinotree> iirc there are three build profiles done, but there are 4 + regression test files + <youpi> yes, but some failing tests are not run in the three build profiles + <youpi> even if they are built for all of them + <pinotree> not even run? which ones? + <youpi> see for instance test-ifloat.out + <youpi> test-ifloat is built in all profiles, but only run in the libc one + <pinotree> don't have a glibc built tree around atm, sorry :/ + <youpi> perhaps because glibc thinks it's not useful to run it again if it + fails on i386 + <youpi> you can check the logs + <pinotree> do you think glibc's build system is that smart? :) + <pinotree> all the builds are done in separate builddirs, so theorically + they should not touch each other... + <youpi> yes + <youpi> that's why I'm surprised + <pinotree> could it be they get not run in optimized/particular builds? + <pinotree> what about linux/kfreebsd i386? + <youpi> I don't see what makes them not run + <youpi> or at least be treated particularly by th eMakefile + <youpi> not run on kfreebsd either + <youpi> pinotree: also, most of the tests now working have been marked as + failing by your patches for 2.17, would it be possible to retry them on + the box you used at that time? + <pinotree> that's the vm on my machine + <youpi> which kind of vm? + <youpi> kvm? + <pinotree> y + <youpi> they are working here + <youpi> with kvm + + # Building Run `debian/rules patch` to apply patches (instead of having it done during the diff --git a/open_issues/glibc/t/tls-threadvar.mdwn b/open_issues/glibc/t/tls-threadvar.mdwn index 609d866b..7ce36f41 100644 --- a/open_issues/glibc/t/tls-threadvar.mdwn +++ b/open_issues/glibc/t/tls-threadvar.mdwn @@ -76,3 +76,43 @@ dropped altogether, and `__thread` directly be used in glibc. <youpi> tschwinge: yes, there were a lot other occurences of threadvars stuff in various places <youpi> I'm building libc again, and will see what issue would remain + + +## IRC, freenode, #hurd, 2013-07-12 + + <youpi> braunr: about the per-thread ports, there is also the mig reply + port + <youpi> (stored in _HURD_THREADVAR_MIG_REPLY) + + +## IRC, freenode, #hurd, 2013-07-15 + + <braunr> and with the branch youpi pushed where he removes threadvars, it + shouldn't get "too" hard + <braunr> (save for the tricky bugs you may encounter) + <youpi> well, that branch is not working yet + + +## IRC, OFTC, #debian-hurd, 2013-09-22 + + <youpi> I'm currently tracking bugs with my threadvars changes + <youpi> some of them seem fine, others, not + <youpi> of course the most complex ones are the most probable culprits for + the issues I'm getting + <youpi> fortunately they're after the process bootstrap + <youpi> so basically that works + <youpi> just a few dozen tests fail + <youpi> mostly about loading .so or raising signals + <youpi> dlopen("bug-dlsym1-lib1.so"): bug-dlsym1-lib1.so: cannot open + shared object file: Function not implemented + <youpi> after having changed errno a bit + <youpi> doesn't that look odd ? :) + <youpi> good, I found an issue with the sigstate + <youpi> now running testsuite again, to see whether there are other issues + with it :) + <youpi> s/sigstate/reply_port/ actually + + +## IRC, OFTC, #debian-hurd, 2013-09-23 + + <youpi> yay, errno threadvar conversion success diff --git a/open_issues/hurd_init.mdwn b/open_issues/hurd_init.mdwn new file mode 100644 index 00000000..b0b58a70 --- /dev/null +++ b/open_issues/hurd_init.mdwn @@ -0,0 +1,216 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + + +# [[!message-id "20130625154749.17799.36923@thinkbox.jade-hamburg.de"]] + + +## IRC, freenode, #hurd, 2013-07-22 + + <teythoon> ok, so back to the drawing board for the next big issue, the + potential proc and init merge + <teythoon> Roland had some harsh words for that proposal, but noone else + raised concerns + <youpi> noone else does not mean much + <youpi> I guess only Roland actually understands the matter + <youpi> so I'd tend to believe him + <teythoon> even though, his criticism was so superficial, he could at least + be a bit more specific... + <braunr> i agree that the argument, being simply based on vague principle, + isn't very convincing + <teythoon> so, what should I do? + <braunr> you can either keep them separate, or fight with roland + <teythoon> common braunr, I need a little more guidance in these kind of + social issues + <teythoon> a statement like this is of little use ;) + <braunr> that's the best i can give you + <teythoon> :/ + <braunr> i have one patch "fixing" HZ on the hurd, and i even get to fight + about it + <teythoon> I understand Roland has been around forever and keeps an eye on + stuff + <teythoon> but could/would he block a patch for hurd if e.g. youpi would + accept it + <teythoon> i.e. how much control has he in practice? + <teythoon> me fighting with him over a patch is of little value for anyone + and I don't care to do so + <braunr> not much i suppose now + <braunr> but we also have to agree with the change + <braunr> with *real* arguments + <braunr> (well, if it was up to me, i'd even merge exec with proc so ..) + <teythoon> ok, so I whip up a patch to see how it goes in practice and + present it so we could talk about the issue with something to look at + first + <braunr> although maybe not ;p + <braunr> you'll hit the same reaction + <teythoon> from Roland? + <braunr> yes + <braunr> and youpi said he tends to trust what roland says + <braunr> so let's discuss the pros and cons a bit more + <teythoon> yes, but I'd honor his concerns if they were properly + presented. just telling me to hack on linux instead even though I think I + have demonstrated that I do want to work on Hurd is so childish in my + eyes that I do not consider that a valid argument at the moment + <teythoon> sure, shoot + <braunr> well, functionally, they're unrelated + <teythoon> head -n1 init/init.c + <teythoon> /* Start and maintain hurd core servers and system run state + <youpi> and thus it makes sense to make them separate, even if it does not + seem to bring anything useful now + <youpi> history has shown that it makes a bed for nice things later + <braunr> teythoon: that's not what proc is about + <teythoon> braunr: I know + <teythoon> braunr: that's what init is about in its own words ;) + <youpi> teythoon: also, "simplifying the code" is not necessarily an + argument that would be considered + <youpi> depending on the simplification + <youpi> linux made it all simple by using a monolithic kernel :) + <youpi> separating concerns is complex + <youpi> but in the end it usually pays off on the Hurd + <youpi> personally, I'd be fine with Guillem's solution, and renumbering + init's pid in Debian + <youpi> there's a pending question from Roland actually: what information + is exchanged between init and proc in the end? + <youpi> that's actually the point of the discussion: is that information + really big or not + <teythoon> I'm sorry, you lost me, where did he ask that question? + <pinotree> $ git grep proc_getmsgport | egrep '[0-9]' ← /hurd/init as pid 1 + is hardcoded in few places + <youpi> teythoon: he didn't ask it this way, but that's the question I had + to be able to answer his + <youpi> Date: Mon, 15 Jul 2013 10:36:35 -0700 (PDT) + <youpi> > That's not what he said. He said there is a lot of information + <youpi> > propagated from init to proc, and thus the separation is + questionable. + <youpi> Are you talking about bootstrap, or what? + <youpi> as I haven't investigated much, I couldn't answer this + <youpi> pinotree: right. We could patch these in Debian + <teythoon> youpi: so, shall I refresh, test and refine Guillems patch and + resend it? + <youpi> it's probably an easier way + <teythoon> ok, I start by doing that + + +## IRC, freenode, #hurd, 2013-07-25 + + <teythoon> pinotree: btw, there are two /sbin/init processes even with my + hacked up init/proc variant where /sbin/init gets to be pid 1 + <pinotree> never seen that + <pinotree> what are their parents? + <teythoon> pinotree: well, pid 1 is /sbin/init now, pid 13 or something has + the parent 1 + <teythoon> looks like init forks or something + <pinotree> i guess your sysvinit is compiled without INITDEBUG? + <pinotree> nothing in syslog either? + <teythoon> pinotree: it's compiled like the sysvinit shipped with debian + <pinotree> teythoon: do you have custom additions in inittab? + <teythoon> pinotree: a terminal for my serial console + <teythoon> *getty + <pinotree> are the getty started correctly for you, btw? + <teythoon> pinotree: yes + <pinotree> interesting + <pinotree> teythoon: back then, they were costantly respawning, with hurd's + getty's failing to start when exec'ed by (sysv)init + <pinotree> wonder what changed + <teythoon> pinotree: cool, magically went away then :) + + +## IRC, freenode, #hurd, 2013-07-29 + + <teythoon> youpi: I need some feedback on the not freezing translators + issue, more specifically whether I understood you correctly in your mail + from wednesday (20130724131552.GG9576@type.bordeaux.inria.fr) + <teythoon> oh yeah, and I had some questions yesterday too, about rpctrace + and dead-name notifications, specifically why /hurd/init is not receiving + any for the root translator and the exec server + <braunr> teythoon: more details please + <teythoon> ok, so /hurd/init is registering for dead name notifications for + essential tasks + <teythoon> the rootfs and exec both register as essential tasks at init and + init requests successfully dead name notifications for them + <teythoon> if you e.g. kill the auth server, /hurd/init will notice and + crash the system + <teythoon> if you kill exec or the rootfs, /hurd/init does not get notified + <teythoon> I verified this with gdb and an subhurd + <teythoon> I'm puzzled by this, as the kernel is the one who sends the + notifications, right? + <braunr> yes + <braunr> teythoon: where is the problem ? + <teythoon> and it is not that the system is not sending any messages, it + is, I see the msgcount increase over time + <teythoon> braunr: dunno, as far as I can tell the kernel does not deliver + the notification for rootfs and exec + <braunr> oh + <teythoon> those are the two processes loaded by grub, maybe they are + different somehow + <braunr> is that affecting your work ? + <teythoon> no, not directly, I strayed around at the weekend, trying to + think of cool stuff hurd could do + <teythoon> youpi: I need some feedback on the not freezing translators + issue, more specifically whether I understood you correctly in your mail + from wednesday (20130724131552.GG9576@type.bordeaux.inria.fr) + <youpi> teythoon: ok, now I'm available for the not-freezing-translators + thing :) + + +## IRC, freenode, #hurd, 2013-08-05 + + <teythoon> youpi: I'm in the process of producing a unified + sysvinit-as-pid1 and please-dont-kill-important-processes patch series + <teythoon> youpi: there is one issue with changing /hurd/inits pid, libcs + reboot() also assumes that it has the pid 1 + <youpi> argl + <youpi> that's bad, because it's then an ABI, not just an internal thing + <teythoon> hardcoding the pid is the worst way of getting a handle of any + server :/ + <teythoon> I've been thinking to make it explicit by binding it to + /servers/startup or something + <youpi> that would be more hurdish than using a pid, yes + <teythoon> yes, and not only does it break the abi, but in a bad way + too. if the libc is updated before the hurd, the shutdown sequence is + broken in a way that the translators aren't synced :/ + <teythoon> youpi: as a workaround, we could make reboot() signal both pid 1 + and 2 + <youpi> at worse pid 1 shouldn't get harmed by receiving a startup_reboot + RPC indeed + <teythoon> yes + + +## IRC, freenode, #hurd, 2013-08-16 + + <teythoon> grml, the procfs hardcodes the kernels pid :/ + <teythoon> there's always one more thing to fix... + <teythoon> uh, and we made pids.h a private header, so no nice constant for + the procfs translator :/ + <teythoon> server lookup by hardcoding the pid should be banned... + + +## IRC, freenode, #hurd, 2013-09-16 + + <teythoon> youpi: I'm thinking about splitting /hurd/init into /hurd/init + and /hurd/startup + <teythoon> that way, you could also merge the init as pid1 patches + <teythoon> that should be doable within the week + <youpi> that would probably be better received by Roland than merging init + into proc :) + <teythoon> yes, I suppose so :D + <youpi> perhaps you should start the discussion on the list about it + already, with just a sketch of which would do what + <teythoon> ok + <teythoon> fwiw I like the name startup b/c it speaks the startup protocol + <braunr> teythoon: +1 startup + + +## IRC, freenode, #hurd, 2013-09-23 + + <teythoon> I've been hacking on init/startup, I've looked into cleaning it + up diff --git a/open_issues/libnetfs_passive_translators.mdwn b/open_issues/libnetfs_passive_translators.mdwn new file mode 100644 index 00000000..db4c9005 --- /dev/null +++ b/open_issues/libnetfs_passive_translators.mdwn @@ -0,0 +1,55 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + + +# IRC, freenode, #hurd, 2013-07-15 + + <teythoon> is there a libnetfs based translator that supports passive + translators? + <youpi> I don't see any at the top of my head, only with active ones such + as hostmux + <teythoon> I suspected as much since as far as I can tell libnetfs lacks + some bits to make that even work + <braunr> teythoon: the problem with passive translators is persistence + <braunr> well, it's easy to store volatile passive translators in a + libnetfs server + <braunr> but usually, there is no backing store for them + <braunr> ext2fs is the only one actually providing space to store their + command line + <teythoon> sure, but at least file_get_translator needs to work so that + procfs can serve a mounts node + <braunr> silly idea but + <braunr> don't you want to directly add it to the procfs translator ? + <teythoon> no, I think it's useful on its own + <braunr> ok + <braunr> then you may need to add the required support + <teythoon> it even doubles as normal command line tool + <teythoon> yes, I almost got it... or so I hope ;) + <braunr> ok + <teythoon> also, netfs_get_translator exists, so not supporting that feels + like a bug to me + <teythoon> could also be useful for a potential devfs translator + <braunr> yes + + <teythoon> uh, the code duplication in lib*fs is really bad :/ + <teythoon> the code is mostly similar, though they have diverged and many + little things are different so diffing them is very noisy + <teythoon> stuff like file names or identifiers + <teythoon> and I cannot figure out why my shiny passive translators are not + started :/ + <teythoon> % showtrans tmp/mounts + <teythoon> /hurd/mtab.fixed / + <teythoon> % wc --bytes tmp/mounts + <teythoon> 0 tmp/mounts + <teythoon> and no mtab translator around either + +[[community/gsoc/project_ideas/mtab/discussion]]. diff --git a/open_issues/libnetfs_argument_parsing.mdwn b/open_issues/libnetfs_vs_libdiskfs.mdwn index e1e0d794..2fcfbea5 100644 --- a/open_issues/libnetfs_argument_parsing.mdwn +++ b/open_issues/libnetfs_vs_libdiskfs.mdwn @@ -10,7 +10,12 @@ License|/fdl]]."]]"""]] [[!tag open_issue_hurd]] -# IRC, freenode, #hurd, 2013-06-27 +[[!toc]] + + +# Argument Parsing + +## IRC, freenode, #hurd, 2013-06-27 <teythoon> the arg parsing in libdiskfs and libnetfs differ :/ <teythoon> afaics libdiskfs gets it right, libnetfs does not @@ -60,3 +65,54 @@ License|/fdl]]."]]"""]] <teythoon> http://paste.debian.net/12938/ <teythoon> whee, remounting proc works :) <braunr> :) + + +# IRC, freenode, #hurd, 2013-07-29 + + <teythoon> so, what do you folks think about refactoring libdiskfs and + libnetfs to be more alike? + <pinotree> what do you mean? + <teythoon> ah, I mentioned that in the context of my mtab prototype + 1374247519-26589-1-git-send-email-4winter@informatik.uni-hamburg.de + <teythoon> they are hard to diff against each other b/c they differ in file + names and identifier names + <teythoon> while working on the mtab stuff I encountered stuff that was + implemented in libdiskfs, but never in libnetfs + <teythoon> mostly support for binding translators to libnetfs nodes + <braunr> teythoon: sure, but looks a little out of scope + <teythoon> braunr: I do not mean now, more in general + <braunr> ok + <tschwinge> teythoon: I wondered about this, too. I don't know if it's + possible to literally merge them (and build the backend-based (libdiskfs) + vs. volatile-backend one (libnetfs) based on a pre-processor define or + similar), or just structure the source code (files) in a way such that + »diff -ru libdiskfs/ libnetfs/« gives meaningful results, figuratively + spoken. + <teythoon> tschwinge: my thoughts exactly + + +# IRC, freenode, #hurd, 2013-08-28 + + <teythoon> braunr: do you think another lib*fs would be frowned uppon? I + like the way procfs is structured and that could be refactored and + generalized into a library + <braunr> i think we need more lib*fs libraries + <braunr> and better integration + <braunr> that's one of the strengths in linux + <braunr> it makes writing file systems very easy + <teythoon> cool :) + <teythoon> now we only need a snappy name, any suggestions? + <braunr> i don't know what you like specificlaly in procfs + <braunr> libpseudofs ? + <teythoon> well, it's not perfect, but i like the way you just have to + implement a function for a node and it magically gains the ability to + being read + <teythoon> for example + <braunr> yes i see + <pinotree> lacks a bit of caching though + <braunr> no caching for such file systems + <teythoon> indeed + <braunr> why would you want caching ? + <pinotree> you might have files that don't change at all, or rarely do + <braunr> the premise is that it's meant for files generated on the fly + <braunr> but are they big ? diff --git a/open_issues/libpthread.mdwn b/open_issues/libpthread.mdwn index 8e3fde71..0b426884 100644 --- a/open_issues/libpthread.mdwn +++ b/open_issues/libpthread.mdwn @@ -974,7 +974,7 @@ Most of the issues raised on this page has been resolved, a few remain. <braunr> exec weights 164M eww, we definitely have to fix that leak <braunr> the leaks are probably due to wrong mmap/munmap usage -[[exec_leak]]. +[[exec_memory_leaks]]. ### IRC, freenode, #hurd, 2012-08-29 diff --git a/open_issues/libpthread/t/fix_have_kernel_resources.mdwn b/open_issues/libpthread/t/fix_have_kernel_resources.mdwn index 4e35161f..6f09ea0d 100644 --- a/open_issues/libpthread/t/fix_have_kernel_resources.mdwn +++ b/open_issues/libpthread/t/fix_have_kernel_resources.mdwn @@ -196,3 +196,220 @@ Address problem mentioned in [[/libpthread]], *Threads' Death*. ## IRC, freenode, #hurd, 2013-07-03 <braunr> grmbl, i don't want to give up thread destruction .. + + +## IRC, freenode, #hurd, 2013-07-15 + + <braunr> btw, my work on thread destruction is currently stalled + <braunr> i don't have much free time right now + + +## IRC, freenode, #hurd, 2013-09-13 + + <braunr> i think i know why my thread_terminate_deallocate patches leak one + receive port :> + <braunr> but now i'm not sure of the proper solution + <braunr> every time a thread is created and destroyed, a receive right is + leaked + <braunr> i guess it's simply the reply port .. + <braunr> grmbl + <braunr> i guess i have to make it a simpleroutine ... + <braunr> hm too bad, it's not the reply port :( + <braunr> it's also leaking some memory + <braunr> it doesn't seem related to my changes though + <braunr> stacks, rights, and threads are correctly destroyed + <braunr> some obscure state is left behind + <braunr> i wonder how exception ports are dealt with + <braunr> vminfo seems to confirm memory is leaking in the heap + <braunr> humpf + <braunr> oh silly me + <braunr> i don't detach threads + <teythoon> well, detach them ;) + <braunr> hm worse :p + <braunr> now i get additional dead names + <braunr> but it's a step forward + + +## IRC, freenode, #hurd, 2013-09-16 + + <braunr> that thread port leak is so strange + <braunr> the leaked port seems to be created when the new thread starts + running + <braunr> so it looks like a port the kernel would implicitely create + <braunr> hm could it be a thread-specific reply port ? + <youpi> ah, yes, there is one of those + <braunr> how come mach/mig-reply.c in glibc isn't thread-safe ? + <youpi> it is overriden by sysdeps/mach/hurd/img-reply.c I guess + <youpi> which uses a threadvar for the mig reply port + <braunr> oh + <youpi> talking of which, there is also last_value in + sysdeps/mach/strerror_l.c + <youpi> strerror_thread_freeres is supposed to get called, but who knows + <braunr> it does look to be that port + <youpi> iirc that's the issue which prevents from letting us make threads + exit on idleness? + <braunr> one of them + <youpi> ok + <braunr> maybe the only one, yes + <braunr> i see memory leaks but they could be related/normal + <braunr> (i.e. not actual leaks) + <braunr> on the other hand, i also can't boot a hurd with my patch + <braunr> but i consider removing such leaks a priority + <braunr> does anyone know the semantic difference between + __mig_put_reply_port and __mig_dealloc_reply_port ? + <braunr> i guess __mig_dealloc_reply_port is actually a destruction + operation, right ? + <youpi> AIUI, dealloc is used when one wants the port not to be reused at + all + <youpi> because it has been used as a reference for something, and can + still be currently in use + <youpi> while put_reply would be when we're really done with it, and won't + use it again, and can thus be used as such + <youpi> or at least something like that + <braunr> heh + <braunr> __mig_dealloc_reply_port calls __mach_port_mod_refs, which is a + RPC, and creates a new reply port when destroying the current one + <youpi> bah + <youpi> that's fine, it's a deref of the old port, which is not in the + reply_port variable any more + <braunr> it's fine, but still a leak + <youpi> well, dealloc does not completely deallocs, yes + <braunr> that's not really the problem here + <braunr> i've introduced a case that wasn't considered at the time, namely + that a thread can destroy itself + <youpi> we probably need another function to be called from the thread exit + <braunr> i'll simply try with mach_port_destroy + <braunr> mach_port_destroy seems to be a RPC too ... + <braunr> grmbl + <youpi> isn't there a trap version somehow ? + <braunr> not in libc + <youpi> erf + <braunr> at least i know what's wrong now :) + <braunr> there still is a small memory leak i have to investigate + <braunr> but outside the stack + <braunr> the stack, the thread name and the thread are correctly destroyed + <braunr> slabinfo confirms only one port leak and nothing else is leaked + <braunr> ok so the port leak was indeed the thread-specific reply port, + taken care of + <braunr> there are also memory leaks too + + +## IRC, freenode, #hurd, 2013-09-17 + + <braunr> teythoon: on my side, i'm getting to know our threading + implementation better + <braunr> closing to clean thread destruction + <braunr> x15 ipc will hide reply ports ;p + <braunr> memory leaks solved \o/ + <braunr> now, have to fix memory release when joining + <braunr> proper reference counting on detach/join/exit, let's see how it + goes .. + <braunr> seems to work fine + + +## IRC, freenode, #hurd, 2013-09-18 + + <braunr> ok i'll soon have gnumach and libc packages including proper + thread destruction :> + <teythoon> braunr: why did you have to touch gnumach? + <braunr> to add a call allowing threads to release ports and memory + <braunr> i.e. their last self reference, their reply port and their stack + <braunr> let me public my current patches + <teythoon> braunr: thread_commit_suicide ? + <braunr> hehe + <braunr> initially thread_terminate_self but + <braunr> it can be used by other threads too + <braunr> to i named it thread_terminate_release + <braunr> http://darnassus.sceen.net/~rbraun/0001-pthread_thread_halt.patch + <braunr> + http://darnassus.sceen.net/~rbraun/0001-thread_terminate_release.patch + <braunr> the pthread patch needs to be polished because it changes the + semantics of pthread_thread_halt + <braunr> but other than that, it should be complete + <pinotree> pthread_thread_halt_reallyhalt + <braunr> ok let's try these libc packages + <braunr> old static ext2fs for the root, but other than that, it boots + <braunr> let's try iceweasel + <braunr> (i'll need to build a hurd package against this new libc, removing + the libports_stability patch which prevents thread destruction in servers + on the way) + <teythoon> prevents thread destruction o_O + <braunr> yes + <braunr> in libports only ;p + <teythoon> oh, *only* in libports, I assumed for a moment that it affected + almost every component of the Hurd... + <teythoon> *phew( + <braunr> ... :) + <braunr> that's why, after a burst of messages, say because of aptitude + (select), you may see a few hundred threads still hanging around + <braunr> also why unused servers remain running even after several minutes, + where the normal timeout is 2mins + <teythoon> I wondered about that, some servers (symlink comes to mind) seem + to go away if unused (or that's how I read the code) + <braunr> symlinks are usually not servers, since most of them actually + exist in file systems, and are implemented through an optimization + <teythoon> yes I know that + <teythoon> trans/symlink.c reads: + <teythoon> /* The timeout here is 10 minutes */ + <teythoon> err = mach_msg_server_timeout (fsys_server, 0, control, + <teythoon> MACH_RCV_TIMEOUT, 1000 * 60 * 10); + <teythoon> if (err == MACH_RCV_TIMED_OUT) + <teythoon> exit (0); + <braunr> ok + <teythoon> hm, /hurd/symlink doesn't feel at all like a symlink... but + works like one + <braunr> well, starting iceweasel makes X on my host freeze oO + <braunr> bbl + <teythoon> /hurd/symlink translators do go away after being unused for 10 + minutes... this is funny if they are set up by hand instead of being + started from a passive translator record + <teythoon> magically vanishing symlinks ;) + + +## IRC, freenode, #hurd, 2013-09-19 + + <braunr> hum, i can't rebuild a hurd package :( + <teythoon> braunr: with your thread destruction patches in libc? + <braunr> yes but it's unrelated + <braunr> In file included from ../../libdiskfs/boot-start.c:38:0: + <braunr> ./fsys_reply_U.h:173:15: error: conflicting types for + ‘fsys_get_children’ + <braunr> i didn't see a new libc debian release + <teythoon> hm, David reported that as well + <teythoon> + id:CAEvUa7=QzOiS41G5Vq8k4AiaN10jAPm+CL_205OHJnL0xpJXbw@mail.gmail.com + <teythoon> uh oh + <teythoon> it seems I didn't add a _reply suffix to the reply routines :/ + <teythoon> there's quite a bit of fallout from my patches, I kinda feel bad + :( + <braunr> teythoon: what i'm wondering is what youpi did too, since he got + hurd binary packages + <teythoon> braunr: well neither he nor I noticed that b/c for us the + declarations were just missing + <braunr> from libc you mean ? + <braunr> or hum gnumach-common ? + <teythoon> not sure actually + <braunr> no it's not a gnumach thing + <braunr> hurd-dev then + <teythoon> the build system should have cought these, or mig... + <braunr> also, i see you changed fsys_reply.defs, but nothing about + fsys_request.defs + <teythoon> I have no fsys_requests.defs + <braunr> looks like there was no fsys_request.defs in the first place + ... *sigh* + <braunr> do you know an application that often creates and destroys threads + ? + <teythoon> no, sorry + <pinotree> maybe some test suite + <braunr> ah right + <braunr> sysbench maybe + <braunr> also, i've been hit by a lot more network deadlocks than usual + lately + <braunr> fixing netdde has gained some priority in my todo list + + +## IRC, freenode, #hurd, 2013-09-20 + + <braunr> oh, git is multithreaded + <braunr> great + <braunr> so i've actually tested my libpthread patch quite a lot diff --git a/open_issues/libpthread_dlopen.mdwn b/open_issues/libpthread_dlopen.mdwn index 05a07ef2..5d574261 100644 --- a/open_issues/libpthread_dlopen.mdwn +++ b/open_issues/libpthread_dlopen.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2012, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -112,7 +113,18 @@ IRC, freenode, #hurd, 2011-08-17 [[packaging_libpthread]] ---- + +# IRC, freenode, #hurd, 2013-09-03 + + <gnu_srs> iceweasel: ./pthread/../sysdeps/generic/pt-mutex-timedlock.c:70: + __pthread_mutex_timedlock_internal: Assertion `__pthread_threads' failed. + <pinotree> LD_PRELOAD libpthread + <gnu_srs> why + <pinotree> missing link to pthread? + <pinotree> and yes, it's known already, just nobody worked on solving it + + +# libthreads vs. libpthread The same symptom appears in an odd case, for instance: diff --git a/open_issues/llvm.mdwn b/open_issues/llvm.mdwn index 30b18edf..4da58579 100644 --- a/open_issues/llvm.mdwn +++ b/open_issues/llvm.mdwn @@ -101,6 +101,35 @@ a06fe9183fbffb78798a444da9bc3040fdd444aa (2013-03-23), test-suite A lot of Linux-specific things. + * IRC, OFTC, #debian-hurd, 2013-09-05: + + <gg0> how can this fix it on {kf,hurd}-i386? + http://anonscm.debian.org/viewvc/pkg-llvm/llvm-toolchain/branches/3.3/debian/patches/libstdc%2B%2B-header-i386.diff?view=markup&sortby=date&pathrev=830 + <pinotree> what makes you think it does? + <pinotree> it fixes #714890, which has nothing to do with hurd or + kfreebsd + <gg0> i simple wouldn't add a patch that fixes it on one i386 arch + only, being aware there are others + <pinotree> meet sylvestre + + * IRC, freenode, #hurd, 2013-09-05: + + <pinotree> tschwinge: iirc you were working on llvm/clang, weren't you? + <tschwinge> pinotree: That's right. I have patches to + follow-up/rework. Stalled at the moment, as you probably already + guessed... %-) + <pinotree> tschwinge: <Sylvestre> by the way, pinotree if you have time + for hurd stuff, I would be glad to have your help to port + llvm-toolchain-3.3 to hurd. I am having some issues with threading + aspects https://paste.debian.net/35466/ + <pinotree> he's the debian packager of llvm + <tschwinge> That paste is for LLDB -- which I'd not assume to be in a + shape usable for Hurd. + <tschwinge> (I didn't look at it at all.) + <pinotree> tschwinge: if you look at the latest llvm-toolchain-3.3 + debian source, there's a lldb-hurd.diff patch, which starts some + include header dance + # Build diff --git a/open_issues/mach_migrating_threads.mdwn b/open_issues/mach_migrating_threads.mdwn index c14ce95a..bbc6ac45 100644 --- a/open_issues/mach_migrating_threads.mdwn +++ b/open_issues/mach_migrating_threads.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -15,3 +15,89 @@ License|/fdl]]."]]"""]] * [[microkernel/mach/memory_object/discussion]] * [[resource_management_problems]] + + +# IRC, freenode, #hurd, 2013-08-13 + +In context of [[resource_management_problems]]. + + <braunr> and thread migration itself is something very confusing + <braunr> it's better to think of it as scheduling context inheritance + <teythoon> braunr: I read the paper I mentioned and then I wanted to find + the sources they modified + <teythoon> I failed + <teythoon> I hate scientific paper about software that fail to provide the + source code + <teythoon> that's not science imho b/c it's not reproducible + <braunr> i have some osf source code here + <braunr> i'll send it if you want + <teythoon> ah interesting + <braunr> but really, when you dive into it, thread migration is merely + scheduling context inheritance with kernel upcalls + <braunr> it's good + <teythoon> I searched for osf mach but google didn't turned up anything + <braunr> but it has nothing to do with resource accounting + <braunr> (well, it may hepl better account for cpu time actually) + <braunr> help* + <braunr> but that's all + <teythoon> why is that all? wouldn't that be transitive and could also be + used for i/o accounting? + <teythoon> also I tried to find alternative mach implementations + <teythoon> I wasn't terribly successful, and some sites are gone or + unmaintained for years :/ + <braunr> we don't need that for io accountin + <braunr> g + <braunr> thread migration is a kernel property + <braunr> on mach with userspace drivers, io isn't + <braunr> mach should only control cpu and memory + <braunr> and you though you can account physical memory, you can't transfer + virtual memory accounting from one task to another + <teythoon> yes, but once all of those resources can be accounted to the + thread initiating whatever it needs doing, shouldn't that be much easier? + <braunr> teythoon: it's not required for that + <braunr> teythoon: keep in mind userspace sees activations + <braunr> in a thread migration enabled kernel, activations are what we + usually call threads, and threads are scheduling contexts + <teythoon> braunr: ok, so TM is not required for accounting, but surely + it's a good thing to have, no? + <braunr> teythoon: it's required for cpu accounting only + <braunr> which is very important :) + <braunr> if you look carefully, you'll see hurd servers are what use most + cpu + <braunr> there is now easy way to know which application actually uses the + server + <braunr> i personally tend to think more and more that servers *should* + impersonate clients + <braunr> TM (or rather, scheduling context inheritance) is one step + <braunr> it's not enough exactly because it doesn't help with resource + accounting + <braunr> teythoon: + ftp://ftp.mklinux.org/pub/mklinux-pre-R1/SRPMS/sources/osfmk.tar.gz + + +# IRC, freenode, #hurd, 2013-09-02 + +[[!taglink open_issue_documentation]]: move information to +[[microkernel/mach/history]]. + + <teythoon> braunr: btw, I just noticed lot's of #ifdef MIGRATING_THREADS in + gnumach, so there was some work being done in that direction? + <braunr> gnumach is a fork of mach4 + <braunr> at a stage whre migration was being worked on, yes + <teythoon> from what I've gathered, gnumach is the only surviving mach4 + fork, right? + <braunr> yes + <braunr> well + <braunr> the macos x version is probably one too + <braunr> i don't know + <teythoon> oh? I read that it was based on mach3 + <braunr> it is + <braunr> i can't tell how much of mach3 versus mach4 it has, and if it's + relevant at all + <teythoon> and the osfmach, was that also based on mach4? + <braunr> yes + <teythoon> ok, fair enough + <braunr> that's why i think macos x is based on it too + <braunr> i initially downloaded osfmach sources to see an example of how + thread migration was used from userspace + <braunr> and they do have a special threading library for that diff --git a/open_issues/memory_object_model_vs_block-level_cache.mdwn b/open_issues/memory_object_model_vs_block-level_cache.mdwn index 7da5dea4..22db9b86 100644 --- a/open_issues/memory_object_model_vs_block-level_cache.mdwn +++ b/open_issues/memory_object_model_vs_block-level_cache.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,6 +10,8 @@ License|/fdl]]."]]"""]] [[!tag open_issue_documentation open_issue_hurd open_issue_gnumach]] +[[!toc]] + # IRC, freenode, #hurd, 2012-02-14 @@ -271,3 +273,242 @@ License|/fdl]]."]]"""]] <mcsim> slpz: When mo_data_return is called, once the memory manager no longer needs supplied data, it should be deallocated using vm_deallocate. So this way pagers acknowledges the end of flush. + + +# IRC, freenode, #hurd, 2013-08-26 + + < Spyro> Ok, so + < Spyro> idiot question: in a nutshell, what is a memory object? + < Spyro> and how is swapping/paging handled? + < braunr> Spyro: a memory object is how the virtual memory system views a + file + < braunr> so it's a sequence of bytes with a length + < braunr> "swapping" is just a special case of paging that applies to + anonymous objects + < braunr> (which are named so because they're not associated with a file + and have no name) + < Spyro> Who creates a memory object, and when? + < braunr> pagers create memory objects when needed, e.g. when you open a + file + < Spyro> and this applies both to mmap opens as well as regular I/O opens + as in for read() and write()? + < braunr> basically, all file systems capable of handling mmap requests + and/or caching in physical memory are pagers + < braunr> yes + < braunr> read/write will go through the page cache when possible + < Spyro> and who owns the page cache? + < Spyro> also, who decides what pages ot evict to swap/file if physical + memory gets tight? + < braunr> the kernel + < braunr> that's one of the things that make mach a hybrid + < Spyro> so the kernel owns the page cage? + < Spyro> ...fml + < Spyro> cache! + < braunr> yes + + +## IRC, freenode, #hurd, 2013-08-27 + + < Spyro> so braunr: So, who creates the memory object, and how does it get + populated? + < Spyro> and how does a process accessing a file get hooked up to the + memory object? + < braunr> Spyro: i told you, pagers create memory objects + < braunr> memory objects are how the VM system views files, so they're + populated from the content of files + < braunr> either true files or virtual files such as in /proc + < braunr> Spyro: processes don't directly access memory objects unless + memory mapping them with vm_map() + < braunr> pagers (basically = file systems) do + <Spyro> ok, so how is a pager/fs involved in handling a fault? + + +## IRC, freenode, #hurd, 2013-08-28 + + <braunr> Spyro: each object is linked to a pager + <braunr> Spyro: when a fault occurs, the kernel looks up the VM map (kernel + or a user one), and the address in this map, then the map entry, checks + access and lots of other details + <Spyro> ok, so it's pager -> object -> vmem + <Spyro> ? + <braunr> Spyro: then finds the object mapped at that address (similar to + how a file is mapped with mmap) + <braunr> from the object, it finds the pager + <Spyro> ok + <braunr> and asks the pager about the data at the appropriate offset + <Spyro> so how does a user process do normal file I/O? is faulting just a + special case of it? + <braunr> it's completely separate + <Spyro> eww + <braunr> normal I/O is done with message passing + <braunr> the hurd io interface + <Spyro> ok + <Spyro> so who talks to who on a file I/O? + <braunr> a client (e.g. cat) talks to a file system server (e.g. ext2fs) + <Spyro> ok so + <Spyro> it's client to the pager for regular file I/O? + <braunr> Spyro: i don't understand the question + <braunr> Spyro: it's client to server, the server might not be a pager + <Spyro> ok + <Spyro> just trying to figure out the difference between paging/faulting + and regular I/O + <braunr> regular I/O is just message passing + <braunr> page fault handling is dealt with by pagers + <Spyro> and I have a hunch that the fs/pager is involved somehow in both, + because the server is the source of the data + <Spyro> I'm getting a headache + <braunr> nalaginrut: a server like ext2fs is both a file server and a pager + <Spyro> oh! + <Spyro> oh btw, does a file server make use of memory objects for caching? + <braunr> Spyro: yes + <Spyro> or rather, can it? + <Spyro> does it have to? + <braunr> memory objects are for caching, and thus for page faults + <braunr> Spyro: for caching, it's a requirement + <braunr> for I/O, it's not + <braunr> you could have I/O without memory objects + <Spyro> ok + <Spyro> so how does the pager/fileserver use memory objects for caching? + <Spyro> does it just map and write to them? + <braunr> basically yes but there is a complete protocol with the kernel for + that + <braunr> + http://www.gnu.org/software/hurd/gnumach-doc/External-Memory-Management.html#External-Memory-Management + <Spyro> heh, lucky guess + <Spyro> ty + <Spyro> I am in way over my head here btw + <Spyro> zero experience with micro kernels in practice + <braunr> it's not trivial + <braunr> that's not a microkernel thing at all + <braunr> that's how it works in monolithic kernels too + <braunr> i recommend netbsd uvm thesis + <braunr> there are nice pictures describing the vm system + <Spyro> derrr...preacious? + <Spyro> wow + <braunr> just ignore the anonymous memory handling part which is specific + to uvm + <Spyro> @_@ + <braunr> the rest is common to practically all VM systems out there + <Spyro> I know about the linux page cache + <braunr> well it's almost the same + <Spyro> with memory objects being the same thing as files in a page cache? + <braunr> memory objects are linux "address spaces" + <braunr> and address spaces are how the linux mm views a file, yes + <Spyro> derp + <Spyro> ... + <Spyro> um... + <braunr> struvt vm_page == struct page + * Spyro first must learn what an address_space is + <braunr> struct vm_map == struct mm_struct + <braunr> struct vm_map_entry == struct vm_area_struct + * Spyro isn't a linux kernel vm expert either + <braunr> struct vm_object == struct address_space + <braunr> roughly + <braunr> details vary a lot + <Spyro> and what's an address_space ? + <braunr> 11:41 < braunr> and address spaces are how the linux mm views a + file, yes + <Spyro> ok + <braunr> see include/linux/fs.h + <braunr> struct address_space_operations is the pager interface + * Spyro should look at the linux kernel sources perhaps, unless you have an + easier reference + <Spyro> embarrassingly, RVR hired me as an editor for the linux-mm wiki + <Spyro> I should know this stuff + <braunr> see + http://darnassus.sceen.net/~rbraun/design_and_implementation_of_the_uvm_virtual_memory_system.pdf + <braunr> page 42 + <braunr> page 66 for another nice view + <braunr> i wouldn't recommend using linux source as refernece + <braunr> it's very complicated, filled with a lot of code dealing with + details + <Spyro> lmao + <braunr> and linux guys have a habit of choosing crappy names + <Spyro> I was only going to + <Spyro> stoppit + <braunr> except for "linux" and "git" + <Spyro> ...make me laugh any more and I'll need rib surgery + <braunr> laugh ? + <Spyro> complicated and crappy + <braunr> seriously, "address space" for a file is very very confusing + <Spyro> oh I agree with that + <braunr> yes, names are crappy + <braunr> and the code is very complicated + <braunr> it took me half an hour to find where readahead is done once + <braunr> and i'm still not sure it was the right code + <Spyro> so in linkern, there is an address_space for each cached file? + <braunr> takes me 30 seconds in netbsd .. + <braunr> yes + <Spyro> eww + <Spyro> yeah, BAD name + <Spyro> but thanks for the explanation + <Spyro> now I finally know what an address space is + <braunr> many linux core developers admit they don't care much about names + <Spyro> so, in hurd, a memory object is to hurd, what an address_space is + to linux? + <braunr> yes + <braunr> notto hurd + <Spyro> ok + <braunr> to mach + <Spyro> you know what I mean + <Spyro> :P + <Spyro> easier than for linux I can tell you that much + <braunr> and the bsd vm system is a stripped version of the mach vm + <Spyro> ok + <braunr> that's why i think it's important to note it + <Spyro> good, I learned something abou tthe linux vm...from the mach guys + <Spyro> this is funny + <braunr> linux did too + <braunr> there is a paper about linux page eviction that directly borrows + the mach algorithm and improves it + <braunr> mach is the historic motivation behind mmap on posix + <Spyro> oh nice! + <Spyro> but yes, linux picked a shitty name + <braunr> is all that clearer to you ? + <Spyro> I think that address_space connection was a magic bolt of + understanding + <braunr> and do you see how I/O and paging are mostly unrelated ? + <Spyro> almost + <Spyro> but how does a file I/O take advantage of caching by a memory + object? + <Spyro> does the file server just nudge the core for a hint? + <braunr> the file system copies from the memory object + * Spyro noddles + <Spyro> I think I understand a bit better now + <braunr> it's message passing + <Spyro> but I havfe too much to digest already + <braunr> memory copying + <braunr> if the memory is already there, good, if not, the kernel will ask + the file system to bring the data + <braunr> if message passing uses zero copy, data retrieval can be deferred + until the client actually accesses it + <Spyro> which is a fancy way of saying demand paging? :P + <braunr> it's always demand paging + <braunr> what i mean is that the file system won't fetch data as soon as it + copies memory + <braunr> but when this data is actually needed by the client + <Spyro> uh... + <Spyro> whta's a precious page? + <braunr> let me check quickly + <braunr> If precious is FALSE, the kernel treats the data as a temporary + and may throw it away if it hasn't been changed. If the precious value is + TRUE, the kernel treats its copy as a data repository and promises to + return it to the manager + <braunr> basically, it's used when you want the kernel to keep cached data + in memory + <braunr> the cache becomes a lossless container for such pages + <braunr> the kernel may flush them, but not evict them + <Spyro> what's the difference? + <braunr> imagine a ramfs + <Spyro> point made + <braunr> ok + <Spyro> would be pretty hard to flush something that doesn't have a backing + store + <braunr> that was quick :) + <braunr> well + <braunr> the normal backing store for anonymous memory is the default pager + <braunr> aka swap + <Spyro> eww + <braunr> but if you want your data *either* in swap or in memory and never + in both + <braunr> it may be useful diff --git a/open_issues/mig_portable_rpc_declarations.mdwn b/open_issues/mig_portable_rpc_declarations.mdwn index 084d7454..91838f60 100644 --- a/open_issues/mig_portable_rpc_declarations.mdwn +++ b/open_issues/mig_portable_rpc_declarations.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -56,3 +56,114 @@ License|/fdl]]."]]"""]] <antrik> braunr: we discussed the problem of expressing structs with MIG in the libburn thread <antrik> (which I still need to follow up on... [sigh]) + + +# IRC, freenode, #hurd, 2013-06-25 + + <teythoon> is there a nice way to get structured data through mig that I + haven't found yet? + <teythoon> say an array of string triples + <braunr> no + <teythoon> :/ + <braunr> but you shouldn't need that + <teythoon> my use case is getting info about fs translators from init to + procfs + +[[community/gsoc/project_ideas/mtab]], +[[community/gsoc/project_ideas/mtab/discussion]]. + + <teythoon> should I go for an iterator like interface instead? + <braunr> depends + <braunr> how many do you need ? + <braunr> you could go for a variable sized array too + <braunr> have a look at what already exists + <teythoon> records, maybe 10-15, depends on many fs translators are running + <braunr> a variable sized array is ok if the size isn't too big (and when i + say too big, i mean hundreds of MiB) + <braunr> an iterator is ok too if there aren't too many items + <braunr> you may want to combine both (i think that's what proc does) + <braunr> be aware that the maximum size of a message is limited to 512 MiB + <teythoon> yeah I saw the array[] of stuff stuff, but array[] of string_t + does not work, I guess b/c string_t is also an array + <teythoon> how would I send an array of variable length strings? + <braunr> i'm not sure you can + <braunr> or maybe out of line + <teythoon> somehow I expected mig to serialize arbitrary data structures, + maybe it's to old for that? + <teythoon> yeah, I read about uot of line, but that seems overkill + <braunr> it is old yes + <braunr> and not very user friendly in the end + <braunr> let me check + <teythoon> we could stuff json into mig... + <braunr> see proc_getallpids for example + <braunr> we could get rid of low level serialization altogether :p + <teythoon> hah, exactly what I was looking at + <braunr> (which is what i'll do in x15) + <braunr> type pidarray_t = array[] of pid_t; + <teythoon> but that is trivial b/c its array[] of pid_t + <braunr> and always have the server writing guide near you + <teythoon> yes + <braunr> well, make one big string and an array of lengths :p + <teythoon> thought about that and said to myself, there must be a better + way that I haven't found yet + <braunr> or one big string filled with real null-terminated c strings that + you keep parsing until you ate all input bytes + <braunr> i'm almost certain there isn't + <braunr> type string_t = c_string[1024]; /* XXX */ + <teythoon> yes + <braunr> even that isn't really variable sized + <teythoon> you think anyone would object to me putting a json encoder in + /hurd/init? it is probably better than me at serializing stuff... + <braunr> try with mig anyway + <braunr> the less dependencies we have for core stuff, the simpler it is + <braunr> but i agree, mig is painful + <teythoon> would it be too hacky if I abused the argz functions? they do + exactly what I'd need + + +## IRC, freenode, #hurd, 2013-06-26 + + <teythoon> there is https://code.google.com/p/protobuf-c/ and it has a rpc + mechanism and I believe one could plug arbitrary transports easily + <braunr> please don't think about it + <braunr> we really don't want to add another layer of serialization + <braunr> it's better to completely redesign mach ipc anyway + <braunr> and there is a project for that :p + <teythoon> ive seen x15 + <teythoon> just food for thought + <braunr> i've studied google protocol buffers + <braunr> and fyi, no, it wouldn't be easy to plug arbitrary transports on + top of mach + <braunr> there is a lot of knowledge about mach ports in mig + +[[community/gsoc/project_ideas/mtab]], +[[community/gsoc/project_ideas/mtab/discussion]]. + + <teythoon> but again I face the challenge of serializing a arbitrary sized + list of arbitrary sized strings + <braunr> yes + <teythoon> list of ports is easier ;) but I think its worthwile + <teythoon> so what about abusing argz* for this? you think it's too bad a + hack? + <braunr> no since it's in glibc + <teythoon> awesome :) + <braunr> but i don't remember the details well and i'm not sure the way you + use it is safe + <teythoon> yeah, I might have got the details wrong, I hadn't had the + chance to test it ;) + + <braunr> about this dynamic size problem + <braunr> a "simple" varying size array should do + <braunr> you can easily put all your strings in there + <teythoon> seperated by 0? + <braunr> yes + <teythoon> that's exactly what the argz stuff does + <braunr> you'll get the size of the array anyway, and consume it until + there is no byte left + <braunr> good + <braunr> but be careful with this too + <braunr> since translators can be run by users, they somtimes can't be + trusted + <braunr> and even a translator running as root may behave badly + <braunr> so careful with parsing + <teythoon> noted diff --git a/open_issues/mig_stub_functions.mdwn b/open_issues/mig_stub_functions.mdwn new file mode 100644 index 00000000..24a582b1 --- /dev/null +++ b/open_issues/mig_stub_functions.mdwn @@ -0,0 +1,41 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_mig]] + +[[!toc]] + + +# RPC Stubs Implemented by Hand + +## IRC, freenode, #hurd, 2013-07-28 + + <teythoon> why is libfshelp/start-translator-long.c doing the fsys_startup + rpcs by hand instead of using the mig generated stubs? + + +## IRC, freenode, #hurd, 2013-07-29 + + <teythoon> btw, anyone knows why libfshelp/start-translator-long.c + implements the fsys_startup rpc by hand? + <braunr> teythoon: no idea + <teythoon> maybe b/c of the need to specify a timeout? can one do that with + the mig stubs? + <braunr> yes + <braunr> select used to be implemented that way + + +# Generate the Request and Reply Routines from the Synchronous Routines + +## IRC, freenode, #hurd, 2013-09-19 + + <teythoon> btw, is there any reason why mig couldn't generate the request + and reply routines from the synchronous routines? + <braunr> i guess it could diff --git a/open_issues/nptl.mdwn b/open_issues/nptl.mdwn index 9ff5fb51..3c84bfb0 100644 --- a/open_issues/nptl.mdwn +++ b/open_issues/nptl.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,7 +10,10 @@ License|/fdl]]."]]"""]] [[!tag open_issue_libpthread open_issue_glibc]] -IRC, #hurd, 2010-07-31 +[[!toc]] + + +# IRC, freenode, #hurd, 2010-07-31 <tschwinge> Other question: how difficult is a NPTL port? Futexes and some kernel interfaces for scheduling stuff etc. -- what else? <youpi> actually NPTL doesn't _require_ futexes @@ -26,6 +29,21 @@ IRC, #hurd, 2010-07-31 <tschwinge> ... and even less so the interfavce that actual applications are using. <tschwinge> We'd need to evaluate which benefits NPTL would bring. + +# IRC, freenode, #hurd, 2013-08-05 + + <gnu_srs> Hi, looks like kfreebsd are now using an NPTL-based pthread + library: FBTL, http://lists.debian.org/debian-bsd/2013/07/msg00060.html + <gnu_srs> Anything of interest for porting to Hurd? See also + http://lists.debian.org/debian-hurd/2013/08/msg00000.html + <azeem> Petr could've been more verbose in his announcements + <pinotree> and there's + http://www.gnu.org/software/hurd/open_issues/nptl.html in our wiki + <azeem> well, it seems to work fine for kFreeBSD: + http://lists.debian.org/debian-bsd/2013/07/msg00134.html + <azeem> and http://lists.debian.org/debian-bsd/2013/07/msg00138.html + + --- # Resources diff --git a/open_issues/pthread_atfork.mdwn b/open_issues/pthread_atfork.mdwn index ac724cf0..1b656f05 100644 --- a/open_issues/pthread_atfork.mdwn +++ b/open_issues/pthread_atfork.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -8,6 +8,13 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!tag open_issue_libpthread]] +[[!tag open_issue_glibc open_issue_libpthread]] -pthread_atfork is not actually implemented, making some programs fail. Code can probably be borrowed from nptl/sysdeps/unix/sysv/linux/register-atfork.c +`pthread_atfork` is not actually implemented, making some programs fail. Code +can probably be borrowed from `nptl/sysdeps/unix/sysv/linux/register-atfork.c`. + + +# IRC, OFTC, #debian-hurd, 2013-08-21 + + <pinotree> SRCDIR/opal/mca/memory/linux/arena.c:387: warning: warning: + pthread_atfork is not implemented and will always fail diff --git a/open_issues/resource_management_problems.mdwn b/open_issues/resource_management_problems.mdwn index 8f752d61..daf97954 100644 --- a/open_issues/resource_management_problems.mdwn +++ b/open_issues/resource_management_problems.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, +[[!meta copyright="Copyright © 2008, 2009, 2010, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -61,7 +61,8 @@ This is, of course, non-trivial to implement, and also requires changing the SPLICE_F_GIFT flag](http://www.kernel.org/doc/man-pages/online/pages/man2/vmsplice.2.html#DESCRIPTION).) -IRC, freenode, #hurd, 2011-07-31 + +## IRC, freenode, #hurd, 2011-07-31 < braunr> one of the biggest problems on the hurd is that, when a client makes a call, kernel (and other) resources are allocated on behalf of the @@ -75,6 +76,20 @@ IRC, freenode, #hurd, 2011-07-31 attempts) +## IRC, freenode, #hurd, 2013-08-13 + +In context of <https://teythoon.cryptobitch.de/posts/my-worst-week-yet/>. + + <braunr> teythoon: actually, thread migration isn't required for resource + accounting + +[[Mach_migrating_threads]]. + + <teythoon> braunr: but it solves it for free, doesn't it? + <braunr> teythoon: no + <braunr> it's really more complicated than that + + # Further Examples * [[hurd/critique]] @@ -83,4 +98,34 @@ IRC, freenode, #hurd, 2011-07-31 * [[translators_set_up_by_untrusted_users]], and [[pagers]] - * [[configure max command line length]] + * [[configure_max_command_line_length]] + + +## [[hurd/translator/exec]] server + +### IRC, freenode, #hurd, 2013-08-05 + + <teythoon> unzipping stuff in the exec server enables a dos on filesystem + translators + <teythoon> https://teythoon.cryptobitch.de/gsoc/heap/hello-1g.bz2 is + /hurd/hello padded with a gig of zeros, compressed with bzip2 + <teythoon> if set as an passive translator, it stalls other requests to the + filesystem, at least it does if ext2fs is used + <braunr> teythoon: ? + <braunr> teythoon: what's the dos here ? + <teythoon> I can prevent you from doing anything with the root filesystem + <teythoon> I'm kind of surprised myself, maybe a lock is held during the + exec of the translator? + <teythoon> the filesystem the hello-1g.bz2 translator is bound to is + affected + <braunr> teythoon: i don't understand + <braunr> have you tried starting something from another file system ? + <braunr> the lock may simply be in the exec server itself + <teythoon> no, starting other things works fine + <teythoon> but on the other hand, a find / is stalled + <braunr> :/ + <braunr> *sigh* + <teythoon> don't worry + <teythoon> there is a solution :p + <braunr> :) + <teythoon> and it only requires deleting code diff --git a/open_issues/robustness.mdwn b/open_issues/robustness.mdwn index 1f8aa0c6..a6b0dbfb 100644 --- a/open_issues/robustness.mdwn +++ b/open_issues/robustness.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,6 +10,7 @@ License|/fdl]]."]]"""]] [[!tag open_issue_documentation open_issue_hurd]] +[[!toc]] # IRC, freenode, #hurd, 2011-11-18 @@ -32,7 +33,9 @@ License|/fdl]]."]]"""]] <etenil> ah yeah I thought so :) -# IRC, freenode, #hurd, 2011-11-19 +# Reincarnation Server + +## IRC, freenode, #hurd, 2011-11-19 <chromaticwt> will hurd ever have the equivalent of a rs server?, is that even possible with hurd? @@ -127,3 +130,40 @@ License|/fdl]]."]]"""]] <spiderweb> neat, thanks <braunr> actually it's not that old at all <braunr> around 2007 + + +## IRC, freenode, #hurd, 2013-08-26 + + < teythoon> I came across some paper about process reincarnation and + created a little prototype a while back: + < teythoon> http://darnassus.sceen.net/gitweb/teythoon/reincarnation.git/ + < teythoon> and I looked into restarting the exec server in case it + dies. the exec server is an easy target since it has no state of its own + < teythoon> the only problem is that there is no exec server around to + start a new one + < youpi> teythoon: there could be another exec server only used to + (re)start the exec server + < youpi> that other exec server could even be restarted by the normal exec + server + < pinotree> what about a watchdog server? + < teythoon> youpi: yes, I had the same idea, i actually patched /hurd/init + to do that, it's just not yet working + < pinotree> make it watch other servers (exec included), and make exec + watch the watchdog only + < teythoon> pinotree: look at my prototype, there is a watchdog server + < braunr> teythoon: what's the point of reincarnation without persistence ? + < teythoon> braunr: there is no point in reincarnation w/o persistence of + course + < teythoon> my prototype does a limited form of persistence + < teythoon> the point was to see whether I can mitm a translator and + restart it on demand and to gain more insight into the whole translator + mechanism + < braunr> teythoon: ok + < teythoon> braunr: see the readme, it retains state across reincarnations + < braunr> teythoon: how ? + < teythoon> braunr: the server can store a checkpoint using the + reincarnation_checkpoint procedure + < teythoon> + http://darnassus.sceen.net/gitweb/teythoon/reincarnation.git/blame/HEAD:/reincarnation.defshttp://darnassus.sceen.net/gitweb/teythoon/reincarnation.git/blame/HEAD:/reincarnation.defs + < teythoon> uh >,< sorry, pasted twice + < braunr> oh ok diff --git a/open_issues/secure_file_descriptor_handling.mdwn b/open_issues/secure_file_descriptor_handling.mdwn index 45e983a7..f4d1d396 100644 --- a/open_issues/secure_file_descriptor_handling.mdwn +++ b/open_issues/secure_file_descriptor_handling.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -16,7 +17,7 @@ on this, posted patches to [[mailing_lists/libc-alpha]]. This works needs to be resumed and finished. ---- +Add tests from Linux kernel commit messages for `t/dup3` et al. In <http://lwn.net/Articles/417421/> an interesting point is made: *you [may] want some [[unix/file_descriptor]] to still be open if 'exec' fails, but you diff --git a/open_issues/systemd.mdwn b/open_issues/systemd.mdwn index c23f887f..d00b3d8a 100644 --- a/open_issues/systemd.mdwn +++ b/open_issues/systemd.mdwn @@ -102,6 +102,939 @@ Likely there's also some other porting needed. <braunr> just assume you can't use systemd on anything else than linux +## IRC, OFTC, #debian-hurd, 2013-08-12 + + <azeem> huh, Lennert Poettering just mentioned the Hurd in his systmd talk + <azeem> well, in the context of you IPC in Unix sucks and kdbus + <azeem> s/you/how/ + <pinotree> QED + <pinotree> what did you expect? :) + <azeem> I didn't quite get it, but he seemed to imply the Hurd was a step + in the right direction over Unix + <azeem> (which is obvious, but it wasn't obvious he had that opinion) + + +## IRC, OFTC, #debian-hurd, 2013-08-13 + + <azeem> so cgroups seems to be most prominent thing the systemd people + think the Hurd lacks + <tschwinge> azeem: In 2010, I came to the same conclusion, + <http://www.gnu.org/software/hurd/open_issues/systemd.html>. ;-) + <azeem> heh + <tschwinge> I don't think of any show-stopper for implementing that -- just + someone to do it. + <youpi> azeem: which part of cgroups, like being able to kill a cgroup? + <youpi> it shouldn't be very hard to implement what systemd needs + <azeem> probably also the resource allocation etc. + <azeem> the questions are I guess (i) do the cgroups semantics make sense + from our POV and/or do we accept that cgroups is the "standard" now and + (ii) should systemd require concrete implementations or just the concept + in a more abstract sense + <teythoon> being the first non Linux OS that runs systemd would be a nice + showcase of Hurds flexibility + <azeem> maybe upstart is less trouble + <pinotree> azeem: possibly + <azeem> teythoon: can you just include upstart in your GSOC? kthxbye + <pinotree> at least libnih (the library with base utilities and such used + by upstart) required a working file monitor (and the current + implementation kind of exposes a fd) and certain semantics for waitid + <pinotree> libnih/upstart have "just" the issue of being under CLA... + <azeem> pinotree: yeah, true + <azeem> I suggested "startup" as a name for a fork + <pinotree> imho there would be no strict need to fork + <teythoon> azeem: but upstart is a lot less interesting. last time I used + it it wasn't even possible to disable services in a clean way + <pochu> pinotree: is that still so now that Scott works for google? + <pinotree> pochu: yeah, since it's a Canonical CLA, not rally something + tied to a person + <pinotree> (iirc) + <pochu> sure, but scott is the maintainer... + <pochu> shrug + <azeem> nah, scott left upstart + <azeem> AFAIK + <azeem> at least James Hunt gave a talk earlier with Steve Langasek and + introduced himself as the upstart maintainer + <azeem> also I heard in the hallway track that the upstart people are + somewhat interested in BSD/Hurd support as they see it as a selling point + against systemd + <pinotree> pochu: it's just like FSF CLA for GNU projects: even if the + maintainers/contributors change altogether, copyright assignment is still + FSF + <azeem> but their accents were kinda annoying/hard to follow so I didn't + follow their talk closesly to see whether they brought it up + <azeem> pinotree: well, it's not + <pochu> azeem: looking at https://code.launchpad.net/libnih, I'm not sure + libnih has a maintainer anymore... + <azeem> pinotree: first off, you're not signing over the copyright with + their CLA, just giving them the right to relicense + <azeem> pinotree: but more importantaly, the FSF announced in a legally + binding way that they will not take things non-free + <azeem> anyway, I'll talk to the upstart guys about libnih + + +## IRC, OFTC, #debian-hurd, 2013-08-15 + + <azeem> btw, I talked to vorlon about upstart and the Hurd + <azeem> so the situation with libnih is that it is basically + feature-complete, but still maintained by Scott + <azeem> upstart is leveraging it heavily + <azeem> and Scott was (back in the days) against patches for porting + <azeem> for upstart proper, Steve said he would happily take porting + patches + + +## IRC, freenode, #hurd, 2013-08-26 + + < youpi> teythoon: I tend to agree with mbanck + < youpi> although another thing worth considering would be adding something + similar to control groups + < youpi> AIUI, it's one of the features that systemd really requires + < braunr> uhg, cgroups already + < braunr> youpi: where is that discussion ? + < youpi> it was a private mail + < braunr> oh ok + < teythoon> right, so about upstart + < teythoon> to be blunt, I do not like upstart, though my experience with + it is limited and outdated + < braunr> that was quick :) + < braunr> i assume this follows your private discussion with youpi and + mbank ? + < teythoon> I used it on a like three years old ubuntu and back then it + couldn't do stufft hat even sysvinit could do + < teythoon> there was not much discussion, mbank suggested that I could + work on upstart + < teythoon> b/c it might be easier to support than systemd + < teythoon> which might be very well true, then again what's the benefit of + having upstart? I'm really curious, I should perhaps read up on its + features + < pinotree> event-based, etc + < youpi> it is also about avoiding being pushed out just because we don't + support it? + < teythoon> yes, but otoh systemd can do amazing things, the featurelist of + upstart reads rather mondane in comparison + < youpi> I don't really have an opinion over either, apart from portability + of the code + < braunr> teythoon: the system requirements for systemd would take much + time to implement in comparison to what we already have + < braunr> i still have maksym's work on last year gsoc on my list + < braunr> waiting to push in the various libpager related patches first + < teythoon> so you guys think it's worthwile to port upstart? + < braunr> no idea + < braunr> teythoon: on another subject + < azeem_> teythoon: I like systemd more, but the hallway track at Debconf + seemed to imply most people like Upstart better except for the CLA + < azeem_> which I totally forgot to address + < youpi> CLA ? + < azeem_> contributor license agreement + < braunr> since you've now done very good progress, is your work available + in the form of ready-to-test debian packages ? + < teythoon> braunr: it is + < teythoon> braunr: http://teythoon.cryptobitch.de/gsoc/heap/debian/ + < braunr> i remember urls in some of your mails + < braunr> ah thanks + < braunr> "cryptobitch" hum :) + < azeem_> in any case, everbody assumed either Upstart or Systemd are way + ahead of systemvinit + < braunr> sysvinit is really ancient :) + < azeem_> apart from the non-event-driven fundamental issue, a lot of + people critized that the failure rate at writing correct init-scripts + appears to be too high + < azeem_> one of the questions brought up was whether it makes sense to + continue to ship/support systemvinit once a switch is made to + systemd/upstart for the Linux archs + < azeem_> systemvinit scripts might bitrot + < azeem_> but anyway, I don't see a switch happen anytime soon + < teythoon> well, did upstart gain the capability of disabling a service + yet? + < azeem_> teythoon: no idea, but apparently: + http://askubuntu.com/questions/19320/recommended-way-to-enable-disable-services/20347#20347 + < teythoon> azeem_: then there is hope yet ;) + < azeem_> the main selling point of Upstart is that it shipped in several + LTS releases and is proven technology (and honestly, I don't read a lot + of complaints online about it) + < azeem_> (I don't agree that SystemD is unproven, but that is what the + Upstart guys implied) + < teythoon> am I the only one that thinks that upstart is rather + unimpressive? + * azeem_ doesn't have an opinion on it + < azeem> teythoon: + http://penta.debconf.org/dc13_schedule/events/1027.en.html has slides and + the video + < azeem> teythoon: eh, appears the link to the slides is broken, but they + are here: + http://people.canonical.com/~jhunt/presentations/debconf13/upstart-debconf-2013.pdf + < braunr> teythoon: actually, from the presentation, i'd tend to like + upstart + < braunr> dependency, parallelism and even runlevel compatibility flows + naturally from the event based model + < braunr> sysv compatibility is a great feature + < braunr> it does look simple + < braunr> i admit it's "unimpressive" but do we want an overkill init + system ? + < braunr> teythoon: what makes you not like it ? + < azeem> Lennart critized that upstart doesn't generate events, just + listens to them + < azeem> (which is a feature, not a bug to some) + < braunr> azeem: ah yes, that could be a lack + < azeem> braunr: http://penta.debconf.org/dc13_schedule/events/983.en.html + was the corresponding SystemD talk by Lennart, though he hasn't posted + slides yet I think + < teythoon> braunr: well, last time I used it it was impossible to cleanly + disable a service + < teythoon> also ubuntu makes such big claims about software they develop, + and when you read up on them it turns out that most of the advertised + functionality will be implemented in the near future + < teythoon> then they ship software as early as possible only to say later + that is has proven itself for so many years + < teythoon> and tbh I hate to be the one that helped port upstart to hurd + (and maybe kfreebsd as a byproduct) and later debian choses upstart over + systemd b/c it is available for all debian kernels + < kilobug> teythoon: ubuntu has a tendency to ship software too early when + it's not fully mature/stable, but that doesn't say anything about the + software itself + < pinotree> teythoon: note the same is sometimes done on fedora for young + technologies (eg systemd) + < azeem> teythoon: heh, fair enough + < p2-mate> braunr: I would prefer if my init doesn't use ptrace :P + < teythoon> p2-mate: does upstart use ptrace? + < p2-mate> teythoon: yes + < teythoon> well, then I guess there won't be an upstart for Hurd for some + time, no? + < kilobug> p2-mate: why does it use ptrace for ? + < p2-mate> kilobug: to find out if a daemon forked + < kilobug> hum I see + < azeem> p2-mate: the question is whether there's a Hurdish way to + accomplish the same + < p2-mate> + http://bazaar.launchpad.net/~upstart-devel/upstart/trunk/view/head:/init/job_process.c + < p2-mate> see job_process_trace_new :) + < kilobug> azeem: it doesn't seem too complicated to me to have a way to + get proc notify upstart of forks + < p2-mate> azeem: that's a good question. there is a linuxish way to do + that using cgroups + < azeem> right, there's a blueprint suggesting cgroups for Upstart here: + https://blueprints.launchpad.net/ubuntu/+spec/foundations-q-upstart-overcome-ptrace-limitations + < teythoon> yes, someone should create a init system that uses cgroups for + tracking child processes >,< + < teythoon> kilobug: not sure it is that easy. who enforces that proc_child + is used for a new process? isn't it possible to just create a new mach + task that has no ties to the parent process? + < teythoon> azeem: what do you mean by "upstart does not generate events"? + there are "emits X" lines in upstart service descrpitions, surely that + generates event X? + < azeem> I think the critique is that this (and those upstart-foo-bridges) + are bolted on, while SystemD just takes over your systems and "knows" + about them first-hand + < azeem> but as I said, I'm not the expert on this + < teythoon> uh, in order to install upstart one has to remove sysvinit + ("yes i am sure...") and it fails to bring up the network on booting the + machine + < teythoon> also, both systemd and upstart depend on dbus, so no cookie for + us unless that is fixed first, right? + < pinotree> true + < teythoon> well, what do you want me to do for the next four weeks? + < youpi> ideally you could make both upstart and systemd work on hurd-i"86 + < pinotree> both in 4 weeks? + < youpi> so hurd-i386 doesn't become the nasty guy that makes people tend + for one or the other + < youpi> I said "ideally" + < youpi> I don't really have any idea how much work is required by either + of the two + < youpi> I'd tend to think the important thing to implement is something + similar to control groups, so both upstart (which is supposed to use them + someday) and systemd can be happy about it + < teythoon> looks like upstarts functionality depending on ptrace is not + required, but can be enabled on a per service base + < teythoon> so a upstart port that just lacks this might be possible + < teythoon> youpi: the main feature of cgroups is that a process cannot + escape its group, no? i'm not sure how this could be implemented atop of + mach in a secure and robust way + < teythoon> b/c any process can just create mach tasks + < youpi> maybe we need to add a feature in mach itself, yes + < teythoon> ok, implementing cgroups sounds fun, I could do that + < youpi> azeem: are you ok with that direction? + < azeem> well, in general yes; however, AIUI, cgroups is being redesigned + upstream, no? + < youpi> that's why I said "something like cgroups" + < azeem> ah, ok + < youpi> we can do something simple enough to avoid design quesetions, and + that would still be enough for upstart & systemd + < azeem> + (http://www.linux.com/news/featured-blogs/200-libby-clark/733595-all-about-the-linux-kernel-cgroups-redesign) + btw + < braunr> p2-mate: upstart uses ptrace ? + < p2-mate> yes + < youpi> teythoon: and making a real survey of what needs to be fixed for + upstart & systemd + < p2-mate> see my link posted earlier + < braunr> ah already answered + < braunr> grmbl + < braunr> it's a simple alternative to cgroups though + < braunr> teythoon: dbus isn't a proble + < braunr> problem + < braunr> it's not that hard to fix + < youpi> well, it hasn't been fixed for a long time now :) + < braunr> we're being slow, that's all + < braunr> and interested by other things + < gg0> 12:58 < teythoon> btw, who is this heroxbd fellow and why has he + suddenly taken interest in so many debian gsoc projects? + < gg0> http://lists.debian.org/debian-hurd/2013/05/msg00133.html + < gg0> i notice nobody mentioned openrc + < pinotree> he's the debian student working on integrating openrc + < gg0> pinotree: no, the student is Bill Wang, Benda as he says is a + co-mentor + https://wiki.debian.org/SummerOfCode2013/Projects#OpenRC_init_system_in_Debian + < pinotree> whatever, it's still the openrc gsoc + < azeem> well, they wanted to look at it WRT the Hurd, did they follow-up + on this? + < gg0> btw wouldn't having openrc on hurd be interesting too? + < pinotree> imho not really + < gg0> no idea whether Bill is also trying to figure out what to do, + probably not + < azeem> somebody could ping that thread you mentioned above to see whether + they looked at the Hurd and/or need help/advice + < gg0> azeem: yeah somebody who could provide such help/advice. like.. you? + for instance + * gg0 can just paste urls + < azeem> they should just follow-up on-list + + +## IRC, freenode, #hurd, 2013-08-28 + + <teythoon> anyone knows a user of cgroups that is not systemd? so far I + found libcg, that looks like a promising first target to port first, + though not surprisingly it is also somewhat linux specific + <taylanub> teythoon: OpenRC optionally uses cgroups IIRC. + <taylanub> Not mandatory because unlike systemd it actually tries (at all) + to be portable. + + +## IRC, freenode, #hurd, 2013-09-02 + + <teythoon> braunr: I plan to patch gnumach so that the mach tasks keep a + reference to the task that created them and to make that information + available + <teythoon> braunr: is such a change acceptable? + <braunr> teythoon: what for ? + <teythoon> braunr: well, the parent relation is currently only implemented + in the Hurd, but w/o this information tracked by the kernel I don't see + how I can prevent malicious/misbehaving applications to break out of + cgroups + <teythoon> also I think this will enable us to fix the issue with tracking + which tasks belong to which subhurd in the long term + <braunr> ah cgroups + <braunr> yes cgroups should partly be implemented in the kernel ... + <braunr> teythoon: that doesn't surprise me + <braunr> i mean, i think it's ok + <braunr> the kernel should implement tasks and threads as closely as the + hurd (or a unix-like system) needs it + <teythoon> braunr: ok, cool + <teythoon> braunr: I made some rather small and straight forward changes to + gnumach, but it isn't doing what I thought it would do :/ + <teythoon> braunr: http://paste.debian.net/33717/ + <braunr> you added a field to task_basic_info + <braunr> thereby breaking the ABI + <teythoon> braunr: my small test program says: my task port is 1(pid 13) + created by task -527895648; my parent task is 31(pid 1) + <teythoon> braunr: no, it is not. I appended a field and these structures + are designed to be extendable + <braunr> hm + <braunr> ok + <braunr> although i'm not so sure + <braunr> there are macros defining the info size, depending on what you ask + <braunr> you may as well get garbage + <braunr> have you checked that ? + <teythoon> i initialized my struct to zero before calling mach + <braunr> teythoon: can you put some hardcoded value, just to make sure data + is correctly exported ? + <teythoon> braunr: right, good idea + <teythoon> braunr: my task port is 1(pid 13) created by task 3; my parent + task is 31(pid 1) -- so yes, hardcoding 3 works + <braunr> ok + <teythoon> braunr: also I gathered evidence that the convert_task_to_port + thing works, b/c first I did not have the task_reference call just before + that so the reference count was lowered (convert... consumes a reference) + and the parent task was destroyed + <teythoon> braunr: I must admit I'm a little lost. I tried to return a + reference to task rather than task->parent_task, but that didn't work + either + <teythoon> braunr: I feel like I'm missing something here + <teythoon> maybe I should get aquainted with the kernel debugger + <teythoon> err, the kernel debugger is not accepting any symbol names, even + though the binary is not stripped o_O + <teythoon> err, neither the kdb nor gdb attached to qemu translates + addresses to symbols, gdb at least translates symbols to addresses when + setting break points + <teythoon> how did anyone ever debug a kernel problem under these + conditions? + <braunr> teythoon: i'll have a look at it when i have some time + + +## IRC, freenode, #hurd, 2013-09-03 + + <teythoon> :/ I believe the startup_notify interface is ill designed... an + translator can defer the system shutdown indefinitely + <braunr> it can + <teythoon> that's bad + <braunr> yes + <braunr> the hurd has a general tendency to trust its "no mutual trust + required" principle + <braunr> to rely on it a bit too much + <teythoon> well, at least it's a privileged operation to request this kind + of notification, no? + <braunr> why ? + <braunr> teythoon: it normally is used mostly by privileged servers + <braunr> but i don't think there is any check on the recipient + <teythoon> braunr: b/c getting the port to /hurd/init is done via + proc_getmsgport + <braunr> teythoon: ? + <teythoon> braunr: well, in order to get the notifications one needs the + msgport of /hurd/init and getting that requires root privileges + <braunr> teythoon: oh ok then + <braunr> teythoon: what's bad with it then ? + <teythoon> braunr: even if those translators are somewhat trusted, they can + (and do) contain bugs and stall the shutdown + <teythoon> I think this even happened to me once, I think it was the pfinet + translator + <braunr> teythoon: how do you want it to behave instead ? + <teythoon> braunr: well, /hurd/init notifies the processes sequentially, + that seems suboptimal, better to send async notifications to all of them + and then to collect all the answers + <teythoon> braunr: if one fails to answer within a rather large time frame + (say 5 minutes) shutdown anyway + <braunr> i agree with async notifications but + <braunr> i don't agree with the timeout + <teythoon> for reference, a (voluntary) timeout of 1 minute is hardcoded in + /hurd/init + <braunr> the timeout should be a parameter + <braunr> it's common on large machines to have looong shutdown delays + <teythoon> of the notification? + <braunr> the answer means "ok i'm done you can shutdown" + <braunr> well this can take long + <braunr> most often, administrators simply prefer to trust their program is + ok and won't take longer than it needs to, even if it's long + <teythoon> and not answering at all causes the shutdown / reboot to fail + making the system hang + <braunr> i know + <teythoon> in a state where it is not easily reached if you do not have + access to it + <braunr> but since it only concerns essential servers, it should befine + <braunr> essential servers are expected to behave well + <teythoon> it concerns servers that have requested a shutdown notification + <braunr> ok so no essential but system servers + <teythoon> essential servers are only exec, proc, / + <teythoon> yes + <braunr> the same applies + <pinotree> init and auth too, no? + <teythoon> yes + <braunr> you expect root not to hang himself + <teythoon> I do expect all software to contain bugs + <braunr> yes but you also expect them to provide a minimum level of + reliability + <braunr> otherwise you can just throw it all away + <teythoon> no, not really + <braunr> well + <teythoon> I know, that's my dilemma basically ;) + <braunr> if you don't trust your file system, you make frequent backups + <braunr> if you don't trust your shutdown code, you're ready to pull the + plug manually + <braunr> (or set a watchdog or whatever) + <braunr> what i mean is + <braunr> we should NEVER interfere with a program that is actually doing + its job just because it seems too long + <braunr> timeouts are almost never the best solution + <braunr> they're used only when necessary + <braunr> e.g. across networks + <braunr> it's much much much worse to interrupt a proper shutdown process + because it "seems too long" than just trust it behaves well 99999%%%% of + the time + <braunr> in particular because this case deals with proper data flushing, + which is an extremely important use case + <teythoon> it's hard/theoretically impossible to distinguish between taking + long and doing nothing + <braunr> it's impossible + <braunr> agreed + <braunr> => trust + <braunr> if you don't trust, you run real time stuff + <braunr> and you don't flush data on disk + <teythoon> ^^ + <braunr> (which makes a lot of computer uses impossible as well) + <teythoon> there are only 2 people I trust, and the other one is not + /hurd/pfinet + <braunr> if this shutdown procedure is confined to the TCB, it's fine to + trust it goes well + <teythoon> tcb? + <braunr> trusted computing base + <braunr> http://en.wikipedia.org/wiki/Trusted_computing_base + * teythoon shudders + <teythoon> "trust" is used way to much these days + <teythoon> and I do not like the linux 2.0 ip stack to be part of our TCB + <braunr> basically, on a multiserver system like the hurd, the tcb is every + server on the path to getting a service done from a client + <braunr> then make it not request to be notified + <braunr> or make two classes of notifications + <braunr> because unprivileged file systems should be notified too + <teythoon> indeed + <teythoon> by the way, we should have a hurdish libnotify or something for + this kind of notifications + <braunr> but in any case, it should really be policy + <braunr> we should ... :) + <teythoon> ^^ + + +## IRC, freenode, #hurd, 2013-09-04 + + <teythoon> braunr: btw, I now believe that no server that requested + shutdown notifications can stall the shutdown for more than 1 minute + *unless* its message queue is full + <teythoon> so any fs should better sync within that timeframe + <braunr> where is this 1 min defined ? + <teythoon> init/init.c search for 60000 + <braunr> ew + <teythoon> did I just find the fs corruption bug everyone was looking for? + <braunr> no + <braunr> what corruption bug ? + <teythoon> not sure, I thought there was still some issues left with + unclean filesystems every now and then + <teythoon> *causing + <braunr> yes but we know the reasons + <teythoon> ah + <braunr> involving some of the funniest names i've seen in computer + terminology : + <braunr> writeback causing "message floods", which in turn create "thread + storms" in the servers receiving them + <teythoon> ^^ it's usually the other way around, storms causing floods >,, + <braunr> teythoon: :) + <braunr> let's say it's a bottom-up approach + <teythoon> then the fix is easy, compile mach with -DMIGRATING_THREADS :) + <braunr> teythoon: what ? + <teythoon> well, that would solve the flood/storm issue, no? + <braunr> no + <braunr> the real solution is proper throttling + <braunr> which can stem from synchronous rpc (which is the real property we + want from migrating threads) + <braunr> but the mach writeback interface is async + <braunr> :p + + +## IRC, freenode, #hurd, 2013-09-05 + + <braunr> teythoon: oh right, forgot about your port issue + <teythoon> don't worry, I figured by now that this must be a pointer + <teythoon> and I'm probably missing some magic that transforms this into a + name for the receiver + <teythoon> (though I "found" this function by looking at the mig + transformation for ports) + <braunr> i was wondering why you called the convert function manually + <braunr> instead of simply returning the task + <braunr> and let mig do the magic + <teythoon> b/c then I would have to add another ipc call, no? + <braunr> let me see the basic info call again + <braunr> my problem with this code is that it doesn't take into account the + ipc space of the current task + <braunr> which means you probably really return the ipc port + <braunr> the internal kernel address of the struct + <braunr> indeed, ipc_port_t convert_task_to_port(task) + <braunr> i'd personally make a new rpc instead of adding it to basic info + <braunr> basic info doesn't create rights + <braunr> what you want to achieve does + <braunr> you may want to make it a special port + <braunr> i.e. a port created at task creation time + <teythoon> y? + <braunr> it also means you need to handle task destruction and reparent + <teythoon> yes, I thought about that + <braunr> see + http://www.gnu.org/software/hurd/gnumach-doc/Task-Special-Ports.html#Task-Special-Ports + <braunr> for now you may simply turn the right into a dead name when the + parent dies + <braunr> although adding a call and letting mig do it is simpler + <braunr> mig handles reference counting, users just need to task_deallocate + once done + <teythoon> o_O mig does reference counting of port rights? + <braunr> mig/mach_msg + <teythoon> is there anything it *doesn't* do? + <braunr> i told you, it's a very complicated messaging interface + <braunr> coffee ? + <braunr> fast ? + <teythoon> ^^ + <braunr> mig knows about copy_send/move_send/etc... + <braunr> so even if it doesn't do reference counting explicitely, it does + take care of that + <teythoon> true + <braunr> in addition, the magic conversions are intended to both translate + names into actual structs, and add a temporary reference at the same time + <braunr> teythoon: everything clear now ? :) + <teythoon> braunr: no, especially not why you suggested to create a special + port. but this will have to wait for tomorrow ;) + + +## IRC, OFTC, #debian-hurd, 2013-09-06 + + <vorlon> teythoon: hi there + <vorlon> so I've been following your blog entries about cgroups on + hurd... very impressive :) + <vorlon> but I think there's a misunderstanding about upstart and + cgroups... your "conjecture" in + https://teythoon.cryptobitch.de/posts/what-will-i-do-next-cgroupfs-o/ is + incorrect + <vorlon> cgroups does not give us the interfaces that upstart uses to + define service readiness; adding support for cgroups is interesting to + upstart for purposes of resource partitioning, but there's no way to + replace ptrace with cgroups for what we're doing + <teythoon> vorlon: hi and thanks for the fish :) + <teythoon> vorlon: what is it exactly that upstart is doing with ptrace + then? + <teythoon> .,oO( your nick makes me suspicious for some reason... ;) + <teythoon> service readiness, what does that mean exactly? + <vorlon> teythoon: so upstart uses ptrace primarily for determining service + readiness. The idea is that traditionally, you know an init script is + "done" when it returns control to the parent process, which happens when + the service process has backgrounded/daemonized; this happens when the + parent process exits + <vorlon> in practice, however, many daemons do this badly + <vorlon> so upstart tries to compensate, by not just detecting that the + parent process has exited, but that the subprocess has exited + <vorlon> (for the case where the upstart job declares 'expect daemon') + <vorlon> cgroups, TTBOMK, will let you ask "what processes are part of this + group" and possibly even "what process is the leader for this group", but + doesn't really give you a way to detect "the lead process for this group + has changed twice" + <vorlon> now, it's *better* in an upstart/systemd world for services to + *not* daemonize and instead stay running in the foreground, but then + there's the question of how you know the service is "ready" before moving + on to starting other services that depend on it + <vorlon> systemd's answer to this is socket-based activation, which we + don't really endorse for upstart for a variety of reasons + <teythoon> hm, okay + <teythoon> so upstart does this only if expect daemon is declared in the + service description? + <vorlon> (in part because I've seen security issues when playing with the + systemd implementation on Fedora, which Lennart assures me are + corner-cases specific to cups, but I haven't had a chance to test yet + whether he's right) + <teythoon> and it is not used to track children, but only to observe the + daemonizing process? + <vorlon> yes + <teythoon> and it then detaches from the processes? + <vorlon> yes + <vorlon> once it knows the service is "ready", upstart doesn't care about + tracking it; it'll receive SIGCHLD when the lead process dies, and that's + all it needs to know + <teythoon> ok, so I misunderstood the purpose of the ptracing, thanks for + clarifying this + <vorlon> my pleasure :) + <vorlon> I realize that doesn't really help with the problem of hurd not + having ptrace + <teythoon> no, but thanks anyway + <vorlon> fwiw, the alternative upstart recommends for detecting service + readiness is for the process to raise SIGSTOP when it's ready + <vorlon> doesn't require ptracing, doesn't require socket-based activation + definitions; does require the service to run in a different mode than + usual where it will raise the signal at the correct time + <teythoon> right, but that requires patching it, same as the socket + activation stuff of systemd + <vorlon> (this is upstart's 'expect stop') + <vorlon> yes + <vorlon> though at DebConf, there were some evil ideas floating around + about doing this with an LD_PRELOAD or similar ;) + <vorlon> (overriding 'daemonize') + <vorlon> er, 'daemon()' + <teythoon> ^^ + <vorlon> and hey, what's suspicious about my /nick? vorlons are always + trustworthy + <vorlon> ;) + <teythoon> sure they are + <teythoon> but could this functionality be reasonably #ifdef'ed out for a + proof of concept port? + <vorlon> hmm, you would need to implement some kind of replacement... if + you added cgroups support to upstart as an alternative + <vorlon> that could work + <vorlon> i.e., you would need upstart to know when the service has exited; + if you aren't using ptrace, you don't know the "lead pid" to watch for, + so you need some other mechanism --> cgroups + <vorlon> and even then, what do you do for a service like openssh, which + explicitly wants to leave child processes behind when it restarts? + <teythoon> right... + <vorlon> oh, I was hoping you knew the answer to this question ;) Since + AFAICS, openssh has no native support for cgroups + <teythoon> >,< I don't, but I'll think about what you've said... gotta go, + catch what's left of the summer ;) + <teythoon> fwiw I consider fork/exec/the whole daemonizing stuff fubar... + <teythoon> see you around :) + <vorlon> later :) + + +## IRC, OFTC, #debian-hurd, 2013-09-07 + + <teythoon> vorlon: I thought about upstarts use of ptrace for observing the + daemonizing process and getting hold of the child + <teythoon> vorlon: what if cgroup(f)s would guarantee that the order of + processes listed in x/tasks is the same they were added in? + <teythoon> vorlon: that way, the first process in the list would be the + daemonized child after the original process died, no? + <vorlon> teythoon: that doesn't tell you how many times the "lead" process + has changed, however + <vorlon> you need synchronous notifications of the forks in order to know + that, which currently we only get via ptrace + + +## IRC, OFTC, #debian-hurd, 2013-09-08 + + <teythoon> vorlon: ok, but why do the notifications have to be synchronous? + does that imply that the processes need to be stopped until upstart does + something? + <vorlon> teythoon: well, s/synchronous/reliable/ + <vorlon> you're right that it doesn't need to be synchronous; but it can't + just be upstart polling the status of the cgroup + <vorlon> because processes may have come and gone in the meantime + <teythoon> vorlon: ok, cool, b/c the notifications of process changes I'm + hoping to introduce into the proc server for my cgroupfs do carry exactly + this kind of information + <vorlon> cool + <vorlon> are you discussing an API for this with the Linux cgroups + maintainers? + <teythoon> otoh it would be somewhat "interesting" to get upstart to use + this b/c of the way the mach message handling is usually implemented + <vorlon> :) + <teythoon> no, I meant in order for me to be able to implement cgroupfs I + had to create these kind of notifications, it's not an addition to the + cgroups api + <teythoon> is upstart multithreaded? + <vorlon> no + <vorlon> threads are evil ;) + <teythoon> ^^ I mostly agree + <vorlon> it uses a very nice event loop, leveraging signalfd among other + things + <teythoon> uh oh, signalfd sounds rather Linuxish + <pinotree> it is + <vorlon> I think xnox mentioned when he was investigating it that kfreebsd + now also supports it + <vorlon> but yeah, AFAIK it's not POSIX + <pinotree> it isn't, yes + <vorlon> but it darn well should be + <vorlon> :) + <vorlon> it's the best improvement to signal handling in a long time + <teythoon> systemd also uses signalfd + <teythoon> umm, it seems I was wrong about Hurd not having ptrace, the wiki + suggests that we do have it + <pinotree> FSVO "have" + <teythoon> ^^ + <xnox> vorlon: teythoon: so ok kFreeBSD/FreeBSD ideally I'd be using + EVFILT_PROC from kevent which allows to receive events & track: exit, + fork, exec, track (follow across fork) + <xnox> upstart also uses waitid() + <xnox> so a ptrace/waitid should be sufficient to track processes, if Hurd + has them. + + +## IRC, freenode, #hurd, 2013-09-09 + + <youpi> teythoon: yes, the shutdown notifications do stall the process + <youpi> but no more than a minute, or so + <youpi> teythoon: btw, did you end up understanding the odd thing in + fshelp_start_translator_long? + <youpi> I haven't had the time to have a look + <teythoon> youpi: what odd thing? the thing about being implemented by hand + instead of using the mig stub? + <youpi> the thing about the port being passed twice + <youpi> XXX this looks wrong to me, please have a look + <youpi> in the mach_port_request_notification call + <teythoon> ah, that was alright, yes + <youpi> ok + <youpi> so I can drop it from my TODO :) + <teythoon> this is done on the control port so that a translator is + notified if the "parent" translator dies + <teythoon> was that in fshelp_start_translator_long though? I thought that + was somewhere else + <youpi> that's what the patch file says + <youpi> +++ b/libfshelp/start-translator-long.c + <youpi> @@ -293,6 +293,7 @@ fshelp_start_translator_long (fshelp_open_fn_t + underlying_open_fn, + <youpi> + /* XXX this looks wrong to me, bootstrap is used twice as + argument... */ + <youpi> bootstrap, + MACH_NOTIFY_NO_SENDERS, 0, + <teythoon> right + <teythoon> I remember that when I got a better grip of the idea of + notifications I figured that this was indeed okay + <teythoon> I'll have a quick look though + <youpi> ok + <teythoon> ah, I remember, this notifies the parent translator if the child + dies, right + <teythoon> and it is a NO_SENDERS notification, so it is perfectly valid to + use the same port twice, as we only hold a receive right + + +## IRC, freenode, #hurd, 2013-09-10 + + <teythoon> braunr: are pthreads mapped 1:1 to mach threads? + <braunr> teythoon: yes + <teythoon> I'm reading the Linux cgroups "documentation" and it talks about + tasks (Linux threads) and thread group IDs (Linux processes) and I'm + wondering how to map this accurately onto Hurd concepts... + <teythoon> apparently on Linux there are PIDs/TIDs that can be used more or + less interchangeably from userspace applications + <teythoon> the Linux kernel however knows only PIDs, and each thread has + its own, and those threads belonging to the same (userspace) PID have the + same thread group id + <teythoon> aiui on Mach threads belong to a Mach task, and there is no + global unique identifier exposed for threads, right? + <teythoon> braunr: ^ + <tschwinge> teythoon: There is its thread port, which in combination with + its task port should make it unique? (I might be missing context.) + <tschwinge> Eh, no. The task port's name will only locally be unique. + * tschwinge confused himself. + <teythoon> tschwinge, braunr: well, the proc server could of course create + TIDs for threads the same way it creates PIDs for tasks, but that should + probably wait until this is really needed + <teythoon> for the most part, the tasks and cgroup.procs files contain the + same information on Linux, and not differentiating between the two just + means that cgroupfs is not able to put threads into cgroups, just + processes + <teythoon> that might be enough for now + + +## IRC, freenode, #hurd, 2013-09-11 + + <teythoon> ugh, some of the half-backed Linux interfaces will be a real + pain in the ass to support + <teythoon> they do stuff like write(2)ing file descriptors encoded as + decimal numbers for notifications :-/ + <braunr> teythoon: for cgroup ? + <teythoon> braunr: yes, they have this eventfd based notification mechanism + <teythoon> braunr: but I fear that this is a more general problem + <braunr> do we need eventfd ? + <teythoon> I mean passing FDs around is okay, we can do this just fine with + ports too, but encoding numbers as an ascii string and passing that + around is just not a nice interface + <braunr> so what ? + <teythoon> it's not a designed interface, it's one people came up with b/c + it was easy to implement + <braunr> if it's meant for compatibility, that's ok + <teythoon> how would you implement this then? as a special case in the + write(2) implementation in the libc? that sounds horrible but I do hardly + see another way + <teythoon> ok, some more context: the cgroup documentation says + <teythoon> write "<event_fd> <control_fd> <args>" to cgroup.event_control. + <teythoon> where event_fd is the eventfd the notification should be sent to + <pinotree> theorically they could have used sendmsg + a custom payload + <teythoon> control_fd is an fd to the pseudo file one wants notifications + for + <teythoon> yes, they could have, that would have been nicer to implement + <teythoon> but this... + + +## IRC, freenode, #hurd, 2013-09-12 + + <teythoon> ugh, gnumachs build system drives me crazy %-/ + <pinotree> oh there's worse than that + <teythoon> I added a new .defs file, did as Makerules.mig.am told me to do, + but it still does not create the stubs I need + <braunr> teythoon: gnumach doesn't + <braunr> teythoon: glibc does + <braunr> well, gnumach only creates the stubs it needs + <braunr> teythoon: you should perhaps simply use gnumach.defs + <teythoon> braunr: sure it does, e.g. vm/memory_object_default.user.c + <braunr> teythoon: what are you trying to add ? + <teythoon> braunr: I was trying to add a notification mechanism for new + tasks + <teythoon> b/c now the proc server has to query all task ports to discover + newly created tasks, this seems wasteful + <teythoon> also if the proc server could be notified on task creation, the + parent task is still around, so the notification can carry a reference to + it + <teythoon> that way gnumach wouldn't have to track the relationship, which + would create all kind of interesting questions, like whether tasks would + have to be reparented if the parent dies + <braunr> teythoon: notifications aren't that simple either + <teythoon> y not? + <braunr> 1/ who is permitted to receive them + <braunr> 2/ should we contain them to hurd systems ? (e.g. should a subhurd + receive notifications concerning tasks in other hurd systems ?) + <teythoon> that's easy imho. 1/ a single process that has a host_priv + handle is able to register for the notifications once + <braunr> what are the requirements so cgroups work as expected concerning + tasks ? + <braunr> teythoon: a single ? + <teythoon> i.e. the first proc server that starts + <braunr> then how will subhurd proc servers work ? + <teythoon> 2/ subhurds get the notifications from the first proc server, + and only those that are "for them" + <braunr> ok + <braunr> i tend to agree + <braunr> this removes the ability to debug the main hurd from a subhurd + <teythoon> this way the subhurds proc server doesn't even have to have the + host_priv porsts + <teythoon> yes, but I see that as a feature tbh + <braunr> me too + <braunr> and we can still debug the subhurd from the main + <teythoon> it still works the other way around, so it's still... + <teythoon> yes + <braunr> what would you include in the notification ? + <teythoon> a reference to the new task (proc needs that anyway) adn one to + the parent task (so proc knows what the parent process is and/or for + which subhurd it is) + <braunr> ok + <braunr> 17:21 < braunr> what are the requirements so cgroups work as + expected concerning tasks ? + <braunr> IOW, why is the parental relation needed ? + <braunr> (i don't know much about the details of cgroup) + <teythoon> well, currently we rely on proc_child to build this relation + <teythoon> but any task can use task_create w/o proc_child + <teythoon> until one claims a newly created task with proc_child, its + parent is pid 1 + <braunr> that's about the hurd + <braunr> i'm rather asking about cgroups + <teythoon> ah + <teythoon> the child process has to end up in the same cgroup as the parent + <braunr> does a cgroup include its own pid namespace ? + <teythoon> not quite sure what you mean, but I'd say no + <teythoon> do you mean pid namespace as in the Linux sense of that phrase? + <teythoon> cgroups group processes(threads) into groups + <teythoon> on Linux, you can then attach controllers to that, so that + e.g. scheduling decisions or resource restrictions can be applied to + groups + <teythoon> braunr: http://paste.debian.net/38950/ + <braunr> teythoon: ok so a cgroup is merely a group of processes supervised + by a controller + <braunr> for resource accounting/scheudling + <braunr> teythoon: where does dev_pager.c do the same ? + <teythoon> braunr: yes. w/o such controllers cgroups can still be used for + subprocess tracking + <teythoon> braunr: well, dev_pager.c uses mig generated stubs from + memory_object_reply.defs + <braunr> ah memory_object_reply ok + <braunr> teythoon: have you tried adding it to EXTRA_DIST ? + <braunr> although i don't expect it will change much + <braunr> teythoon: hum, you're not actually creating client stubs + <braunr> create a kern/task_notify.cli file + <braunr> as it's done with device/memory_object_reply.cli + <braunr> see #define KERNEL_USER 1 + <teythoon> braunr: right, thanks :) + + +## IRC, freenode, #hurd, 2013-09-13 + + <teythoon> hm, my notification system for newly created tasks kinda works + <teythoon> as in I get notified when a new task is created + <teythoon> but the ports for the new task and the parent that are carried + in the notification are both MACH_PORT_DEAD + <teythoon> do I have to add a reference manually before sending it? + <teythoon> that would make sense, the mig magic transformation function for + task_t consumes a reference iirc + <braunr> ah yes + <braunr> that reference counting stuff is some hell + <teythoon> braunr: ah, there's more though, the mig transformations are + only done in the server stub, not in the client, so I still have to + convert_task_to_port myself afaics + <teythoon> awesome, it works :) + <braunr> :) + <teythoon> ugh, the proc_child stuff is embedded deep into libc and signal + handling stuff... + <teythoon> "improving" the child_proc stuff with my shiny new notifications + wrecks havoc on the system + + # Required Interfaces In the thread starting diff --git a/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn b/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn index cf41550d..7159551d 100644 --- a/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn +++ b/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -16,26 +16,37 @@ License|/fdl]]."]]"""]] IRC, unknown channel, unknown date: - <youpi> azeem, marcus: ext2fs.static: thread-cancel.c:55: hurd_thread_cancel: Assertion '! __spin_lock_locked (&ss->critical_section_lock)' failed + <youpi> azeem, marcus: ext2fs.static: thread-cancel.c:55: + hurd_thread_cancel: Assertion '! __spin_lock_locked + (&ss->critical_section_lock)' failed <youpi> I actually don't understand this assertion <youpi> it's just before __spin_lock (&ss->critical_section_lock); <youpi> why should one check that a lock is free before taking it ? <youpi> just the same in hurdexec.c - <youpi> (no, ss is not our own sigstate, so it's not safe to assume no other path can take it) + <youpi> (no, ss is not our own sigstate, so it's not safe to assume no + other path can take it) <youpi> there's another one in sysdeps/mach/hurd/spawni.c <youpi> and jmp-unwind.c - <antrik> youpi: why do you think it's nonsense?... the fact that we take the lock (so we can't be interrupted) doesn't mean we are willing to wait for others to release the lock... maybe the code path should never be reached while others have a lock, or something + <antrik> youpi: why do you think it's nonsense?... the fact that we take + the lock (so we can't be interrupted) doesn't mean we are willing to wait + for others to release the lock... maybe the code path should never be + reached while others have a lock, or something <youpi> then it's useless to take the lock - <youpi> "we take the lock (so we can't be interrupted)": no, it's not _our_ lock here, it's the lock of the thread we want to cancel - <antrik> what exactly is cancelling a thread?... (sorry, I don't really have experience with thread programming) + <youpi> "we take the lock (so we can't be interrupted)": no, it's not _our_ + lock here, it's the lock of the thread we want to cancel + <antrik> what exactly is cancelling a thread?... (sorry, I don't really + have experience with thread programming) <youpi> ~= killing it - <antrik> well, we take the lock so nobody can mess with the thread while we are cancelling it, no?... + <antrik> well, we take the lock so nobody can mess with the thread while we + are cancelling it, no?... <youpi> yes <youpi> that is fine - <youpi> but checking that the lock is free before taking it doesn't make sense + <youpi> but checking that the lock is free before taking it doesn't make + sense <youpi> why nobody should be able to take the lock ? - <youpi> and if nobody is, why do we take it ? (since nobody would be able to take it) - <antrik> well, maybe after taking the lock, we do some action that might result in others trying to take it... + <youpi> and if nobody is, why do we take it ? (since nobody would be able + to take it) + <antrik> well, maybe after taking the lock, we do some action that might + result in others trying to take it... <youpi> nope: look at the code :) <youpi> or maybe the cancel_hook, but I really doubt it - diff --git a/open_issues/time.mdwn b/open_issues/time.mdwn index becb88b0..367db872 100644 --- a/open_issues/time.mdwn +++ b/open_issues/time.mdwn @@ -11,6 +11,11 @@ License|/fdl]]."]]"""]] [[!tag open_issue_porting]] +[[!toc]] + + +# `time` + Neither the `time` executable from the GNU time package work completely correctly, nor does the GNU Bash built-in one. @@ -56,20 +61,20 @@ As above; also here all the running time should be attributed to *user* time. This is probably a [[!taglink open_issue_gnumach]]. -# 2011-09-02 +## 2011-09-02 Might want to revisit this, and take Xen [[!tag open_issue_xen]] into account -- I believe flubber has already been Xenified at that time. -## IRC, freenode, #hurd, 2011-09-02 +### IRC, freenode, #hurd, 2011-09-02 While testing some [[performance/IPC_virtual_copy]] performance issues: <tschwinge> And I can confirm that with dd if=/dev/zero of=/dev/null bs=4k running, a parallel sleep 10 takes about 20 s (on strauss). -# 2013-03-30/31 +## 2013-03-30/31 Investigating time's `configure`, a difference of the output between Linux and Hurd shows: @@ -81,3 +86,754 @@ This causes a different code path in `resuse.c` to be used; such code path does not get a define for `HZ`, which is then defined with a fallback value of 60. [[!debbug 704283]] has been filed with a fix for this no-wait3 case. + + +# `times` + +## guile + +### IRC, freenode, #hurd, 2013-08-21 + + <nalaginrut> does guile2 on hurd fixed? times issue + <teythoon> nalaginrut: does not look good + <teythoon> scheme@(guile-user)> (times) + <teythoon> $1 = #(0 0 0 0 0) + <nalaginrut> well, seems not a fixed version, if there's fixed version + <nalaginrut> since it's not Guile's bug, I can do nothing for it + <teythoon> ah + <nalaginrut> in spite of this, Guile2 works I think + <nalaginrut> all tests passed but 2 fail + <nalaginrut> one of the failure is version shows "UNKNOWN" which is + trivials + <teythoon> well, did you try to fix the times issue in Hurd? + <nalaginrut> I didn't , I have to get more familiar with hurd first + <nalaginrut> I'm playing hurd these days + <teythoon> :) + <nalaginrut> anyway, I think times issue is beyond my ability at present + <nalaginrut> ;-P + <teythoon> times is implemented in the glibc, in sysdeps/mach/hurd/times.c + <teythoon> don't say that before you had a look + <nalaginrut> yes, you're right + <nalaginrut> but I think times has something to do with the kernel time + mechanism, dunno if it's related to the issue + <nalaginrut> how did you get the times.c under hurd? + <nalaginrut> apt-get source glibc? + <teythoon> well, I'd clone git://sourceware.org/git/glibc.git + <teythoon> and yes, the kernel is involved + <teythoon> task_info is used to obtain the actual values + <teythoon> + http://www.gnu.org/software/hurd/gnumach-doc/Task-Information.html + <teythoon> I'd guess that something fails, but the times(2) interface is + not able to communicate the exact failure + <nalaginrut> maybe it's not proper to get src from upstream git? since it's + OK under Linux which uses it too + <nalaginrut> but apt-get source glibc has nothing + <teythoon> so I would copy the times(2) implementation from the libc so + that you can modify it and run it as a standalone program + <teythoon> well, the libc has system dependent stuff, times(2) on Linux is + different from the Hurd version + <teythoon> it has to be + <nalaginrut> alright, I got what you mean ;-) + <teythoon> and the debian libc is built from the eglibc sources, so the + source package is called eglibc iirc + <nalaginrut> ah~I'll try + <teythoon> have you tried to rpctrace your times test program? the small c + snippet you posted the other day? + <nalaginrut> I haven't build all the tools & debug environment on my hurd + ;-( + <teythoon> what tools? + <nalaginrut> well, I don't even have git on it, and I'm installing but + speed is slow, I'm looking for a new mirror + <teythoon> ah well, no need to do all this on the Hurd directly + <teythoon> building the libc takes like ages anyway + <nalaginrut> oops ;-) + <nalaginrut> I'll take your advice to concentrate on times.c only + <teythoon> oh well, it might be difficult after all, not sure though + <teythoon> times sends two task_info messages, once with TASK_BASIC_INFO, + once with TASK_THREAD_TIMES_INFO + <teythoon> here is the relevant rpctrace of your test program: + <teythoon> task131(pid14726)->task_info (1 10) = 0 {0 25 153427968 643072 0 + 0 0 0 1377065590 570000} + <teythoon> task131(pid14726)->task_info (3 4) = 0 {0 0 0 10000} + <teythoon> ok, I don't know enough about that to be honest, but + TASK_THREAD_TIMES_INFO behaves funny + <teythoon> I put a sleep(1) into your test program, and if I rpctrace it, + it behaves differently o_O + * nalaginrut is reading task-information page to get what it could be + <nalaginrut> maybe I have to do the same steps under Linux to find some + clue + <teythoon> no, this is Mach specific, there is no such thing on Linux + <teythoon> on Linux, times(2) is a system call + <teythoon> on Hurd, times is a function implemented in the libc that + behaves roughly the same way + <nalaginrut> OK~so different + <teythoon> look at struct task_basic_info and struct task_thread_times_info + in the task-information page for the meaning of the values in the + rpctrace + <teythoon> yes, very + <braunr> nalaginrut: you may want to try a patch i did but which is still + waiting to be merged in glibc + <nalaginrut> braunr: ah~thanks for did it ;-) + <nalaginrut> can I have the link? + <braunr> i'm getting it + <braunr> teythoon: funny things happen with rpctrace, that's expected + <braunr> keep in mind rpctrace doesn't behave like ptrace at all + <braunr> it acts as a proxy + <braunr> nalaginrut: + http://git.savannah.gnu.org/cgit/hurd/glibc.git/commit/?h=rbraun/getclktck_100_hz&id=90404d6d1aa01f6ce1557841f5a675bb6a30f508 + <braunr> nalaginrut: you need to add it to the debian eglibc patch list, + rebuild the packages, and install the resulting .debs + <braunr> if you have trouble doing it, i'll make packages when i have time + <nalaginrut> braunr: I think your test result is expected? ;-) + <braunr> what test result ? + <nalaginrut> times test under that patch + <braunr> yes + <braunr> but i have no idea if it will work + <braunr> my patch fixes a mismatch between glibc and the procfs server + <braunr> nothing more + <braunr> it may help, it may not, that's what i'd like to know + <nalaginrut> hah~thanks for that + <nalaginrut> I get source from apt-get, then manually modified the files, + no much code ;-) + <nalaginrut> compiling + <nalaginrut> there is no cpuinfo in /proc? + <teythoon> no + <nalaginrut> a feature need to be done? or there's another way for that? + <teythoon> well, it hasn't been implemented + <teythoon> do you need that? what for? + <nalaginrut> compiling error, I realized I should use gcc-4.7 + <pinotree> how are you building? + <nalaginrut> I just happened to play proc while compiling, and found + there's no + <nalaginrut> cxa_finalize.c:48:1: error: tcbhead_t has no member + named multiple_threads + <nalaginrut> I changed to gcc-4.7 + <pinotree> just edit the sources, and then dpkg-buildpackage -nc -us -uc + <pinotree> that will rebuild the debian package as it would be in a debian + build, making sure all the build dependencies are there, etc + <pinotree> doing it different than that is just wrong™ + <nalaginrut> ok, doing + <pinotree> were you really doing ./configure etc yourself? + <nalaginrut> well, I can't wait till it's done, I'll let it compile and + check it out tomorrow + <nalaginrut> I used configure, yes ;-P + <pinotree> not good + <nalaginrut> I have to go, thanks for help guys + + +### IRC, freenode, #hurd, 2013-08-22 + + < nalaginrut> eglibc was done by dpkg-buildpackage, then how to install it? + (sorry I'm a brand new debian users) + < nalaginrut> oh~I found it + < nalaginrut> yes, (times) returns reasonable result ;-) + * nalaginrut is trying 'make check' + < nalaginrut> unfortunately, it can't pass the test though, I'm researching + it, anyway, we made first step + < nalaginrut> for Hurd internal-time-units-per-second will be 1000 + < nalaginrut> , but the elapsed time is far larger than (* 2 + internal-time-units-per-second) + < nalaginrut> I think the different of two returned clocks after 1 second + should be the TIME_UNITS_PER_SECOND, in principle + < nalaginrut> but I'm not sure if it's elibc or Guile bug + < nalaginrut> dunno, maybe clock tick should be 1000? + < nalaginrut> well, I'll try clock per second as 1000 + < braunr> nalaginrut: clock tick (or actually, the obsolete notion of a + clock tick in userspace) should be 100 + < braunr> nalaginrut: how did you come with 1000 ? + < nalaginrut> braunr: Guile set TIME_UNITS_PER_SECOND to 1000 when there's + no 8bytes size and doesn't define HAVE_CLOCK_GETTIME + < nalaginrut> #if SCM_SIZEOF_LONG >= 8 && defined HAVE_CLOCK_GETTIME + < nalaginrut> #define TIME_UNITS_PER_SECOND 1000000000 + < nalaginrut> #else + < nalaginrut> #define TIME_UNITS_PER_SECOND 1000 + < nalaginrut> #endif + < nalaginrut> and the test for 'times' used time-units-per-second + < pinotree> what has sizeof(long) have to do with time units per second? + < nalaginrut> dunno, maybe the representation of time? + < nalaginrut> the test failed since the difference between two clocks after + 1sec is too large + < nalaginrut> and for the test context, it should small than 2 times of + units-per-second + < nalaginrut> should be smaller + < nalaginrut> sorry for bad English + < pinotree> aren't you basically looking for clock_getres? + < nalaginrut> pinotree: I don't understand what you mean + < pinotree> + http://pubs.opengroup.org/onlinepubs/9699919799/functions/clock_getres.html + < nalaginrut> I wonder if there's a standard CLK_PER_SEC for Hurd + < nalaginrut> or it can be modified as wish + < pinotree> why do you need it? + < nalaginrut> the difference is 10,000,000, which can never less than + 2*clock_per_second + < nalaginrut> pinotree: I don't need it, but I want to know if there's a + standard value + < braunr> nalaginrut: ok so, this is entirely a guile thing + < braunr> nalaginrut: did you test with my patch ? + < nalaginrut> braunr: yes, 'times' works fine + < braunr> but even with that, a tets fails ? + < braunr> test* + < nalaginrut> well, I can't say works fine, the proper description is "now + it has reasonable result" + < braunr> youpi: could you bring + http://darnassus.sceen.net/gitweb/savannah_mirror/glibc.git/commit/90404d6d1aa01f6ce1557841f5a675bb6a30f508 + into debian glibc btw ? + < nalaginrut> braunr: it failed the test since the clock run too fast, but + it should be smaller than 2*clk-per-sec + < braunr> i don't get that + < braunr> can you show the code that checks the condition ? + < nalaginrut> braunr: http://pastebin.com/sG3QxnPt + < braunr> * 0.5 internal-time-units-per-second ? + < nalaginrut> for C users, it's just like + a=times(...);sleep(1);b=times(...); then time-units-per-sec/2 <= (b-a) <= + time-units-per-sec*2 + < braunr> ah ok + < nalaginrut> the test passes when it's true + < braunr> so basically, it says sleep(1) sleeps for more than 2 seconds + < braunr> can you check the actual value ? + < braunr> b-a + < nalaginrut> hold on for minutes + < nalaginrut> it's 10,000,000 + < nalaginrut> for clk-per-sec=1000,000,000, it's OK + < nalaginrut> but for 100 or 1000, it's too small + < braunr> let's forget 100 + < braunr> guile uses 1000 + < nalaginrut> OK + < braunr> but i still don't get why + < nalaginrut> so I asked if there's standard value, or it can be ajustified + < nalaginrut> adjusted + < braunr> ok so, times are expressed in clock ticks + < braunr> are you sure you're using a patched glibc ? + < nalaginrut> yes I used your patch, and the 'times' get reasonable result + < braunr> then + < braunr> 11:28 < nalaginrut> it's 10,000,000 + < braunr> doesn't make sense + < nalaginrut> hmm + < braunr> anhd i don't understand the test + < braunr> what's tms:clock new ? + < nalaginrut> it's actually the return value of 'times' + < nalaginrut> Guile wrap the clock_t and tms to a vector, then we can get + all the thing in a row + < nalaginrut> 'new' is a variable which was gotten after 1 sec + < braunr> let's see what this does exactly + < nalaginrut> equal to "new = times(...)" + < nalaginrut> 'tms' equal to (clock_t (struct tms)) + < nalaginrut> we have to pass in the struct pointer to get the struct + values filled, but for Guile we don't use pointer, times actually returns + two things: clock_t and struct tms + < nalaginrut> and Guile returns them as a vector in a row, that's it + < braunr> nalaginrut: test this please: + http://darnassus.sceen.net/~rbraun/test.c + < braunr> i don't have a patched libc here + < braunr> i'll build one right now + < nalaginrut> clock ticks: 1000000 + < braunr> and this seems reasonable to you ? + < braunr> anyway, i think the guile test is bugged + < nalaginrut> no, the reasonable is not for this + < braunr> does it ever get the clock tick value from sysconf() ? + < nalaginrut> I say reasonable since it's always 0 both for clock and tms, + before apply your patch + < braunr> uh no + < braunr> i have the same value, without my patch + < nalaginrut> so I said "I can't say it works fine" + < braunr> either the test is wrong because it doesn't use sysconf() + < nalaginrut> anyway, I don't think times should return "all zero" + < braunr> or the clock values have already been ocnverted + < braunr> but it doesn't + < braunr> you did something wrong + < nalaginrut> with your patch it doesn't + < braunr> without neither + < braunr> 11:43 < braunr> i have the same value, without my patch + < nalaginrut> well, it's too strange + < braunr> check how the test actually gets the clock values + < braunr> also, are your running in vbox ? + < braunr> you* + < nalaginrut> no ,it's physical machine + < braunr> oh + < braunr> nice + < braunr> note that vbox has timing issues + < nalaginrut> I thought I should give you some info of CPU, but there's no + /proc/cpuinfo + < braunr> shouldn't be needed + < nalaginrut> OK + < braunr> run my test again with an unpatched glibc + < braunr> just to make sure it produces the same result + < braunr> and + < nalaginrut> so the clock-per-sec is machine independent for Hurd I think + < braunr> 11:46 < braunr> check how the test actually gets the clock values + < nalaginrut> since it's implemented in userland + < braunr> clock-per-sec is always system dependent + < braunr> All times reported are in clock ticks. + < braunr> The number of clock ticks per second can be obtained + using: + < braunr> sysconf(_SC_CLK_TCK); + < braunr> 11:46 < braunr> check how the test actually gets the clock values + < braunr> to see if they're converted before reaching the test code or not + * nalaginrut is building eglibc + < braunr> building ? + < braunr> what for ? + < nalaginrut> I modified it to 1000, now it's useless + < braunr> we want it to 100 either way + < nalaginrut> and how to reinstall eglibc under debian? + < braunr> it's obsolete, procfs already uses 100, and 100 is low enough to + avoid overflows in practically all cases + < braunr> aptitude install libc0.3=<version> + < nalaginrut> OK + < braunr> aptitude show -v libc0.3 + < braunr> for the list of available versions + < nalaginrut> out of topic, what's the meaning of the code in + quantize_timeval ? + < nalaginrut> tv->tv_usec = ((tv->tv_usec + (quantum - 1)) / quantum) * + quantum; + < nalaginrut> I can't understand this line + < braunr> scaling and rounding i guess + < nalaginrut> hmm...but quantum seems always set to 1? + < nalaginrut> 100/__getclktck() + < braunr> ah right + < braunr> old crap from the past + < nalaginrut> and clk-tck is 100 + < braunr> the author probably anticipated clk_ticks could vary + < braunr> in practice it doesn't, and that's why it's been made obsolete + < nalaginrut> I wonder if it could be vary + < braunr> no + < nalaginrut> alright + < nalaginrut> why not just assign it to 1? + < braunr> 11:55 < braunr> old crap from the past + < braunr> the hurd is 20 years old + < braunr> like linux + < nalaginrut> oh~ + < braunr> but with a lot less maintenance + < nalaginrut> braunr: well, I tried the original eglibc, your test was + clock ticks: 1000000 + < nalaginrut> but in Guile, (times) ==> (0 0 0 0 0) + < nalaginrut> the reasonable result maybe: #(4491527510000000 80000000 0 0 + 0) + < braunr> 11:46 < braunr> check how the test actually gets the clock values + < braunr> ah, he left + + +### IRC, freenode, #hurd, 2013-08-23 + + < braunr> nalaginrut: times() doesn't seem to be affected by my patch at + all + < nalaginrut> braunr: but it did in my machine + < nalaginrut> well, I think you mean it doesn't affect your C test code + < braunr> i'm almost sure something was wrong in your test + < braunr> keep using the official debian glibc package + < nalaginrut> I don't think it's test issue, since every time (times) + return zero, the test can never get correct result + < braunr> times doesn't return 0 + < braunr> for sleep(1), i always have the right result, except in + microseconds + < nalaginrut> times in Guile always return #(0 0 0 0 0) + < braunr> (microseconds is the native mach time unit) + < braunr> well, guile does something wrong + < nalaginrut> after sleep 1, it's 0 again, so it's none sense + < braunr> 11:46 < braunr> check how the test actually gets the clock values + < braunr> not on my system + < nalaginrut> but (times) returns reasonable result after applied your + patch + < braunr> that's not normal, since times isn't affected by my patch + < nalaginrut> oops + < braunr> you need to look for what happens in guile between the times() + call and the #(0 0 0 0 0) values + < nalaginrut> well, I tried many times between patch or non-patch, I think + there's no mistake + < nalaginrut> I read the 'times' code in Guile, there's nothing strange, + just call 'times' and put all the result to a vector + < braunr> which means there is no conversion + < braunr> in which case the test is plain wrong since there MUST also be a + call to sysconf() + < braunr> to obtain the right clock ticks value + < braunr> is your box reachable with ssh ? + < nalaginrut> oh~wait, seems there's a quotient operation, I'm checking + < nalaginrut> factor = scm_quotient (scm_from_long (TIME_UNITS_PER_SECOND), + < nalaginrut> scm_from_long (ticks_per_second)); + < braunr> iirc, TIME_UNITS_PER_SECOND is hardcoded + < nalaginrut> unless factor is zero + < nalaginrut> yes, it's hardcoded + < braunr> that's completely non portable and wrong + < nalaginrut> you suggest to call sysconf? + < braunr> yes + < braunr> but i don't have the code in mind + < braunr> what is ticks_per_second ? + < nalaginrut> OK, that's one issue, we have to find why times return 0 + < braunr> 14:14 < braunr> is your box reachable with ssh ? + < braunr> i'd like to make sure times returns 0 at your side + < braunr> because it doesn't at mine + < nalaginrut> no + < braunr> until i can reproduce, i can't consider there is a problem + < nalaginrut> I think it's unreachable for outer space + < nalaginrut> well, if you want to reproduce, just get guile src of debian + < braunr> guile 2.0 ? + < nalaginrut> yes, apt-get source guile-2.0 + < nalaginrut> I'm checking ticks_per_second + < braunr> got the source, how do i test + < braunr> ? + < nalaginrut> you have to build it, and run ./meta/guile, then you don't + have to install it + < nalaginrut> and try (times) + < braunr> aw libgc + < nalaginrut> the reasonable result should be #(4313401920000000 110000000 + 20000000 0 0) or something alike + < nalaginrut> but #(0 0 0 0 0) in each time is not reasonable apparently + < nalaginrut> maybe you need apt-get build-dep guile-2.0? + < braunr> already done + < nalaginrut> building Guile2 may take very long time + < nalaginrut> about 30 minutes in my old machine + < braunr> then it should take just a few minutes on mine + < nalaginrut> alright it's not very long, I've spent 8 hours for gcc in LFS + < braunr> 8 hours ? + < braunr> takes 5-10 minutes on a common machine .. + < nalaginrut> but it's Celeron566 at that time... + < braunr> ah, that old + < nalaginrut> include bootstrap, so very long + < braunr> nalaginrut: i got the test failure from the build procedure, how + do i run it manually ? + < nalaginrut> braunr: ./meta/guile -L test-suite + test-suite/tests/time.test + < nalaginrut> braunr: or make check for all + < braunr> put a print after the schedule() and before the return nil; in + runtime_mstart, since that's the body of new threads + < nlightnfotis> unfortunately, I can't confirm this with goroutines + running; the assertion failure aborts before I can get anything useful + < braunr> you can + < braunr> make sure there is a \n in the message, since stdout is line + buffered by default + < braunr> if you don't reach that code, it means threads don't exit + < braunr> at least goroutine threads + < braunr> btw, where is the main thread running ? + < nlightnfotis> I just checked there is a \n at the end. + < nlightnfotis> "<braunr> btw, where is the main thread running " could you + elaborate a little bit on this? + < braunr> what does main() after initializing the runtime ? + < braunr> +do + < nlightnfotis> the runtime main or the process's main? + < braunr> the process + < braunr> nlightnfotis: what we're interested in is knowing whether main() + exits or not + < nlightnfotis> braunr: I can see there are about 4 functions of interest: + runtime_main (the main goroutine, and I can imagine 1st thread) + < nlightnfotis> main_init (I don't know what it does, will check this out + now) + < nlightnfotis> main_main (not sure about this one either) + < nlightnfotis> and runtime_exit (0) + < braunr> i can see that too + < braunr> i'm asking about main() + < nlightnfotis> which seems to be the function that terminates the main + thread + < nlightnfotis> <braunr> nlightnfotis: what we're interested in is knowing + whether main() exits or not --> my theory is runtime_exit (0) exits the + process' main. Seeing as at various times go programs echo $? == 0. + < nlightnfotis> let me research that a little bit + < nlightnfotis> braunr: that will require a bit more studying. main_main() + and main_init() are both expanded to assembly tags if I understand it + correctly. + < nlightnfotis> main.main and __go_init_main respectively. + < braunr> why are you looking from there instead of looking from main() ? + < nlightnfotis> are we not looking out if main exits? + < braunr> we are + < braunr> so why look at main_main ? + < braunr> or anything else than main ? + < nlightnfotis> these are called inside runtime_main and I figured out they + might have a clue + < braunr> runtime_main != main + < braunr> (except if there is aliasing) + < nlightnfotis> there is still the possibility that runtime_main is the + main function and that runtime_exit(0) exits it. + < braunr> there is no doubt that main is main + < braunr> (almost) + < nlightnfotis> and I just found out that there is no main in assembly + produced from go. Only main.main + < braunr> check the elf headers for the entry point then + < nlightnfotis> braunr: I went through the headers, and found the process' + main. You can find it in <gcc_root>/libgo/runtime/go-main.c + < nlightnfotis> it seems very strange though: It creates a new thread, then + aborts? + < braunr> nlightnfotis: see :) + < braunr> nlightnfotis: add traces there + < nlightnfotis> braunr: can you look into that piece of code to check out + something I don't understand? + < nlightnfotis> braunr: I can not seem able to find __go_go 's definition + < nlightnfotis> only a declaration in runtime.h + < braunr> + https://github.com/NlightNFotis/gcc/blob/master/libgo/runtime/proc.c, + line 1552 + < nlightnfotis> gee thanx. For a strange kind of fashion, I was looking for + it in runtime.c + < braunr> use git grep + < braunr> or tags/cscope + < nlightnfotis> braunr: yep! runtime_exit does seem to terminate a go + process that was not otherwise abnormally terminated. + < braunr> ? + < braunr> is it called or not ? + < braunr> runtime_exit is a macro on exit() + < braunr> so we already know what it does + < nlightnfotis> it is called + < braunr> ok + < braunr> that's not normal :) + < nlightnfotis> for a simple program + < braunr> uh ? + < nlightnfotis> for one that has a go routine + < braunr> but + < nlightnfotis> it doesn't + < nlightnfotis> it's expected + < braunr> ok + < braunr> that makes sense + < braunr> well, trace + < braunr> keep tracing + < braunr> for example in main() + < braunr> is runtime_mstart() actually reached ? + < nlightnfotis> yeah main and runtime_main were my next two targets + < braunr> good + < nlightnfotis> and now I followed your advice and it does compiler much + faster + < braunr> so, it looks like the main thread just becomes a mere kernel + thread + < braunr> running runtime_mstart() and fetching goroutines as needed + < braunr> after your traces, i'd suggest running a small go test program, + with one simple goroutine (doesn't crash right ?) + < braunr> and trace context switching + < braunr> but after the traces + < braunr> one important trace is to understand why runtime_exit gets called + < nlightnfotis> it does crash even with 1 goroutine + < braunr> oh + < braunr> when doesn't it crash ? + < nlightnfotis> when it has 0 goroutines + < nlightnfotis> it works as expected + < nlightnfotis> but anything involving goroutines crashes + < nlightnfotis> and goroutines are very important; everything in the + standard library involves goroutines + < braunr> ok + < braunr> doesn't change what i suggested, good + < braunr> 1/ find out why runtime_exit gets called + < braunr> 2/ trace context switching with 1 goroutine + < nlightnfotis> on it. + < braunr> in all cases, make all your goroutines (including the main one) + *not* return + < braunr> so that you don't deal with goroutine destruction yet + < nlightnfotis> runtime_mstart in main doesn't to be run at all. So the + path is __go_go and then return from it. + < nlightnfotis> *doesn't seem + + +### IRC, freenode, #hurd, 2013-08-26 + + < braunr> youpi: my glibc clock patch looks incomplete btw + < youpi> which one? + < youpi> ah, the ticks one? + < braunr> yes + < braunr> it doesn't change the values returned by times + < braunr> as a side effect, the load average bumps to 2+ on an idle machine + + +### IRC, freenode, #hurd, 2013-08-27 + + < nalaginrut> braunr: have you tried Guile2 on your machine? ;-) + < braunr> nalaginrut: no + < braunr> nalaginrut: but i saw the code actually does use sysconf() + < nalaginrut> braunr: yes, for ticks_per_second + < braunr> i had to look myself to find it out, you didn't say it, despite + me asking multiple times + < braunr> it won't make debugging easier ;p + < braunr> nalaginrut: also, the return value of times is actually *never* + used + < braunr> i don't know why you've been talking about it so much + < nalaginrut> braunr: I'm sorry, it's first time to look stime.c for me + < braunr> the interesting function is get_internal_run_time_times() + < nalaginrut> what do you mean about "the return value of times is actually + *never* used"? in which context? + < braunr> see get_internal_run_time_times + < braunr> struct tms time_buffer; + < braunr> times(&time_buffer); + < braunr> return ... + < braunr> and yes, the user and system time reported in struct tms are 0 + < braunr> let's see what posix has to say about it + < pinotree> it says it will return (clock_t)-1 for errors, but no standard + errors are defined yet + < nalaginrut> but I don't think get_internal_run_time_times has something + to do with scm_times + < braunr> well, i don't see any other call to times() + < braunr> i've asked you repeatedly to look for how guile fetches the data + < braunr> i think it's done in get_internal_run_time_times + < braunr> what makes you think otherwise ? + < braunr> our times() seems to behave fine, other than the units of the + return value + < nalaginrut> I don't understand what do you mean? + get_internal_run_time_times is unrelated to scm_times which is actually + "times" in Scheme code + < braunr> ok + < nalaginrut> I think we're talking about "times" activity, right? + < braunr> ok so result is a vector + < braunr> with the return value and the four values in struct tms + < nalaginrut> yes + < braunr> and what looks interesting is + < braunr> factor = scm_quotient (scm_from_long (TIME_UNITS_PER_SECOND), + scm_from_long (ticks_per_second)); + < braunr> SCM_SIMPLE_VECTOR_SET (result, 0, scm_product (scm_from_long + (rv), factor)); + < braunr> TIME_UNITS_PER_SECOND is 1000 + < nalaginrut> yes, it means (clock_t * + (TIME_UNITS_PER_SECOND/ticks_per_second)), though I've no idea why it + does this + < braunr> normalizing values i guess + < nalaginrut> I wonder if the factor should be 1, just guessing + < braunr> let's see what our clock tick really is + < braunr> 1000000 on an unmodified libc + < braunr> 100 with my patch + < nalaginrut> so what's the problem? + < nalaginrut> all the values were multiplied by ticks, it's fair for the + subtraction + < nalaginrut> I think the problem is clock is too large for the difference + between utime and utime(sleep 1) + < nalaginrut> oops, is too small + < nalaginrut> sorry, I confused, + < nalaginrut> the problem is the difference of clock is too large for + 2*internal-time-units-per-second + < nalaginrut> and actually, internal-time-units-per-second is + SCM_TIME_UNITS_PER_SECOND + < nalaginrut> but without your patch, 'times' would return zeros all the + time, which is never meet the condition: SCM_TIME_UNITS_PER_SECOND/2 <= + (clock2 - clock1) + < nalaginrut> well, maybe your point is + TIME_UNITS_PER_SECOND/ticks_per_second is too small without your patch, + which causes the scm_to_long cast give a 0 value + < nalaginrut> s/cast/casting + < nalaginrut> when ticks_per_second is 100, the factor would be 10, which + seems to be reasonable + < nalaginrut> s/scm_to_long/scm_from_long + < nalaginrut> well, I have to checkout this + < nalaginrut> OK, let me reconstruct the point: ticks_per_second so too + large that makes the factor becomes zero + < nalaginrut> but decrease ticks_per_second to 100 causes the clock become + too large than TIME_UNITS_PER_SECOND + < braunr> 10:59 < nalaginrut> but without your patch, 'times' would return + zeros all the time, which is never meet the condition: + SCM_TIME_UNITS_PER_SECOND/2 <= (clock2 - clock1) + < braunr> until you prove me otherwise, this is plain wrong + < braunr> times() never returned me 0 + < braunr> so let's see, this gives us a factor of 1000 / 1000000 + < braunr> so the problem is factor being 0 + < braunr> that's why *guile* times returns 0 + < braunr> with my patch it should return 10 + < nalaginrut> braunr: I'm sorry I mean "stime" in Scheme returns zeros + < nalaginrut> yes, I think the problem is factor + < nalaginrut> the factor + < braunr> now why doesn't my patch fix it all ? + < braunr> ah yes, rv is still in microseconds + < braunr> that's what i've been telling youpi recently, my patch is + incomplete + < braunr> i'll cook a quick fix, give me a few minutes please + < nalaginrut> but it fixed something ;-) + < braunr> well, guile makes a stupid assumption here + < braunr> so it's not really a fix + < nalaginrut> braunr: should I ask some info about TIME_UNITS_PER_SECOND + from Guile community? + < nalaginrut> or it doesn't help + < braunr> what do you want to ask them ? + < nalaginrut> since I don't know how this value was chosen + < nalaginrut> dunno, I'll ask if you need it + < nalaginrut> I just think maybe you need this info + < braunr> well + < braunr> my plan is to align the hurd on what other archs do + < braunr> i.e. set clk_tck to 100 + < braunr> in which case this won't be a problem any more + < braunr> now you could warn them about the protability issue + < braunr> i'm not sure if they would care though + < nalaginrut> the warning is useful for the future + < nalaginrut> and it's not hard to make a change I think, for a constant, + but it depends on the maintainers + < braunr> it's not that simple + < braunr> time related things can easily overflow in the future + < nalaginrut> alright + < braunr> refer to the 2038 end-of-the-world bug + < nalaginrut> so how can I describe the warning/suggestion to them? + < braunr> i'm not sure + < braunr> tell them the TIME_UNITS_PER_SECOND isn't appropriate for larger + values of clk_tck + < braunr> dammit, microseconds are hardcoded everywhere in + sysdeps/mach/hurd ... >( + < braunr> nalaginrut: my new patch seems to fix the problem + < braunr> nalaginrut: i've built debian packages with which you can + directly test + < braunr> nalaginrut: deb http://ftp.sceen.net/debian-hurd-i386 + experimental/ + < braunr> Totals for this test run: + < braunr> passes: 38605 + < braunr> failures: 0 + < braunr> unexpected passes: 0 + < braunr> expected failures: 7 + < braunr> unresolved test cases: 578 + < braunr> untested test cases: 1 + < braunr> unsupported test cases: 10 + < braunr> errors: 0 + < braunr> PASS: check-guile + < braunr> ============= + < braunr> 1 test passed + < braunr> ============= + < braunr> :) + < braunr> youpi: the branch i added to glibc contains a working patch for + clock_t in centiseconds + < youpi> k + + +### IRC, freenode, #hurd, 2013-08-28 + + <nalaginrut> braunr: well, looks great! I'll try it soon~ + <nalaginrut> braunr: BTW, where is the patch/ + <mark_weaver> braunr: what was needed to get guile working on the hurd? + <mark_weaver> well, if the fix wasn't to guile, I don't need the details. + <braunr> 04:53 < nalaginrut> braunr: BTW, where is the patch/ + <braunr> there is hardly anyone here at 5am + <braunr> nalaginrut: + http://git.savannah.gnu.org/cgit/hurd/glibc.git/log/?h=rbraun/clock_t_centiseconds + <nalaginrut> braunr: thanks for that, but why not use a constant for 100? + <braunr> nalaginrut: i don't know where to define it + <braunr> it's glibc, you don't define new stuff mindlessly + <youpi> braunr: about your centiseconds patch, did you run the libc + testsuite with it? + <mark_weaver> it does seem a shame to reduce the resolution of the timers + from microseconds to centiseconds. I wonder if that could be avoided. + <youpi> by fixing all applications which assume centiseconds + <mark_weaver> *nod* well, if there's such a problem in Guile, I'd be glad + to fix that. + <braunr> youpi: no + <mark_weaver> I see that there's a macro CLOCKS_PER_SEC that programs + should consult. + <youpi> braunr: ok, I'll do then + <braunr> mark_weaver: why is it a shame ? + <braunr> it's not clock or timer resolution + <youpi> it's clock_t resolution + <braunr> it's an obsolete api to measure average cpu usage + <braunr> having such a big value on the other hand reduces the cpu usage + durations + <mark_weaver> braunr: good point :) I confess to being mostly ignorant of + these APIs. + <mark_weaver> Though Guile should still consult CLOCKS_PER_SEC instead of + assuming centiseconds. If it's making an improper assumption, I'd like + to know so I can fix it. + <braunr> the improper assumption is that there are less than 1000 clock + ticks per second + <mark_weaver> do you know off-hand of some code in Guile that is making + improper assumptions? + <braunr> yes + <braunr> let me find it + <mark_weaver> thanks + <braunr> factor = scm_quotient (scm_from_long (TIME_UNITS_PER_SECOND), + <braunr> scm_from_long (ticks_per_second)); + <braunr> it seems guile attempts to normalize all times values to + TIME_UNITS_PER_SECOND + <braunr> while i think it would be better off using ticks_per_second (clock + ticks as provided by sysconf()) + <braunr> attempting to normalize here causes factor to become 0 if + TIME_UNITS_PER_SECOND < ticks_per_second + <mark_weaver> ah, I see. + <mark_weaver> I'll take care of it. thanks for the pointer! + <youpi> braunr: I've commited the centisecond patch to debian's glibc + + +### IRC, freenode, #hurd, 2013-08-29 + + <nalaginrut> braunr: Guile2 works smoothly now, let me try something cool + with it + <braunr> nalaginrut: nice diff --git a/open_issues/tmux.mdwn b/open_issues/tmux.mdwn new file mode 100644 index 00000000..f71d13e1 --- /dev/null +++ b/open_issues/tmux.mdwn @@ -0,0 +1,24 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_porting]] + +# IRC, freenode, #hurd, 2013-08-01 + + <braunr> teythoon: can you stop tmux on darnassus please ? + <braunr> i'd like to check something + <teythoon> done + <braunr> tmux makes load average grow to 5 without any visible activity :/ + <braunr> can't reproduce it with my instances though + <braunr> anyway, that's minor + <teythoon> I used tmux before and never encountered that + <teythoon> sometimes tmux would hang on attaching or detaching though, but + overall I had less problems with tmux than with screen + <teythoon> ah, I tried to start tmux on darnassus and now it hangs diff --git a/open_issues/virtualization/fakeroot.mdwn b/open_issues/virtualization/fakeroot.mdwn index f4739776..f9dd4756 100644 --- a/open_issues/virtualization/fakeroot.mdwn +++ b/open_issues/virtualization/fakeroot.mdwn @@ -22,3 +22,46 @@ License|/fdl]]."]]"""]] <youpi> btw, I believe our fakeroot-hurd is close to working actually <youpi> it's just a argv[0] issue supposed to be fixed by exec_file_name but apparently not fixed in that case, for some reason + + +## IRC, freenode, #hurd, 2013-08-26 + + < teythoon> also I looked into the fakeroot issue, aiui the problem is that + scripts are not handled correctly, right? + < teythoon> the exec server fails to locate the scripts file name, and so + it hands the file_t to the interpreter process and passes /dev/fds/3 as + script name + < teythoon> afaics that breaks e.g. python + < youpi> yes + < youpi> pinotree's exec_file_name is supposed to fix that, but for some + reason it doesn't work here + < pinotree> it was pochu's, not mine + < youpi> ah, right + < teythoon> ah I see, I was wondering about that + < pochu> it was working for a long time, wasn't it? + < pochu> and only stopped working recently + < youpi> did it completely stop? + < youpi> I have indeed seen odd issues + < youpi> I haven't actually checked whether it has completely stopped + working + < youpi> probably worth looking there first + < pinotree> gtk+3.0 fails, but other stuff like glib2.0 and gtester-using + stuff works + < teythoon> huh? I created tests like "#!/bin/sh\necho $0" and that says + /dev/fd..., and a python script doing the same doesn't even run, so how + can it work for a package build? + < youpi> it works for me in plain bash + < youpi> #!/bin/sh + < youpi> echo $0 + < youpi> € $PWD/test.sh + < youpi> /home/samy/test.sh + < teythoon> it does !? + < youpi> yes + < youpi> not in fakeroot-hurd however, as we said + < teythoon> well, obviously it works when not being run under + fakeroot-hurd, yes + < youpi> ok, so we weren't talking about the same thing + < youpi> a mere shell script doesn't work in fakeroot-hurd indeed + < youpi> that's why we still use fakeroot-sysv + < teythoon> right + < youpi> err, -tcp diff --git a/open_issues/virtualization/networking.mdwn b/open_issues/virtualization/networking.mdwn index 7a6474a1..f8bda063 100644 --- a/open_issues/virtualization/networking.mdwn +++ b/open_issues/virtualization/networking.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -28,3 +28,73 @@ Collection about stuff that is relevant for *virtualization* and *networking*. [[hurd/translator/pfinet]] by setting environment variables. Project is now part of [[Virtual_Square_View-OS]]. + + +# OpenVPN + +[[community/meetings/GHM2013]]. + + +## IRC, freenode, #hurd, 2013-08-23 + + <youpi> good news + <youpi> with a couple small patches, openvpn does work as joe user + + +## IRC, freenode, #hurd, 2013-08-30 + + <youpi> it's really cool that openvpn ended up working completely the day + before :) + + +## IRC, freenode, #hurd, 2013-09-03 + + <_d3f> Hey guys, how did you get openvpn working on the Hurd? just curious + as I saw it in the GHM video + <_d3f> no one here who has a clue how to get *vpn working on the Hurd? + <braunr> _d3f: youpi did it + <braunr> i don't know the details + <_d3f> okay, I will question him when I see him around, thx. Do you know if + it was a lot of work to get the tun device working? Because I would like + to use tinc on the Hurd. + <braunr> _d3f: a bit but not that much either + <_d3f> braunr: well, okay. Do you know if the source of his 'port' is + online, I haven't found it :/ + <braunr> it should be soon + + +## IRC, freenode, #hurd, 2013-09-04 + + <_d3f> youpi: you are the guy who has brought openvpn to the hurd, right? I + would like to know how you got the tun/tap thing working as I would like + to use tinc on it. :) + <youpi> _d3f: essentially no modification of openvpn iirc + <youpi> just tell it to open the tun node created by pfinet + <youpi> and read/write it + <youpi> i.e. the existing generic code in place in openvpn + <_d3f> I will have a look at it, somekind tinc builds with the linux + specific device.c but I wasn't able to exchange keys. I will have a look + at the device handling again and try to get the pfinet tun node used. + + +## IRC, freenode, #hurd, 2013-09-07 + + <d3f> anyone here knows how /dev/net is handled on the hurd? Programs using + it say it's not a directory. I tried creating one and setting a netdde + translator for a tun device in it, but this may be wrong as it doesn't + work + <teythoon> d3f: what does /dev/net do? + <teythoon> ah, its tun/tap stuff... + <d3f> on my gnu/linux it includes a tun device + <teythoon> right + <d3f> I am still reading about the Hurd and try to understand /hurd/netdde + and devnode but by now I am quite sure I will need those to set a tun + networktranslator on /dev/net/tun? + <teythoon> hm, I don't think netdde or devnode will be of any help + <teythoon> afaiui devnode makes mach devices available in the hurdish way, + i.e. available for lookup in the filesystem + <teythoon> d3f: ping youpi if he shows up, he hacked up openvpn to work on + the hurd + <d3f> yeah I know, I talked to him as I am tring to get tinc working on the + Hurd (tinc builds by now). I will give him a shot about creating the + "tun" device |