path: root/open_issues/libpthread_dlopen.mdwn
diff options
authorSamuel Thibault <>2015-02-18 00:58:35 +0100
committerSamuel Thibault <>2015-02-18 00:58:35 +0100
commit49a086299e047b18280457b654790ef4a2e5abfa (patch)
treec2b29e0734d560ce4f58c6945390650b5cac8a1b /open_issues/libpthread_dlopen.mdwn
parente2b3602ea241cd0f6bc3db88bf055bee459028b6 (diff)
Revert "rename open_issues.mdwn to service_solahart_jakarta_selatan__082122541663.mdwn"
This reverts commit 95878586ec7611791f4001a4ee17abf943fae3c1.
Diffstat (limited to 'open_issues/libpthread_dlopen.mdwn')
1 files changed, 244 insertions, 0 deletions
diff --git a/open_issues/libpthread_dlopen.mdwn b/open_issues/libpthread_dlopen.mdwn
new file mode 100644
index 00000000..a825fdff
--- /dev/null
+++ b/open_issues/libpthread_dlopen.mdwn
@@ -0,0 +1,244 @@
+[[!meta copyright="Copyright © 2011, 2012, 2013, 2014 Free Software Foundation,
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+[[!tag open_issue_libpthread]]
+# [[FAQ entry|faq/libpthread_dlopen]]
+# IRC, freenode, #hurd, 2010-01-24
+ <pinotree> youpi: hm, thought about the pthread/stubs issue w/ dlopen'ed
+ libraries
+ <pinotree> currently looks like libstdc++ on hurd links to pthread-stubs,
+ we're the only one with such configuration
+ <pinotree> i was looking at the gcc 4.4 patch hurd-pthread.diff, could it
+ be it does not set THREADLIBS in the switch case?
+ <youpi> that's expected
+ <youpi> on linux the libc provides hooks itself, on hurd-i386 it's
+ pthread-stubs
+ <pinotree> why not explicitly link to pthread though?
+ <youpi> because there is no strict need to, for applications that don't
+ need libpthread
+ <youpi> the dlopen case is a tricky case that pthread-stubs had not thought
+ about
+ <pinotree> hm
+ <pinotree> what if the pthread stubs would be moved in our glibc?
+ <youpi> that's what we should do yes
+ <youpi> (ideally)
+ <youpi> but for this we need to build libpthread along glibc, to get it
+ really working
+ <youpi> and that's the tricky part (Makefile & such) which hasn't been done
+ yet
+ <pinotree> why both (stubs + actual libpthread)?
+ <youpi> because you need the stubs to be able to call the actual libpthread
+ <youpi> as soon libpthread gets dlopened for instance
+ <youpi> +as
+ <pinotree> i see
+ <youpi> (remember that nptl does this if you want to see how)
+ <youpi> (it's the libc files in nptl/)
+ <youpi> (and forward.c)
+ <guillem> also if libpthreads gets integrated with glibc don't we need to
+ switch the hurd from cthreads then? Which has been the blocker all this
+ time AFAIR?
+ <youpi> we don't _need_ to
+ <guillem> ok
+# IRC, OFTC, #debian-hurd, 2011-07-21
+ <youpi> there's one known issue with pthreads
+ <youpi> you can't dlopen() it
+... if the main application is not already linked against it.
+ <youpi> which also means you can't dlopen() a module which depends on it if
+ the main application hasn't used -lpthread already
+ <youpi> (so as to get libpthread initialized early, not at the dlopen()
+ call)
+ <lucas> I get this while building simgrid:
+ <lucas> cd /home/lucas/simgrid-3.6.1/obj-i486-gnu/examples/gras/console &&
+ /usr/bin/cmake -E create_symlink
+ /home/lucas/simgrid-3.6.1/obj-i486-gnu/lib/
+ /home/lucas/simgrid-3.6.1/obj-i486-gnu/examples/gras/console/
+ <lucas> cd /home/lucas/simgrid-3.6.1/obj-i486-gnu/examples/gras/console &&
+ lua /home/lucas/simgrid-3.6.1/examples/gras/console/ping_generator.lua
+ <lucas> lua:
+ /home/buildd/build/chroot-sid/home/buildd/byhand/hurd/./libpthread/sysdeps/generic/pt-mutex-timedlock.c:68:
+ __pthread_mutex_timedlock_internal: Assertion `__pthread_threads' failed.
+ <lucas> Aborted (core dumped)
+ <youpi> that's it, yes
+ <youpi> (or at least it has the same symptoms)
+ <lucas> it would need fixing in lua, not in SG, then, right?
+ <youpi> yes
+ <lucas> ok, thanks
+The fix thus being: link the main application with -lpthread.
+IRC, freenode, #hurd, 2011-08-17
+ < youpi> i.e. openjade apparently dlopen()s modules which use pthreads, but
+ openjade itself is not liked against libpthread
+ < youpi> which means unexpectedly loading pthreads on the fly, which is
+ not implemented
+ < youpi> (and hard to implement of course)
+ < youpi> gnu_srs: so simply tell openjade people to link it with -lpthread
+ < gnu_srs> Shuoldn't missing linking with pthread create an error when
+ building openjade then?
+ < youpi> no
+ < youpi> because it's just a module which needs pthread
+ < youpi> and that module _is_ linked with -lpthread
+ < youpi> and dlopen() loads libpthreads too due to that
+ < youpi> but that's unexpected, for the libpthread initialization stuff
+ < youpi> (and too late to fix initlaization)
+ < gnu_srs> How come that other OSes build opensp w/o problems?
+ < youpi> because there are stubs in the libc
+ < gnu_srs> Sorry for the delay: What hinders stubs to be present also in
+ the Hurd libc parts too, to cope with this problem?
+ < youpi> doing it
+ < youpi> which is hard because you need libpthread bits inside the libc
+ < youpi> making it simpler would need building libpthread at the same time
+ as libc
+# IRC, freenode, #hurd, 2013-09-03
+ <gnu_srs> iceweasel: ./pthread/../sysdeps/generic/pt-mutex-timedlock.c:70:
+ __pthread_mutex_timedlock_internal: Assertion `__pthread_threads' failed.
+ <pinotree> LD_PRELOAD libpthread
+ <gnu_srs> why
+ <pinotree> missing link to pthread?
+ <pinotree> and yes, it's known already, just nobody worked on solving it
+# IRC, freenode, #hurd, 2014-01-28
+ <gnu_srs> braunr: Is this fixed by your recent patches? test_dbi:
+ ./pthread/../sysdeps/generic/pt-mutex-timedlock.c:70:
+ <gnu_srs> __pthread_mutex_timedlock_internal: Assertion `__pthread_threads'
+ failed.
+ <youpi> faq/libpthread_dlopen.mdwn:
+ ./pthread/../sysdeps/generic/pt-mutex-timedlock.c:70:
+ __pthread_mutex_time
+ <gnu_srs> youpi: tks. A workaround seems to be available:
+ LD_PRELOAD=/lib/i386-gnu/
+ <gnu_srs> Is that possible on a buildd?
+ <youpi> it would be simpler to just make the package explicitly link
+ libpthread
+ <gnu_srs> Package is libdbi-drivers, providing libdbd-sqlite3 needed by
+ gnucash
+# IRC, freenode, #hurd, 2014-02-17
+ <braunr> hm ok, looks like iceweasel errors all have something to do with
+ the libc dns resolver
+ <braunr>
+ <braunr> apparently, it's simply because the memory chunk isn't page
+ aligned ..
+ <braunr> looks like not preloading libpthread tirggers lots of tricky
+ issues
+ <braunr> anyway, apparently, the malloc/free calls in libresolv don't use
+ locks if libpthread isn't preloaded, which explains why the program state
+ looked impossible to reach and why crashes look random
+ <congzhang> debian linux does not have the pthread load problem.
+ <braunr> congzhang: it had it
+ <braunr> maybe not debian but i've found one such report for opensuse
+ <braunr> ok the bug is simple
+ <braunr> for some reason, our glibc still uses a global _res state for dns
+ resolution instead of per thread ones
+ <braunr> uh, apparently, it's libpthread's job to define a __res_state
+ function for that :(
+## IRC, freenode, #hurd, 2014-02-18
+ <braunr> usually when i say it, it crashes soon after, so let's try it :
+ <braunr> i've been running iceweasel 27 fine for like 10 minutes with a
+ patched libpthread
+ <braunr> still no crash ;p
+ <braunr> with luck this extremely lightweight patch will fix all
+ multithreaded applications doing concurrent name resolution .... :)
+ <teythoon> nice :)
+ <braunr> let's try gnash ....
+ <braunr> uh, segfault on termination
+ <braunr> gnash works :)
+ <teythoon> sweet :)
+ <braunr> i'm very surprised we could live so long with that resolv bug
+## IRC, freenode, #hurd, 2014-02-19
+ <braunr> youpi: the eglibc bug is about libresolv
+ <braunr> it uses a global resolver state even in multithreaded applications
+ <youpi> libresolv is a horrible part of glibc :)
+ <braunr> which is obviously bad
+ <braunr> yes .. :)
+ <braunr> here is the patch :
+ <braunr> it's very short, it basically allocates a resolver state per
+ thread in the pthread struct, and sets the TLS variable __resp when the
+ thread starts
+ <braunr> should we make that hurd-specific ?
+ <braunr> or enclose that assignment with #ifdef ENABLE_TLS ?
+ <youpi> well, ENABLE_TLS is now always 1, iirc :)
+ <braunr> for the hurd, yes
+ <youpi> I'm surprised linux never had the issue
+ <youpi> no, not for the hurd
+ <braunr> ah
+ <youpi> I *had* to implement TLS for hurd because it was always 1 for
+ everybody :)
+ <braunr> ok
+ <braunr> so all those ifdefs could be removed and libpthread can assume tls
+ is enabled
+ <braunr> in which case my patch looks fine
+ <youpi> ah, thats a libpthread patch, not glibc patch
+ <braunr> yes
+ <braunr> nptl obviously did that from the start . :)
+ <braunr> linuxthreads had the problem a looong time ago
+ <youpi> ok
+ <braunr> i'm surprised we overlooked it for so long
+ <braunr> but anyway, that's a good fix
+ <youpi> indeed
+ <youpi> it seems all good to me
+ <braunr> well, __resp is a __thread variable
+ <braunr> i could add #ifdef ENABLE_TLS, but then what of the case where TLS
+ isn't enabled, and do we actually care ?
+ <braunr> #error maybe ?
+ <braunr> or #warning ?
+ <youpi> I don't think we care about the non-TLS case any more
+ <braunr> ok
+ <braunr> topgit branch i suppose ?
+ <youpi> well, not, hurd libpthread repo :)
+ <braunr> oh right ... :)
+# libthreads vs. libpthread
+The same symptom appears in an odd case, for instance:
+ buildd@hurd:~$ ldd /usr/bin/openjade
+ => /lib/ (0x0103d000)
+ => /usr/lib/ (0x01044000)
+ => /lib/ (0x01221000)
+ => /lib/i386-gnu/ (0x01232000)
+ [...]
+openjade links against *both* libthreads and libpthread. The result is that libc
+early-initializes libthreads only, and thus libpthread is not early-initialized,
+and later on raises assertions. The solution is to just get rid of libthreads,
+to have only one threading library.