summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--community/gsoc/2013/nlightnfotis.mdwn15
-rw-r--r--community/gsoc/project_ideas/object_lookups.mdwn63
-rw-r--r--faq/sata_disk_drives/discussion.mdwn18
-rw-r--r--glibc/process.mdwn2
-rw-r--r--hurd/authentication.mdwn226
-rw-r--r--hurd/console.mdwn20
-rw-r--r--hurd/console/discussion.mdwn47
-rw-r--r--hurd/libfuse.mdwn19
-rw-r--r--hurd/porting/guidelines.mdwn20
-rw-r--r--hurd/running/virtualbox.mdwn39
-rw-r--r--hurd/subhurd/discussion.mdwn34
-rw-r--r--hurd/translator.mdwn1
-rw-r--r--hurd/translator/auth.mdwn3
-rw-r--r--hurd/translator/discussion.mdwn26
-rw-r--r--hurd/translator/ext2fs.mdwn38
-rw-r--r--hurd/translator/fifo.mdwn6
-rw-r--r--hurd/translator/magic.mdwn262
-rw-r--r--hurd/translator/mtab/discussion.mdwn482
-rw-r--r--hurd/translator/proc.mdwn29
-rw-r--r--hurd/translator/procfs/jkoenig/discussion.mdwn82
-rw-r--r--hurd/translator/term.mdwn207
-rw-r--r--hurd/translator/tmpfs/discussion.mdwn37
-rw-r--r--microkernel/mach/deficiencies.mdwn307
-rw-r--r--microkernel/mach/gnumach/boot_trace.mdwn22
-rw-r--r--open_issues/64-bit_port.mdwn7
-rw-r--r--open_issues/anatomy_of_a_hurd_system.mdwn8
-rw-r--r--open_issues/boehm_gc.mdwn19
-rw-r--r--open_issues/code_analysis/discussion.mdwn56
-rw-r--r--open_issues/dbus.mdwn112
-rw-r--r--open_issues/debugging_gnumach_startup_qemu_gdb.mdwn34
-rw-r--r--open_issues/emacs.mdwn17
-rw-r--r--open_issues/exec_memory_leaks.mdwn25
-rw-r--r--open_issues/ext2fs_libports_reference_counting_assertion.mdwn13
-rw-r--r--open_issues/gdb_qemu_debugging_gnumach.mdwn19
-rw-r--r--open_issues/gdb_signal_handler.mdwn71
-rw-r--r--open_issues/git-core-2.mdwn107
-rw-r--r--open_issues/glibc.mdwn319
-rw-r--r--open_issues/glibc/t/tls-threadvar.mdwn37
-rw-r--r--open_issues/gnumach_page_cache_policy.mdwn60
-rw-r--r--open_issues/hurd_101.mdwn38
-rw-r--r--open_issues/hurd_init.mdwn8
-rw-r--r--open_issues/libpthread/t/fix_have_kernel_resources.mdwn64
-rw-r--r--open_issues/lsof.mdwn40
-rw-r--r--open_issues/mach-defpager_swap.mdwn21
-rw-r--r--open_issues/multiprocessing.mdwn6
-rw-r--r--open_issues/performance.mdwn4
-rw-r--r--open_issues/performance/io_system/read-ahead.mdwn10
-rw-r--r--open_issues/performance/microkernel_multi-server.mdwn183
-rw-r--r--open_issues/pthread_atfork.mdwn86
-rw-r--r--open_issues/smp.mdwn8
-rw-r--r--open_issues/strict_aliasing.mdwn15
-rw-r--r--open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn2
-rw-r--r--open_issues/time.mdwn14
-rw-r--r--open_issues/wine.mdwn15
-rw-r--r--unix/process.mdwn8
55 files changed, 3338 insertions, 93 deletions
diff --git a/community/gsoc/2013/nlightnfotis.mdwn b/community/gsoc/2013/nlightnfotis.mdwn
index a9176f51..83e97bc7 100644
--- a/community/gsoc/2013/nlightnfotis.mdwn
+++ b/community/gsoc/2013/nlightnfotis.mdwn
@@ -3035,3 +3035,18 @@ But not the [[open_issues/libpthread_dlopen]] issue?
<nlightnfotis> and we wanna prove that go violates this rule right? That
the stack pointer is not pointing at the initial stack
<braunr> yes
+
+
+# IRC, freenode, #hurd, 2013-10-09
+
+ <gnu_srs> braunr: The crash is not in the assembly code, but in the called
+ function from it:
+ <gnu_srs> pthread_sigmask (how=2, set=0xf9cac <server_block_set>,
+ oset=oset@entry=0x0) at ./pthread/pt-sigmask.c:29
+ <gnu_srs> 29 struct __pthread *self = _pthread_self ();
+ <gnu_srs> Program received signal SIGSEGV, Segmentation fault.
+ <braunr> gnu_srs: ok so, same problem as in gcc go
+ <braunr> changing the stack pointer prevents libpthread from correctly
+ fetching thread-specific data (including _pthread_self()) correctly
+ <braunr> this will be fixed when threadvards are finally replaced with true
+ tls
diff --git a/community/gsoc/project_ideas/object_lookups.mdwn b/community/gsoc/project_ideas/object_lookups.mdwn
index 88ffc633..ca586dea 100644
--- a/community/gsoc/project_ideas/object_lookups.mdwn
+++ b/community/gsoc/project_ideas/object_lookups.mdwn
@@ -69,3 +69,66 @@ In context of [[!message-id "20130918081345.GA13789@dalaran.sceen.net"]].
<neal> braunr: That's called protected payload.
<neal> braunr: The idea is that the kernel appends data to the message in
flight.
+
+
+## IRC, freenode, #hurd, 2013-10-24
+
+ <teythoon> and, with some effort, getting rid of the hash table lookup by
+ letting the kernel provide the address of the object (iirc neil knew the
+ proper term for that)
+ <braunr> teythoon: that is a big interface change
+ <teythoon> how so
+ <braunr> optimizing libihash and libpthread should already be a good start
+ <braunr> well how do you intend to add this information ?
+ <braunr> ok, "big" is overstatement, but still, it's a low level interface
+ change that would probably break a lot of things
+ <teythoon> store a pointer in the port structure in gnumach, make that
+ accessible somehow
+ <braunr> yes but how ?
+ <teythoon> interesting question indeed
+ <braunr> my plan for x15 is to make this "label" part of received messages
+ <braunr> which means you need to change the format of messages
+ <braunr> that is what i call a big change
+ <teythoon> ok, so we need to provide an update path
+ <teythoon> but once done, the change to hurd will be minimal, patching
+ libports should cover most of that
+ <braunr> normally yes
+ <teythoon> so this amounts to messing with gnumach and mig and designing a
+ clever way to make the update process safe
+
+ <braunr> libihash is known to show high collision rates
+ <teythoon> right, libihash
+ <teythoon> it could use an integer hash function on the keys to distribute
+ them better
+ <braunr> i think that's already what it tries to do
+ <braunr> so merely using a better hash algorithm such as murmur should do
+ the job
+ <braunr> or use another data structure altogether
+ <teythoon> no, it does no hashing of its own on the keys
+ <braunr> are you sure ?
+ <teythoon> well, it uses only prime numbers as sizes, and computes key %
+ size
+ <braunr> well that's hashing .. :)
+ <teythoon> but this is not really a good hash
+ <braunr> yes
+ <braunr> isn't that what i said ?
+ <teythoon> right
+ <teythoon> ok, I didn't get that ;)
+ <teythoon> also, the sizes start quite small, 3, 7, 19...
+ <teythoon> and each time the hash table is grown, all items will have to be
+ updated
+ <braunr> which is why we could consider another data structure
+ <teythoon> or, for starters, to thin out that list of sizes
+ <braunr> my personal preference being radix trees
+ <teythoon> I assume you have an implementation handy?
+ <braunr> yes
+ <teythoon> cool :D
+ <braunr> but good hashing is excellent too
+ <braunr> radix trees have their own issues
+ <teythoon> braunr: http://burtleburtle.net/bob/hash/integer.html
+ <braunr> i use thomas wang's hashing function in x15
+ <braunr> or rather, my own personal c utility library, since x15 doesn't
+ hash anything currently
+ <braunr> but murmur is better
+ <braunr> we prefer distribution over hashing performances
+ <braunr> https://131002.net/siphash/
diff --git a/faq/sata_disk_drives/discussion.mdwn b/faq/sata_disk_drives/discussion.mdwn
index e9da8560..d05566b6 100644
--- a/faq/sata_disk_drives/discussion.mdwn
+++ b/faq/sata_disk_drives/discussion.mdwn
@@ -238,3 +238,21 @@ License|/fdl]]."]]"""]]
<braunr> i'll stick with ide for now, but at least setting sata with
libvirt was quite easy to do
<braunr> so we can easily switch later
+
+
+## IRC, freenode, #hurd, 2013-10-22
+
+ <teythoon> youpi: do I need to do anything to enable the ahci driver?
+ gnumach 1.4 should include it, right?
+ <youpi> it should, yes
+ <youpi> make sure to put your board in ahci mode, not raid mode
+ <youpi> (and not ata mode)
+ <teythoon> youpi: hm, I will try to do so
+ <teythoon> youpi: does the driver print anything to the console?
+ <youpi> teythoon: yes, AHCI SATA 00:04.0 BAR 0xfebf1000 IRQ 11
+ <teythoon> youpi: well, the bios has two modes of operation, 'raid' and
+ 'ide', I selected 'ide'
+ <youpi> ergl
+ <teythoon> youpi: hm, I think my board has no ahci controller, linux uses
+ the sata_via module to talk to it :/
+ <youpi> ah :/
diff --git a/glibc/process.mdwn b/glibc/process.mdwn
index ded2e1f7..c8a1ce79 100644
--- a/glibc/process.mdwn
+++ b/glibc/process.mdwn
@@ -13,7 +13,7 @@ The GNU Hurd uses a similar concept to [[UNIX processes|unix/process]].
As a [[Mach task|microkernel/mach/task]] only implements a part of a UNIX
process, there is additional work to be done, for example for [[signal]]s,
-[[environment_variable]]s, [[file_descriptor]]s.
+[[environment_variable]]s, [[file_descriptor]]s, [[hurd/authentication]].
# Startup
diff --git a/hurd/authentication.mdwn b/hurd/authentication.mdwn
index 2d6084bf..36d18fbb 100644
--- a/hurd/authentication.mdwn
+++ b/hurd/authentication.mdwn
@@ -1,18 +1,22 @@
-[[!meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2007, 2008, 2013 Free Software Foundation,
+Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
-is included in the section entitled
-[[GNU Free Documentation License|/fdl]]."]]"""]]
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
-UIDs on the Hurd are separate from processes. A process has
+UIDs on the Hurd are separate from [[glibc/process]]es. A process has
[[capabilities|capability]] designating so-called UID vectors that
are implemented by an [[translator/auth]] server. This
makes them easily [[virtualizable|virtualization]].
+The standard POSIX interfaces to a [[glibc/process]]' UID management are
+implemented in [[glibc]].
+
When a process wishes to gain access to a resource provided by a third
party (e.g., a file system) and that party wishes to authenticate the client
so as to implement some identity-based access control ([[IBAC]]) policy,
@@ -25,3 +29,217 @@ naming a newly authenticated session with the server
and the server is delivered the client's designated UID vector.
For more details, see section 2.3 of the [[critique]].
+
+
+# Open Issues
+
+[[!tag open_issue_hurd]]
+
+
+## IRC, freenode, #hurd, 2013-09-28
+
+ <braunr> mhmm
+ <braunr> this process has no uid
+ <braunr> isn't it a security issue that processes can remove their identity
+ ?
+ <braunr> i really don't like that we allow processes to loose their
+ identity ...
+ <teythoon> braunr: y not? I think that's a killer feature
+ <teythoon> one that is notoriously absent in unices
+ <braunr> not exactly
+ <braunr> gaining rights to switch your identity is ok
+ <braunr> since you have proven that you are allowed to do it
+ <braunr> now, if you can remove your identity, you can create "ghost"
+ processes
+ <braunr> processes that can spend their day causing denial of services
+ without the possibility for the administrator to know who is responsible
+ <braunr> the unix "way" of dealing with DoS is to warn and ban users after
+ they violated the rules
+ <braunr> we need to have at least that possibility
+ <youpi> perhaps we need to add an "initial" uid
+ <teythoon> otoh the unix way of dropping privileges is hardly being able to
+ do so at all ;)
+ <braunr> teythoon: ?
+ <braunr> on unix, you need privileges to drop your identity :)
+ <braunr> i understand it involves security risks, but that's understandable
+ <braunr> the thing is, we actually don't care about dropping privileges
+ <braunr> we care about gaining them
+ <teythoon> you cannot drop your identity, you can just use another one
+ <braunr> exactly
+ <braunr> that's what i want
+ <braunr> and the way the hurd does it is superior
+ <braunr> let's keep that
+ <braunr> processes that should run with least privileges can simply have
+ their own user/group as it's done on unix
+ <teythoon> then how do you obtain such a uid/gid?
+ <braunr> teythoon: you gain the right, use it to prove who you can be, and
+ ask an identity switch
+ <braunr> identities would then be managed at server side (in proc for
+ example)
+ <teythoon> I know how it's done on the Hurd, but who creates them for you?
+ <braunr> the password server
+ <braunr> well no
+ <braunr> the password server gives you the right you need to prove who you
+ can be
+ <braunr> then i'd assume you'd ask the proc server for the switch
+ <teythoon> but who creates the uid for you in the first place, who sets up
+ a passwd entry
+ <braunr> the administrator ?
+ <braunr> what bothers me is that it goes directly against the main goal of
+ the hurd
+ <teythoon> indeed
+ <braunr> but i think it's a better compromise of freedom/order
+ <teythoon> I always thought that the ability to drop the unix-like
+ credentials is really nice
+
+
+### IRC, freenode, #hurd, 2013-09-29
+
+ <antrik> braunr: dropping privileges doesn't unregister a process from the
+ proc server... so it shouldn't be too hard to find out who is
+ responsible. however, there are more effective ways to create ghost
+ processes -- a special executable skipping the ordinary bootstrap won't
+ be registered with proc at all...
+ <antrik> however, that would be a non-issue if we had proper resource
+ accounting
+ <teythoon> antrik: I do not believe that this is correct. every mach task
+ will eventually be picked up by the proc server
+ <teythoon> eventually being next time someone fork(2)s or so
+ <teythoon> but if noone uses proc_child to claim the new process ones
+ child, it will not be presented by the proc server as unix process (aiui)
+ <antrik> teythoon: not sure what you mean by "pick up"
+ <antrik> of course proc will see the process when listing all tasks on the
+ system; but it will have no additional knowlegde about it
+ <antrik> (which is the whole purpose of proc)
+
+
+### IRC, freenode, #hurd, 2013-09-30
+
+ <braunr> antrik: proc should be redesigned to fix these issues
+ <braunr> in particular, the way that proc lists mach tasks to show them to
+ the rest of the system is something i find deeply disturbing
+ <braunr> hurd processes should be forced to go through proc
+ <antrik> braunr: well, if we are talking about fundamentally fixing things,
+ I believe the central proc server is not a good idea in the first place
+ :-)
+ <antrik> I believe hierarchical management of resource management and
+ information flow -- cf. nghurd and genode -- is a much better approach
+ <braunr> antrik: i agree with hierarchical management of resources, but i
+ don't see why this prevents a central proc server
+ <braunr> i.e. one proc server per hurd instance
+
+
+### IRC, freenode, #hurd, 2013-10-06
+
+ <antrik> braunr: hierarchical management of resources doesn't prevent a
+ central proc server of course; but a central proc server would fit really
+ ill with hierarchical management of permissions...
+
+
+### IRC, freenode, #hurd, 2013-10-07
+
+ <braunr> antrik: does proc manage permissions ?
+ <antrik> braunr: well, it manages some permissions... like who is allowed
+ to send signals
+ <antrik> hm... or perhaps proc is not involved in signal delivery itself?
+ don't remember. but at any rate, proc decides whether it hands out
+ privileged task ports
+ <braunr> antrik: yes, it decides whether or not a client is allowed to
+ obtain the message port of another task
+ <braunr> antrik: but i don't see why this is a problem
+ <braunr> what we have now is one proc server per hurd instance
+ <braunr> how is that not both central (inside the hurd instance) and
+ hierarchical with regard to resource management ?
+ <antrik> braunr: we are probably talking past each other
+ <antrik> all I'm saying is that in an ideal world, there should not be a
+ central server deciding who is allowed to see and/or control other
+ processes
+ <braunr> antrik: how should it be structured then ?
+ <braunr> i mean, how would you see it ?
+ <antrik> child processes should be fully controlled by their parent --
+ including outside communication
+ <antrik> (like in genode AIUI)
+ <braunr> isn't that conflicting with the unix design ?
+ <kilobug> antrik: maybe I'm saying silly stuff since I don't have all the
+ background, but seems problematic to me with SUID/SGID programs
+ <kilobug> antrik: in which a child can be more privilegied than the parent
+ <braunr> kilobug: that's part of my question too
+ <kilobug> and it's even "worse" with Hurd's addauth in which any process
+ can be given additional rights in runtime, but not its parent
+ <antrik> braunr: one of the conclusions from ngHurd was that suid as such
+ is problematic. it makes more sense to have privileged services actually
+ run by the privileged user, and only requested by the unprivileged one
+ using RPC
+ <antrik> admittedly, I'm not sure how this interacts with UNIX
+ compatibility ;-)
+ <antrik> kilobug: in the genode approach, the parent would control that as
+ well
+ <braunr> in unix, the idea of parent processes doesn't imply much
+ <braunr> parents merely exist to reap resources from their children
+ <braunr> and as templates when forking
+ <antrik> yeah, but that's one of the major shortcomings of UNIX in my
+ book...
+ <braunr> sure
+ <braunr> i'm just thinking out loud
+ <braunr> if we want to maintain posix compatibility, it seems reasonable to
+ keep it that way
+ <braunr> despite the shortcomings
+ <braunr> does that imply a centralized proc server anyway ?
+ <antrik> I suspect we could similate suid in the hierarchical design, only
+ that the resources would be accounted to the user under whose ID the
+ process runs, rather than to the one who invoked it
+ <braunr> i also have a hard time seeing what the completely hierarchical
+ model brings compared to what the hurd does (isolating system instances)
+ <antrik> and I don't think we need a central proc server. it could probably
+ be done similar to the VFS: each process implements the interfaces,
+ passing on to the children as appropriate
+ <antrik> braunr: it's much easier to isolate parts of the system for
+ security and/or customisation
+ <antrik> that's actually one of the things discussed in the "critique"
+ IIRC...
+ <braunr> i'm not sure
+ <braunr> anyway, processes implementing the interface looks bad to me
+ <braunr> that's already a problem with the current hurd
+ <braunr> using normal client processes as servers means we rely on them to
+ behave nicely
+ <antrik> you have a point there: while untrusted filesystems can be ignored
+ easily, ignoring untrusted proc providers would be problematic...
+ <antrik> (I don't think it's strictly necessary to know about other user's
+ processes; but we could hardly keep a UNIX feel without it...)
+ <antrik> other users'
+ <braunr> i feel the hierarchical model may imply some unnecessary burden
+ <braunr> capabilities along with resource containers look much more
+ flexible
+ <braunr> and not less secure
+ <braunr> children would share the same container as their parent by
+ default, unless they obtain the right to use another or create their own
+ <antrik> braunr: decoupling control from resource ownership is *always*
+ problematic. that's pretty much the major takeaway from ngHurd
+ discussions (and the major reason why Coyotos was abandoned as a base for
+ ngHurd)
+ <antrik> if a process runs on your resources, you should have full control
+ over it. anything else faciliates DRM & Co.
+ <braunr> antrik: i see
+ <braunr> nonetheless, i don't see the point of that restriction, since the
+ simplest way to avoid drms in the first place is not using them
+ <braunr> and that restriction makes posix compatibility hard to provide
+ <antrik> I'm not sure it's really all that hard...
+ <antrik> IIRC genode is aiming at POSIX compatibility
+ <antrik> I'm not sure it's any harder than with the current Hurd
+ architecture
+ <braunr> i didn't see anything like that
+ <braunr> they provide posix compatibility by running legacy systems on top
+ of it
+ <braunr> well, namely linux
+ <antrik> hm... they have some UNIX compatibility at least... perhaps not
+ aiming at full POSIX. don't remember the details
+ <antrik> Linux on genode? that's news to me... I know they do run genode on
+ Linux
+ <braunr> anyway, i'll probably stick with the close unix approach for x15
+ <braunr> http://genode.org/documentation/general-overview/
+ <braunr> In an preliminary study, a user-level version of the Linux kernel
+ (L4Linux) was successfully ported to the Genode OS Framework running on a
+ L4 kernel. This study suggests that Genode combined with an appropriate
+ kernel or hypervisor is suited as a virtual-machine hosting platform.
+ <antrik> hm... that's boring though ;-)
+ <braunr> isn't it :)
diff --git a/hurd/console.mdwn b/hurd/console.mdwn
index 55581870..10c74bf9 100644
--- a/hurd/console.mdwn
+++ b/hurd/console.mdwn
@@ -1,5 +1,5 @@
[[!meta copyright="Copyright © 2002, 2003, 2004, 2005, 2006, 2007, 2009, 2011,
-2012 Free Software Foundation, Inc."]]
+2012, 2013 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -37,8 +37,8 @@ where the numbered nodes represent virtual consoles and their contents are all
alike.
As the following graph shows, the console, input and display nodes are the
-interfaces used by the terminal server, input driver and display drivers
-respectively.
+interfaces used by the [[terminal server|translator/term]], input driver and
+display drivers respectively.
+------------------+ +-----------------+
| Input driver | | Terminal Server |
@@ -67,7 +67,8 @@ respectively.
+----------------+ +-----------------+
The input driver takes scancodes from the in-kernel kbd queue, translates them
-into characters and writes them to the input node. Then the terminal server
+into characters and writes them to the input node. Then the [[terminal
+server|translator/term]]
reads the console node taking the characters out of the console server.
Each of theese actions is actually an RPC handled by the translator on
@@ -110,7 +111,8 @@ Additional information about the console can be found in the [Hurd Console Tutor
**_The new Hurd console features:_**
-**A console server**, which provides a number of virtual consoles to term servers, with a full set of terminal capabilities.
+**A console server**, which provides a number of virtual consoles to [[term
+servers|translator/term]], with a full set of terminal capabilities.
The console server supports any encoding supported by iconv, but uses Unicode internally. The default encoding is ISO8859-1, another useful variant is UTF-8.
@@ -307,7 +309,13 @@ If you use mutt, install `mutt-utf8` package. For lynx, edit `/etc/lynx.cfg`, ma
If you use other applications, try to search with google for "application-name utf8" or "application-name unicode". Often you find what you need. The issues are the same for the GNU/Hurd and GNU/Linux systems, so most of the information can be shared, except how to setup the system console to support Unicode, of course.
-The `console-server` watches for new hurdio terms (devices translated with `/hurd/term`) and adds them to `/dev/vcs` automatically. What this means is, if you create a new tty with `MAKEDEV`, and then attach something to it, it will now appear in `/dev/vcs`. When a term is disconnected from, it disappears from `/dev/vcs`. `/libexec/getty` is what is usually attached to a term. You can see this automatic adding and removing of terms from the `console-server` by typing the following:
+The `console-server` watches for new [[hurdio terms (devices translated with
+`/hurd/term`)|translator/term]] and adds them to `/dev/vcs` automatically.
+What this means is, if you create a new tty with `MAKEDEV`, and then attach
+something to it, it will now appear in `/dev/vcs`. When a term is disconnected
+from, it disappears from `/dev/vcs`. `/libexec/getty` is what is usually
+attached to a term. You can see this automatic adding and removing of terms
+from the `console-server` by typing the following:
# cd /dev
# ls vcs/
diff --git a/hurd/console/discussion.mdwn b/hurd/console/discussion.mdwn
index 0022ec23..73d605ed 100644
--- a/hurd/console/discussion.mdwn
+++ b/hurd/console/discussion.mdwn
@@ -48,3 +48,50 @@ License|/fdl]]."]]"""]]
<pinotree> http://xkbcommon.org/ ‘¡û sounds interesting for our console
translator
+
+
+# IRC, freenode, #hurd, 2013-10-01
+
+[[!tag open_issue_hurd]]
+
+ <pinotree> teythoon_: df: `/dev/cons': Operation not supported
+ <pinotree> missing/stub implementation in the console translator?
+
+
+## IRC, freenode, #hurd, 2013-10-02
+
+ <teythoon_> pinotree: yes, df does file_statfs which fails
+
+
+# IRC, freenode, #hurd, 2013-10-22
+
+ <C-Keen> hello hurders! I happened to watch samuel's gnu hackers talk and
+ wanted to start to use the hurd more regularily. However I noticed that
+ when I use the preinstalled image, there seems to be some issue with the
+ console driver
+ <C-Keen> when I start emacs the mode line is drawn 3 times above the bottom
+ of the screen
+ <C-Keen> is this know or did I miss a step in setting it up? Or should I
+ use the debian installer and start from scratch again?
+ <youpi> C-Keen: it's probably unknown, and not an issue on your side. Did
+ you try to upgrade to the latest packages?
+ <C-Keen> youpi: doing that now
+ <C-Keen> my base image is debian-hurd-20130504.img
+ <youpi> still an issue with the latest packages indeed
+ <youpi> it seems emacs and the hurd console don't agree on the number of
+ lines...
+ <youpi> C-Keen: you can set TERM=vt100 to work around the issue
+ <C-Keen> ah alright.
+ <youpi> or TERM=linux
+ <C-Keen> youpi: can you start the emacs in X? I get an empty window here
+ <youpi> I never tried
+ <youpi> I never use emacs :)
+ <C-Keen> I see ;)
+ <youpi> it seems there's a bug in cud1 indeed
+ <C-Keen> what's cud1?
+ <youpi> see man 5 terminfo
+ <braunr> yes it's a terminfo problem
+ <braunr> the hurd console isn't well defined there
+ <youpi> braunr: actually it seems like a bug in emacs
+ <youpi> cud may or may not scroll the screen, depending on the
+ implementation
diff --git a/hurd/libfuse.mdwn b/hurd/libfuse.mdwn
index 78e96022..28125dd9 100644
--- a/hurd/libfuse.mdwn
+++ b/hurd/libfuse.mdwn
@@ -49,6 +49,25 @@ etc.
<braunr> and they could almost readily use our libfuse version
+## IRC, freenode, #hurd, 2013-10-01
+
+ <pinotree> our libfuse implementation is still basic atm (there's a wiki
+ page about it)
+ <alsuren> okay... talk to me about FUSE
+ <pinotree> even with the improvements i have in my public branch, it still
+ cannot do real-world fs'es
+ <alsuren> okay, so you're the person to ask about FUSE
+ <alsuren> it strikes me that HURD not having FUSE support is a bit of an
+ architectural oversight
+ <pinotree> i'm not sure what's your point about fuse, since what fuse on
+ linux (and not only) does is done *natively* by the hurd
+ <alsuren> exactly
+ <pinotree> all of the hurd filesystems (which are just a type of servers)
+ run in userspace already
+ <alsuren> so FUSE should Just Work
+ <pinotree> well no
+
+
# Source
[[source_repositories/incubator]], libfuse/master.
diff --git a/hurd/porting/guidelines.mdwn b/hurd/porting/guidelines.mdwn
index d28a777e..a9acd9f9 100644
--- a/hurd/porting/guidelines.mdwn
+++ b/hurd/porting/guidelines.mdwn
@@ -1,5 +1,5 @@
[[!meta copyright="Copyright © 2002, 2003, 2005, 2007, 2008, 2009, 2010, 2011,
-2012 Free Software Foundation, Inc."]]
+2012, 2013 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -83,6 +83,24 @@ because else *-gnu* would catch i386-pc-linux-gnu for instance...
Note: some of such statements are not from the source package itself, but from aclocal.m4 which is actually from libtool. In such case, the package simply needs to be re-libtoolize-d.
+
+## Preprocessor Define
+
+### IRC, freenode, #hurd, 2013-10-23
+
+ <C-Keen> Is there a preprocessor define gcc sets for hurd which I can check
+ in my code?
+ <braunr> __GNU__
+ <braunr> glibc sets it if i'm right
+ <C-Keen> I also see that __MACH__ gets set
+ <azeem> that's also set on Mac OS X
+ <C-Keen> right, which uncovered a bug in the code
+ <braunr> the microkernel doesn't always implies what operating system runs
+ on top of it
+ <C-Keen> braunr: but __GNU__ is the correct define for hurd specific code?
+ <braunr> yes
+
+
## <a name="Undefined_bits_confname_h_tt_mac"> Undefined `bits/confname.h` macros (`PIPE_BUF`, ...) </a>
If macro `XXX` is undefined, but macro `_SC_XXX` or `_PC_XXX` is defined in `bits/confname.h`, you probably need to use `sysconf`, `pathconf` or `fpathconf` to obtain it dynamicaly.
diff --git a/hurd/running/virtualbox.mdwn b/hurd/running/virtualbox.mdwn
index 822f771d..23a0b156 100644
--- a/hurd/running/virtualbox.mdwn
+++ b/hurd/running/virtualbox.mdwn
@@ -40,7 +40,44 @@ To convert the image you need the VirtualBox package properly installed with a V
If [[QEMU with KVM|qemu]] is not available, VirtualBox reportedly has better
performance.
-IRC, freenode, #hurd, 2011-10-31:
+
+# Open Issues
+
+## IRC, freenode, #hurd, 2011-10-31
<youpi> I don't know what virtualbox does with hardware emulation, but
gnumach is awfully slow to probe things there
+
+
+## IRC, freenode, #hurd, 2013-09-28
+
+ <snadge> the problem is if i giveit more than 1855 it says truncating to
+ that
+ <snadge> so i give it that.. then it has kmem alloc error
+ <snadge> 1536mb same.. 1024 isok
+ <braunr> hum
+ <braunr> that's weird
+ <braunr> virtual box ?
+ <snadge> yeah
+ <snadge> i wonder what cpu features i should enable/disable
+ <snadge> pae ?
+ <braunr> make sure vbox doesn't count on the so called memory balloon
+ <braunr> pae isn't used except on xen
+ <braunr> disable apic
+ <braunr> enable host io cache in disk controllers
+ <youpi> do we have these written on the wiki?
+ <braunr> no because i didn't run into these problems
+ <braunr> but since i know the system well enough to avoid them in the first
+ place ..
+ <braunr> we need real users to report them
+ <braunr> i'm not sure we have anything about vbox in the wiki actually
+ <youpi> ./hurd/running/virtualbox.mdwn
+ <youpi> we seem to have a page at least
+ <snadge> it seems to be okay with 1024MiB
+ <braunr> still weird
+ <braunr> looks more random than buggy with more memory
+ <braunr> do you have the exact error message you got during your previous
+ attempts ?
+ <snadge> no.. i should have taken a screenshot.. its easy enough to
+ reproduce though
+ <snadge> i'll wait until after its installed
diff --git a/hurd/subhurd/discussion.mdwn b/hurd/subhurd/discussion.mdwn
index fac93625..892387ef 100644
--- a/hurd/subhurd/discussion.mdwn
+++ b/hurd/subhurd/discussion.mdwn
@@ -180,3 +180,37 @@ License|/fdl]]."]]"""]]
<braunr> safer
<braunr> perhaps more powerful too, but that entirely depends on the
features you want inside
+
+
+# IRC, freenode, #hurd, 2013-10-04
+
+ <braunr> hm, looks like we broke subhurds again
+ <braunr> freezes after starting exec
+ <teythoon> o_O
+ <braunr> looks like some translator refuses to start
+ <braunr> teythoon: we need better error reporting first :)
+
+[[open_issues/subhurd_error_messages]].
+
+ <braunr> and better visibility in general
+ <braunr> teythoon: it may be that the subhurd i'm using is a bit od
+ <braunr> old
+ <braunr> one weird thing about subhurds is that they actually use the
+ ext2fs and linker from the host
+ <braunr> so it's better if the subhurd and the host uses the same bootstrap
+ protocol :)
+ <teythoon> braunr: isn't boot --boot-root=DIR there to specify which root
+ translator and linker to use?
+ <braunr> teythoon: yes but you don't want your root file system mounted
+ from the host when starting your subhurd
+ <teythoon> you can mount it r/o just fine, no?
+ <braunr> ideally, we'd have a userspace version of grub reading the files
+ from the disk, as it's done when booting
+ <braunr> hm
+ <braunr> right
+
+
+## IRC, freenode, #hurd, 2013-10-07
+
+ <teythoon> braunr: btw, did you straighten out your subhurd issue?
+ <braunr> teythoon: no i haven't
diff --git a/hurd/translator.mdwn b/hurd/translator.mdwn
index 52cd09f7..32562a8b 100644
--- a/hurd/translator.mdwn
+++ b/hurd/translator.mdwn
@@ -106,6 +106,7 @@ The [[concept|concepts]] of translators creates its own problems, too:
* [[symlink]]
* [[firmlink]]
* [[fifo]]
+* [[term]]
* ...
diff --git a/hurd/translator/auth.mdwn b/hurd/translator/auth.mdwn
index 7fd4832c..10cfb3aa 100644
--- a/hurd/translator/auth.mdwn
+++ b/hurd/translator/auth.mdwn
@@ -8,7 +8,8 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
-The *auth server* (or, *authentification server*).
+The *auth server* (or, *authentification server*) is a key component managing
+[[authentication]] in a Hurd system.
It is stated by `/hurd/init`.
diff --git a/hurd/translator/discussion.mdwn b/hurd/translator/discussion.mdwn
index e038ba84..95f5ab0c 100644
--- a/hurd/translator/discussion.mdwn
+++ b/hurd/translator/discussion.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -10,7 +10,8 @@ License|/fdl]]."]]"""]]
[[!tag open_issue_documentation open_issue_hurd]]
-IRC, freenode, #hurd, 2011-08-25:
+
+# IRC, freenode, #hurd, 2011-08-25
< frhodes> how can I replace an existing running server with a new one
without rebooting?
@@ -23,3 +24,24 @@ IRC, freenode, #hurd, 2011-08-25:
nature
< antrik> in some cases, you might even be able simply to remove the old
translator... but obviously only for non-critical stuff :-)
+
+
+# IRC, freenode, #hurd, 2013-10-21
+
+ <braunr> mhmm, there is a problem with thread destruction
+
+[[open_issues/libpthread/t/fix_have_kernel_resources]].
+
+ <braunr> actually, translator self destruction
+ <braunr> if a request arrives after the last thread servicing a port set
+ returns from mach_msg because of a timeout, but before the translator is
+ detached from its parent, the client will get an error
+ <braunr> it should very rarely happen, but if it does, we could face the
+ same kind of issues we have when a server crashes
+ <braunr> e.g. sshd looping over select() returning EBADF, consuming all cpu
+ <braunr> not sure we want to introduce such new issues
+
+ <braunr> i don't think i'll be able to make translators disappear reliably
+ ..
+ <braunr> but at least, thread consumption will correctly decrease with
+ inactivity
diff --git a/hurd/translator/ext2fs.mdwn b/hurd/translator/ext2fs.mdwn
index e2f6b044..cfd09502 100644
--- a/hurd/translator/ext2fs.mdwn
+++ b/hurd/translator/ext2fs.mdwn
@@ -163,6 +163,11 @@ small backend stores, like floppy devices.
<youpi> ok
+#### IRC, freenode, #hurd, 2013-10-08
+
+ <braunr> ogi: your ext2fs patches were finally merged upstream :)
+
+
## Sync Interval
[[!tag open_issue_hurd]]
@@ -209,39 +214,6 @@ That would be a nice improvement, but only after writeback throttling is impleme
<teythoon> tschwinge: well, thanks anyway ;)
-## Increased Memory Consumption
-
-### IRC, freenode, #hurd, 2013-09-18
-
- <braunr> ext2fs is using a ginormous amount of memory on darnassus since i
- last updated the hurd package :/
- <braunr> i wonder if my ext2fs large store patches rework have introduced a
- regression
- <braunr> the order of magnitude here is around 1.5G virtual space :/
- <braunr> it used to take up to 3 times less before that
- <braunr> looks like my patches didn't make it into the latest hurd package
- <braunr> teythoon: looks like there definitely is a new leak in ext2fs
- <teythoon> :/
- <braunr> memory only
- <braunr> the number of ports looks stable relative to file system usage
- <teythoon> braunr: I tested my patches on my development machine, it's up
- for 14 days (yay libvirt :) and never encountered problems like this
- <braunr> i've been building glibc to reach that state
- <teythoon> hm, that's a heavy load indeed
- <teythoon> could be the file name tracking stuff, I tried to make sure that
- everything is freed, but I might have missed something
- <braunr> teythoon: simply running htop run shows a slight, regular increase
- in physical memory usage in ext2fs
- <pinotree> old procfs stikes again? :)
- <teythoon> braunr: I see that as well... curious...
- <braunr> 16:46 < teythoon> could be the file name tracking stuff, I tried
- to make sure that everything is freed, but I might have missed something
- <braunr> how knows, maybe completely unrelated
- <teythoon> the tracking patch isn't that big, I've gone over it twice today
- and it still seems reasonable to me
- <braunr> hm
-
-
# Documentation
* <http://e2fsprogs.sourceforge.net/ext2.html>
diff --git a/hurd/translator/fifo.mdwn b/hurd/translator/fifo.mdwn
index 857922fc..4132e94a 100644
--- a/hurd/translator/fifo.mdwn
+++ b/hurd/translator/fifo.mdwn
@@ -46,3 +46,9 @@ The *fifo* translator implements named pipes (FIFOs).
<pochu> gg0: got an example?
<gg0> http://bugs.debian.org/629184
<gg0> i didn't close it myself
+
+
+## IRC, OFTC, #debian-hurd, 2013-10-04
+
+ <braunr> there is new-fifo, which you can try
+ <braunr> i guess none of us know what it was really meant for
diff --git a/hurd/translator/magic.mdwn b/hurd/translator/magic.mdwn
index 84bacdfb..2b0d1bf7 100644
--- a/hurd/translator/magic.mdwn
+++ b/hurd/translator/magic.mdwn
@@ -1,5 +1,5 @@
-[[!meta copyright="Copyright © 2006, 2007, 2008, 2010 Free Software Foundation,
-Inc."]]
+[[!meta copyright="Copyright © 2006, 2007, 2008, 2010, 2013 Free Software
+Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -9,7 +9,13 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
-The magic translator provides `/dev/fd`.
+The `magic` translator returns magic retry results, which are then resolved by
+[[glibc]]'s *name lookup* routines.
+
+[[!toc]]
+
+
+# `/dev/fd`.
$ showtrans /dev/fd
/hurd/magic --directory fd
@@ -20,3 +26,253 @@ individually like this:
$ ls -l /dev/fd/0
crw--w---- 1 bing tty 0, 0 Nov 19 18:00 /dev/fd/0
+
+
+# `/dev/tty`
+
+ $ showtrans /dev/tty
+ /hurd/magic tty
+
+
+## Open Issues
+
+### IRC, OFTC, #debian-hurd, 2013-06-18
+
+ <XTaran> http://www.zsh.org/mla/workers/2013/msg00547.html
+
+
+#### IRC, OFTC, #debian-hurd, 2013-06-19
+
+ <XTaran> youpi: http://www.zsh.org/mla/workers/2013/msg00548.html -- Is
+ that realistic? If yes, can someone of you test it? I though would expect
+ that if /dev/tty exists everywhere, it's a chardev everywhere, too.
+ <youpi> that's not impossible indeed
+ <youpi> I've noted it on my TODO list
+
+
+#### IRC, OFTC, #debian-hurd, 2013-06-20
+
+ <pinotree> youpi: wrt the /dev/tty existance,
+ https://buildd.debian.org/status/fetch.php?pkg=mksh&arch=hurd-i386&ver=46-2&stamp=1371553966
+ <pinotree> For the build logs, demonstrate that /dev/null and /dev/tty
+ exist:
+ <pinotree> ls: cannot access /dev/tty: No such device or address
+ <youpi> uh?!
+ <youpi> ah, ENODEV
+ <youpi> so that's what we was thinking, no tty -> no /dev/tty
+
+
+#### IRC, OFTC, #debian-hurd, 2013-09-20
+
+ <XTaran> Hi. zsh still FTBFS on Hurd due to some test failure:
+ https://buildd.debian.org/status/package.php?p=zsh -- IIRC I checked last
+ time on some porterbox and couldn't reproduce the failure there. Any
+ insight if /dev/tty is not accessible on the buildds inside the chroot?
+ Or is it no character device there? I checked on strauss and there it is
+ a character device.
+ <XTaran> My only other option to debug this (didn't think of that yesterday
+ before the upload unfortunately) would be to override dh_auto_test with
+ "ls -l /dev/tty; dh_auto_test". Do you think that would be helpful?
+ <pinotree> i see /dev/tty on exodar, in the root system and in the chroot
+ <XTaran> pinotree: And it is a character device?
+ <XTaran> ... in both cases?
+ <pinotree> crw--w---- 1 pino tty 0, 0 Sep 20 10:20 /dev/tty
+ <pinotree> yes
+ <XTaran> pinotree: Hrm.
+ <pinotree> (/dev in the chroot is a firmlink to the system /dev, iirc)
+ <XTaran> pinotree: What is a firmlink? :)
+ <XTaran> pinotree: /dev/tty belongs to your user in the example above.
+ <pinotree> something between a (sym)link and an union mount
+ <XTaran> pinotree: Is it possible that /dev/tty is not visible if the
+ buildd runs without a connected terminal?
+ <pinotree> that i'm not sure
+ <XTaran> I see.
+ <pinotree> wouldn't it be possible to skip only that check, instead of the
+ whole test suite?
+ <pinotree> maybe something like
+ <pinotree> tty=$(find /dev/ -name 'tty*' -type c -print)
+ <pinotree> if [[ -n $tty ]]; then / [[ -c $tty[(f)1] && ! -c $zerolength ]]
+ / else / print -u$ZTST_fd 'Warning: Not testing [[ -c tty ]] (no tty
+ found)' / [[ ! -c $zerolength ]] / fi
+ <pinotree> (never used zsh, so please excuse me if i wrote something silly
+ above)
+ <XTaran> re
+ <XTaran> pinotree: Yeah, sure. That would be one way to get the thing
+ building again, if that's really the cause.
+ <pinotree> i guess it would find any of the available tty* devices
+ <pinotree> it does that for block devices, why not with tty devices, after
+ all? :)
+ <XTaran> pinotree: I just wonder if the failing test is because the test
+ doesn't work properly on that architecture or because it indicates that
+ there is a bug in zsh which only is present on hurd.
+ <pinotree> wouldn't the change proposed above help in determining it?
+ <XTaran> If I'm sure that it's a broken test, I'll try to disable that
+ one. If not I'd report (more details) to upstream. :)
+ <XTaran> pinotree: Oh, indeed.
+ <pinotree> if you get no warning, then a tty device was found with find
+ (using its -type c option), so the failing condition would be a zsh (or
+ maybe something in the stack below) bug
+ <pinotree> with the warning, somehow there were no tty devices available,
+ hence nothing to test -c with
+ <XTaran> So basically doing a check with dash to see if we should run the
+ zsh test.
+ <pinotree> dash?
+ <XTaran> Well, whatever /bin/sh points to. :)
+ <pinotree> ah, do you mean because of $(find ...)?
+ <XTaran> Ah, right, -type c is from find not /bin/sh
+ <XTaran> pinotree: That's my try:
+ http://anonscm.debian.org/gitweb/?p=collab-maint/zsh.git;a=commitdiff;h=ba5c7320d4876deb14dba60584fcdf5d5774e13b
+ <pinotree> o_O
+ <pinotree> isn't that a bit... overcomplicated?
+ <XTaran> pinotree: Yeah, it's a little bit more complicated as the tests
+ itself are not pure shell code but some format on their own.
+ <pinotree> why not the "thing" i wrote earlier?
+ <XTaran> pinotree: Actually it is what I understand you wanted to do, just
+ with more debug output. Or I dunderstood
+ <XTaran> pinotree: Actually it is what I understand you wanted to do, just
+ with more debug output. Or I understood your thing wrongly.
+ <pinotree> <pinotree> tty=$(find /dev/ -name 'tty*' -type c -print)
+ <pinotree> <pinotree> if [[ -n $tty ]]; then / [[ -c $tty[(f)1] && ! -c
+ $zerolength ]] / else / print -u$ZTST_fd 'Warning: Not testing [[ -c tty
+ ]] (no tty found)' / [[ ! -c $zerolength ]] / fi
+ <XTaran> pinotree: Yeah, I know.
+ <pinotree> that is, putting these lines instead of the current two
+ tty=/dev/tty + following
+ <pinotree> imho that should be fit for upstream
+ <XTaran> pinotree: You mean inside C02cond.ztst?
+ <pinotree> yep
+ <XTaran> pinotree: No, IMHO that's a bad idea.
+ <pinotree> why?
+ <XTaran> pinotree: That file is to test the freshly compiled zsh. I can't
+ rely on their code if I'm testing it.
+ <pinotree> uh?
+ <pinotree> the test above for -b is basically doing the same
+ <XTaran> pinotree: Indeed. Hrm.
+ <pinotree> that's where i did c&p most of it :)
+ <XTaran> So upstream relies on -n in the testsuite before it has tested it?
+ Ugly.
+ <pinotree> if upstream does it, why cannot i too? :D
+ <XTaran> pinotree: You've got a point there.
+ <XTaran> Ok, rethinking. :)
+ <pinotree> otoh you could just move the testcase for -n up to that file, so
+ after that you know it works already
+ <XTaran> pinotree: Well, if so, upstream should do that, not me. :)
+ <pinotree> you could suggest them to, given the -n usage in the -b testcase
+ <XTaran> pinotree: Looks alphabetically sorted, so I guess that's at least
+ not accidentially.
+ <XTaran> pinotree: Ok, you've convinced me. :)
+ <pinotree> :D
+ <XTaran> Especially because this is upstream-suitable once it proved to fix
+ the Hurd FTBFS. :)
+ <XTaran> pinotree: The previous upstream code (laast change 2001) instead
+ of the hardcoded /dev/tty was btw "char=(/dev/tty*([1]))", so I suspect
+ that the find may work on Cygwin, too.
+ <XTaran> s/aa/a/
+ <pinotree> ah, so that's that comment about globbing on cygwin was
+ referring to
+ <XTaran> Yep
+ <pinotree> cool, so incidentally i've solved also that small issue :9
+ <pinotree> :)
+ <XTaran> pinotree: I hope so. :)
+ <XTaran> Then again, I hope, external commands like find are fine for
+ upstream.
+ <pinotree> then they should rework the already existing testcases ;)
+ <XTaran> pinotree: Ah, I fall again for the same assumptions. :)
+ <XTaran> Seems as I would really build test suites with a different
+ approach. :)
+ <pinotree> nothing bad in that, i'd say
+ <XTaran> I'd try to make the tests as far as possible independent from
+ other tools or features to be sure to test only the stuff I want to test.
+ <XTaran> Warning: Not testing [[ -c tty ]] (no tty found)
+ <XTaran> Interesting. I didn't expect that outside a chroot. :)
+ <pinotree> where's that?
+ <XTaran> pinotree: A plain "debuild on my Sid VM.
+ <pinotree> ah
+ <XTaran> Linux, amd64
+ <XTaran> (and Debian of course ;-)
+ <XTaran> pinotree: Ah, my fault, I kept upstreams char= but didn't change
+ it in your code. :)
+ <pinotree> hehe
+ <XTaran> pinotree: Will be included in the next zsh upload. But I don't
+ want to upload a new package before the current one moved to testing (or
+ got an RC bug report to fix :-)
+ <pinotree> oh sure, that's fine
+ <XTaran> pinotree:
+ http://anonscm.debian.org/gitweb/?p=collab-maint/zsh.git;a=commitdiff;h=22bc9278997a8172766538a2ec6613524df03742
+ <XTaran> (I've reverted my previous commit)
+ <pinotree> \o/
+
+
+#### IRC, OFTC, #debian-hurd, 2013-09-30
+
+ <XTaran> Anyone knows why the building of zsh on ironforge restarted? It
+ was at something like "building 4h20m" when I looked last and it now is
+ at "building 1h17m" but there's no old or last log, so it does still look
+ like the first build.
+ <pinotree> most probably got stuck
+ <XTaran> Oh, ok.
+ <XTaran> pinotree: So there are cases where the log is not kept?
+ <pinotree> looks so
+ <youpi> when the machine crashes, yes :)
+ <XTaran> youpi: Ooops. Was that me?
+ <youpi> no, I just rebooted the box
+ <youpi> I didn't easily find which process to kill
+ <XTaran> Ok. Then I'll check back tomorrow morning if pinotree's fix for
+ zsh's test suite on hurd worked. :)
+ <youpi> it seems to be hung on
+ /build/buildd-zsh_5.0.2-5-hurd-i386-vO9pnz/zsh-5.0.2/obj/Test/../Src/zsh
+ <youpi> ../Src/zsh ../../Test/ztst.zsh ../../Test/Y02compmatch.ztst
+ <XTaran> :(
+ <XTaran> At least pinotree's patch worked as it then likely passed
+ C02cond.ztst. :)
+ <XTaran> youpi: For how long? There are multiple tests which take at least
+ 3 seconds per subtest.
+ <youpi> one hour already
+ <XTaran> Ok.
+ <XTaran> That's far too long
+
+
+#### IRC, OFTC, #debian-hurd, 2013-10-01
+
+ <XTaran> pinotree: I've just checked
+ https://buildd.debian.org/status/fetch.php?pkg=zsh&arch=hurd-i386&ver=5.0.2-5&stamp=1380608100
+ manually: Your fix unfortunately seemed not to help, but another test
+ failed, too, and that one came later and was hence suspected as primary
+ failing issue.
+ <XTaran> pinotree: But "+ find: `/dev/tty': No such device or address"
+ gives some hint. I just have no idea, why find issues that message.
+ * XTaran really wonders how that message can be caused.
+ <XTaran> So find sees /dev/tty, but gets an error if it tries to access
+ (maybe only stat) it while not being connected to a terminal.
+ <XTaran> Bingo: This reproduces the issue (note the missing -t option to
+ ssh): ssh exodar.debian.net "find /dev/ -nowarn -maxdepth 1 -name 'tty*'
+ -type c -ls"
+ <XTaran> Even clearer: $ ssh exodar.debian.net "ls -l /dev/" | grep 'tty$'
+ <XTaran> ls: cannot access /dev/tty: No such device or address
+ <XTaran> ?????????? ? ? ? ? ? tty
+ <XTaran> I'd say this is a bug somewhere deep down, either in libc or the
+ kernel.
+ <pinotree> or in the console translator
+ <XTaran> pinotree: Never heard of that so far. :)
+ <XTaran> pinotree: Someone from zsh upstream suggests to use /dev/null or
+ /dev/zero instead of /dev/tty* -- will try that for the next upload.
+ <pinotree> ah right, /dev/null should be standard POSIX
+ <XTaran> I hope so. :)
+ <pinotree> http://pubs.opengroup.org/onlinepubs/9699919799/ check in POSIX
+ <pinotree> in any case, sorry for the troubles it is giving you...
+ <XTaran> pinotree: I'm more concerned about the hanging second test. I
+ think I can get that test working with using /dev/null.
+ <XTaran> Now that I've understood why the original test is failing.
+ <XTaran> pinotree: Shall I write a bug report for that issue? If so,
+ against which package?
+ <pinotree> XTaran: not sure it is worth at this stage, having a clearer
+ situation on what happens could be useful
+ <pinotree> it is something that can happen sporadically, though
+ <XTaran> pinotree: Well, it seems a definitely unwanted inconsistency
+ between what the directory listing shows and which (pseudo) files are
+ accessible. Independently of where the bug resides, this needs to be
+ fixed IMHO.
+ <pinotree> sure, nobody denies that
+ <XTaran> pinotree: I'd call it easily reproducible. :)
+ <pinotree> not really
+ <XTaran> ... once you know where to look for.
diff --git a/hurd/translator/mtab/discussion.mdwn b/hurd/translator/mtab/discussion.mdwn
index 0734e1e6..973fb938 100644
--- a/hurd/translator/mtab/discussion.mdwn
+++ b/hurd/translator/mtab/discussion.mdwn
@@ -2103,7 +2103,245 @@ In context of [[open_issues/mig_portable_rpc_declarations]].
<youpi> anyway, got to run
-## IRC, freenode, #hurd, 2013-09-20
+## Memory Leak
+
+### IRC, freenode, #hurd, 2013-09-18
+
+ <braunr> ext2fs is using a ginormous amount of memory on darnassus since i
+ last updated the hurd package :/
+ <braunr> i wonder if my ext2fs large store patches rework have introduced a
+ regression
+ <braunr> the order of magnitude here is around 1.5G virtual space :/
+ <braunr> it used to take up to 3 times less before that
+ <braunr> looks like my patches didn't make it into the latest hurd package
+ <braunr> teythoon: looks like there definitely is a new leak in ext2fs
+ <teythoon> :/
+ <braunr> memory only
+ <braunr> the number of ports looks stable relative to file system usage
+ <teythoon> braunr: I tested my patches on my development machine, it's up
+ for 14 days (yay libvirt :) and never encountered problems like this
+ <braunr> i've been building glibc to reach that state
+ <teythoon> hm, that's a heavy load indeed
+ <teythoon> could be the file name tracking stuff, I tried to make sure that
+ everything is freed, but I might have missed something
+ <braunr> teythoon: simply running htop run shows a slight, regular increase
+ in physical memory usage in ext2fs
+ <pinotree> old procfs stikes again? :)
+ <teythoon> braunr: I see that as well... curious...
+ <braunr> 16:46 < teythoon> could be the file name tracking stuff, I tried
+ to make sure that everything is freed, but I might have missed something
+ <braunr> how knows, maybe completely unrelated
+ <teythoon> the tracking patch isn't that big, I've gone over it twice today
+ and it still seems reasonable to me
+ <braunr> hm
+
+
+### IRC, freenode, #hurd, 2013-09-25
+
+ <braunr> seems like a small leak per file access
+ <braunr> but htop makes it obvious because it makes lots of them
+ <braunr> shouldn't be too hard to find
+ <braunr> since it might also come from the large store patch, i'll take a
+ look at it
+
+
+### IRC, freenode, #hurd, 2013-09-27
+
+ <braunr> teythoon: found the leak :)
+ <braunr> although its origin is weird
+ <teythoon> braunr: where is it?
+ <braunr> i'm still building packages to make sure that's it
+ <braunr> see
+ http://darnassus.sceen.net/gitweb/savannah_mirror/hurd.git/blob/HEAD:/libdiskfs/dir-lookup.c
+ <braunr> which you changed in
+ http://darnassus.sceen.net/gitweb/savannah_mirror/hurd.git/commit/06d49cdadd9e96361f3fe49b9c940b88bb869284
+ <braunr> line 306 is "return error" instead of "goto out"
+ <braunr> has been so since 1994
+ <braunr> what is unclear is why this code path is now run
+ <braunr> patch is here:
+ http://darnassus.sceen.net/~rbraun/0001-Fix-memory-leak-in-libdiskfs.patch
+ <teythoon> I see, weird indeed
+ <braunr> teythoon: the system also feels slower somehow
+ <braunr> such errors might have introduced unexpected retries
+ <teythoon> i think it's possible to write a coccinelle patch to find such
+ errors
+
+
+### IRC, freenode, #hurd, 2013-09-28
+
+ <youpi> braunr: bah, I havent noticed the leak on my box, even after
+ building eglibc & hurd several times
+ <braunr> that's weird
+ <braunr> are you sure it's up to date ?
+ <braunr> also, is procfs correctly attached to /proc ?
+ <braunr> that's what seems to trigger it
+ <youpi> yes, 20130924-2, with procfs on /proc
+
+ <teythoon> braunr: that turned out to be the leak indeed? and somehow my
+ changes triggered it? did you discover why?
+ <braunr> teythoon: yes, yes, no
+ <braunr> but youpi didn't see the leak on his system
+ <teythoon> ^^ cool that you found it
+ <teythoon> I did
+ <braunr> oh yes you mean you saw the leak
+ <teythoon> yes
+
+
+### IRC, freenode, #hurd, 2013-10-01
+
+ <braunr> the fix i did in libdiskfs might have fixed other issues
+ <braunr> apparently, it's the code path taken when error isn't ENOENT,
+ including no error (translator started)
+ <pinotree> the memory leak fix, you mean?
+ <braunr> yes
+ <braunr> it might haved fixed reference counting too
+ <braunr> although i'm not sure if we actually ever run into that issue in
+ the past
+ <braunr> the weird thing is, that path is taken when starting a passive
+ translator
+ <braunr> (i think)
+ <braunr> (it might be any kind of translator, and just doing nothing if
+ alcready active)
+ <braunr> already*
+ <braunr> anyway, the fact that the leak was so visible means this code was
+ run very often
+ <braunr> which doesn't make sense
+ <braunr> hm ok, it seems that code was run every time actually
+ <braunr> but the leak became visible when it concerned memory
+ <pinotree> which side-effects did the old code produce?
+ <braunr> teythoon added a dynamically allocated path that wasn't freed
+ <braunr> reference leaks
+ <braunr> which might explain the assertion on reference we sometimes see
+ with ext2fs
+ <braunr> when a counter overflows and becomes 0
+
+[[open_issues/ext2fs_libports_reference_counting_assertion]]?
+
+ <pinotree> hmm
+ <braunr> which is why i'm mentioning it
+ <braunr> :)
+ <braunr> i'll try to reproduce the assertion
+ <pinotree> libdiskfs/node-drop.c: assert (np->dn_stat.st_size == 0); ←
+ this one?
+ <braunr> yes
+ <braunr> hm no
+ <pinotree> oho
+ <braunr> no, not that one
+ <pinotree> no-oho
+ <braunr> well maybe by side effect
+ <braunr> but i doubt it
+ <pinotree> iirc you constantly get that when building ustr
+ <braunr> (e.g., because the object was freed and reallocated quickly,
+ st_size has been reset, something like that)
+ <braunr> is ustr a package ?
+ <pinotree> yes
+ <braunr> ok
+ <braunr> thanks
+ <braunr> pinotree: indeed, it's still present
+ <braunr> pinotree: actually, after a more in-depth look, reference counting
+ looks valid before the fix too
+ <pinotree> ok, thanks for checking
+ <braunr> pinotree: the assertion affects the root translator, and is
+ triggered by a test that stresses memory
+ <pinotree> memory as in ram, or as in disk storage?
+ <braunr> malloc
+ <pinotree> ok
+ <braunr> i suspect the code doesn't handle memory failure well
+ <pinotree> iirc the ustr tests are mostly disk-intensive
+ <braunr> this one is really about enonmem
+ <braunr> enomem
+ <braunr> i'll make ext2fs print a stack trace
+ <pinotree> (might be wrong, but did not investigate further, sorry)
+ <braunr> no worries
+ <braunr> i'm doing it now :)
+
+
+### IRC, freenode, #hurd, 2013-10-02
+
+ <braunr> i've traced the problem up to truncate
+ <braunr> which gets a negative size
+ <braunr> shouldn't take long to find out where it comes from now
+ <pinotree> it seems our truncate doesn't handle negative values well though
+ <braunr> EINVAL The argument length is negative or larger than the
+ maximum file size.
+ <braunr> i still have to see whether it comes from the user (unlikely) or
+ if it's an internal inconsistency
+ <braunr> i suspect some code wrongly handles vm_map failures
+ <braunr> leading to that inconsistency
+ <braunr> pinotree: looks like glibc doesn't check for length >= 0
+ <pinotree> yeah
+ <braunr> servers should do it nonetheless
+ <pinotree> should we fix glibc, libdiskfs/libnetfs/libtrivfs/etc, or both?
+ <braunr> it appears a client does the truncate
+ <braunr> i'd say both
+ <braunr> can you take the glibc part ? :)
+ <pinotree> i was going to do the hurd part... :p
+ <pinotree> ok, i'll pick libc
+ <braunr> well i'm doing it already
+ <braunr> i want to write a test case first
+ <braunr> to make sure that's the problem
+ <pinotree> already on the hurd part, you mean?
+ <braunr> yes
+ <pinotree> ok
+ <braunr> ok looks like it
+ <pinotree> would you share the test you are doing, so i don't need to write
+ it again? :)
+ * pinotree lazy
+ <braunr> :)
+ <braunr> as soon as darnassus is restarted
+ <pinotree> ideally we could have some repository with all the testcases
+ written over time to fix bugs in implementations/compatibility/etc
+ <braunr> i noticed the system doesn't automatically reboot when e2fsck says
+ reboot, and no unexpected inconsistency was found
+ <braunr> is that normal ?
+ <pinotree> or having something like posixtestsuite, but actively maintained
+ <braunr> pinotree: polishing the test before sending it
+ <pinotree> sure, no hurry :)
+ <braunr> i can't reproduce the assertion but it does make ext2fs freeze
+ <braunr> pinotree: http://darnassus.sceen.net/~rbraun/test_ftruncate.c
+ <pinotree> merci
+ <braunr> pinotree: ustr builds
+ <pinotree> wow
+ <braunr> the client code (ustr) seems to perform a ftruncate with size
+ ((size_t)-1) whereas lengths are signed ..
+ <braunr> i'll check other libraries and send a patch soon
+
+ <teythoon_> braunr: btw, did you fix the leak?
+ <braunr> yes
+ <braunr>
+ http://darnassus.sceen.net/gitweb/savannah_mirror/hurd.git/commit/a81c0c28ea606b0d0a2ad5eeb74071c746b7cdeb
+ <braunr> 1h after tagging 0.5 (
+ <braunr> :(
+ <teythoon> ah yes, I've seen that commit
+ <teythoon> I just wanted to know whether this settled the issue
+ <braunr> it does :)
+ <teythoon> good
+ <braunr> i still can't figure out why youpi didn't had it
+ <braunr> the code path is run when no error (actually error != ENOENT)
+ <braunr> which explains why the leak was so visible
+ <teythoon> so my patch exposed this b/c of the allocation I added, makes
+ sense
+ <teythoon> it's funny actually, b/c this wasn't an issue for me as well, I
+ had my development vm running on that patches for two weeks
+
+
+### IRC, freenode, #hurd, 2013-10-03
+
+ <braunr> youpi: i've committed a fix to hurd that checks for negative sizes
+ when truncating files
+ <braunr> this allows building the ustr package without making ext2fs choke
+ on an assertion
+ <braunr> pinotree is preparing a patch for glibc
+ <braunr> see truncate/ftruncate
+ <braunr> with an off_t size parameter, which can be negative
+ <braunr> EINVAL The argument length is negative or larger than the
+ maximum file size.
+ <braunr> hurd servers were not conforming to that before my change
+
+
+## Multiple mtab Translators Spawned
+
+### IRC, freenode, #hurd, 2013-09-20
<braunr> teythoon: how come i see three mtab translators running ?
<braunr> 6 now oO
@@ -2113,10 +2351,250 @@ In context of [[open_issues/mig_portable_rpc_declarations]].
<braunr> teythoon: more bug fixing for you :)
-## IRC, freenode, #hurd, 2013-09-23
+### IRC, freenode, #hurd, 2013-09-23
<teythoon> so it might be a problem with either libnetfs (which afaics has
never supported passive translator records before) or procfs, but tbh I
haven't investigated this yet
[[open_issues/libnetfs_passive_translators]].
+
+### IRC, freenode, #hurd, 2013-09-26
+
+ <braunr> teythoon: hum, i just saw something disturbing
+ <braunr> teythoon: to isolate the leak, i created my own proc directory
+ <braunr> and the mtab translators it spawns seem to be owned by root oO
+ <teythoon> braunr: but how is that possible? are you sure? have you checked
+ with 'ids'?
+ <braunr> no i'm not sure
+ <braunr> also, ext2fs seems to ignore --writable when started as a passive
+ translator
+ <braunr> < teythoon> braunr: but how is that possible?
+ <braunr> messup with passive translators i guess
+ <braunr> teythoon: actually, it looks like it has effective/available id
+ <braunr> it has no*
+ <braunr> this feature doesn't map well in unix
+ <teythoon> braunr: ah yes, htop doesn't handle this well and shows root
+ indeed, our ps shows - as username
+ <braunr> yes
+
+
+### [[!debbug 724868]]
+
+
+### IRC, freenode, #hurd, 2013-10-03
+
+ <braunr> i can't manage to find out where the hurd stores information about
+ active translators ...
+ <braunr> there is this transbox per node
+ <braunr> but where are nodes stored ?
+ <braunr> what if they are are dropped ?
+ <pinotree> braunr: iirc, see libfshelp
+ <braunr> well i have
+ <braunr> i still can't find it
+ <braunr> i fear that it works for ext2fs because that particular translator
+ implements a cache of open nodes
+ <braunr> whereas things like procfs drop and recreate nodes per open
+ <braunr> which would be the root cause for the multiple mtab bug
+ <pinotree> doesn't tmpfs support translators?
+ <braunr> good idea
+ <braunr> although it's still a libdiskfs based one
+ <braunr> no problem for tmpfs, so it would be a netfs/procfs issue
+ <braunr> better than what i feared :)
+ <braunr> now, how is libdiskfs able to find active translators ..
+ <braunr> ah, there is a name cache in libdiskfs ..
+ <braunr> nope, looks fine
+
+
+### IRC, freenode, #hurd, 2013-10-04
+
+ <braunr> nodes with a translator seem to keep a reference in libdiskfs and
+ not in libnetfs
+ <braunr> mhmmpf
+ <braunr> oh great ..
+ <braunr> each libdiskfs that "works" seems to implement its own
+ diskfs_cached_lookup function
+ <braunr> so both ext2fs and tmpfs actually maintain a list of nodes,
+ keeping a reference on those with a translator
+ <braunr> while procfs simply doesn't
+ <braunr> teythoon: ^
+ <braunr> *sigh*
+ <teythoon> braunr: ok, thanks, I'll look into that
+ <braunr> i'm not sure how to fix it
+ <braunr> we can either fix node destruction to cleanly shut down
+ translators
+ <braunr> but this would mean starting mtab on each access
+ <braunr> or we could implement a custom cache in procfs
+ <braunr> or perhaps a very custom change in the lookup callback for mounts
+ <braunr> i'll try the latter
+ <teythoon> err, shouldn't we try to fix this in lib*fs?
+ <braunr> unless you really want to work on it
+ <braunr> i dont' know
+ <teythoon> ah, so the node is destroyed but the translator is kept running?
+ that's what you mean by the above?
+ <teythoon> and ext2fs makes an effort of killing it in its node cleanup
+ code?
+ <braunr> yes
+ <braunr> grmbl, i'm lagging a lot
+ <braunr> i'm not sure
+ <braunr> ext2fs maintains it
+ <braunr> with ext2fs, translators can only be explicitely removed
+ <braunr> i mean, ext2fs keeps all node descriptors alive once accessed
+ <braunr> while procfs doesn't
+ <braunr> teythoon: ok, looks like i have a working patch that merely caches
+ the node for mounts
+ <braunr> libnetfs suffers from the same leak as libdiskfs when looking up a
+ translator
+ <braunr> i'll fix it too
+
+ <braunr> i installed my fixed procfs on darnassus, only one mtab :)
+ <teythoon> nice :)
+ <braunr> now, why is there no /home in df output ?
+ <teythoon> not sure
+ <teythoon> note how /dev/tty* end up in /proc/mounts, those are passive
+ translators too, no?
+ <braunr> yes
+ <braunr> but that's a good thing i guess
+ <braunr> or was mounts intended for file systems only ?
+ <braunr> well, in the unix traditional meaning
+ <teythoon> I think its nice too, yes
+ <teythoon> but why are they fine and your /home is not...
+ <braunr> that's weirder
+ <braunr> also, mounts actually doesn't show passive translators
+ <braunr> teythoon: does your code perform any kind of comparison ?
+ <braunr> i see /servers/socket/26 but not /servers/socket/2
+ <braunr> s/comparison/filter/g
+ <teythoon> hmm
+ <teythoon> well, yes, try /hurd/mtab --insecure /
+ <teythoon> (I cannot connect to darnassus from here...)
+ <braunr> ok but that looks unrelated
+ <braunr> both /servers/socket/26 and /servers/socket/2 refer to the same
+ translator
+ <braunr> i was wondering if mtab was filtering similar entries based on
+ that
+ <teythoon> no
+ <braunr> that's weird too then, isn't it ?
+ <teythoon> yes ;)
+ <braunr> ok
+ <teythoon> btw, how is that done with the same traanslator being bound to
+ two nodes? settrans cannot do that, can it?
+ <braunr> no it can't
+ <braunr> the translator does it when started
+ <teythoon> ah
+ <braunr> (which means there is a race if both are started simulatneously,
+ although it's very rare and not hard to solve)
+ <teythoon> a weird beaving translator then :)
+
+ <braunr> i have a fix for the multiple mtab issue, will send a patch
+ tonight
+
+ <braunr> teythoon: if ext2fs is set active, mtab output reports it
+
+ <braunr> teythoon: looks like this bug is what allows mtab not to deadlock
+ <braunr> teythoon: when i attach it as an active translator, cat freezes
+
+ <braunr> teythoon: if (control && control->pi.port_right == fsys)
+ <braunr> that's the filtering i was previously talking about
+ <braunr> oh please don't name global variables "path" ...
+
+ <braunr> youpi: i fixed procfs on ironforge and exodar to be started as
+ procfs -c -k 3
+ <braunr> without -k 3, many things as simple as top and uptime won't work
+
+
+### IRC, freenode, #hurd, 2013-10-06
+
+ <antrik> teythoon: pty-s also bind to two nodes, not only pfinet
+
+
+### IRC, freenode, #hurd, 2013-10-07
+
+ <braunr> teythoon: please tell us when you're available, we need to work
+ out the last mtab issues
+ <teythoon> braunr: I'm available now :)
+ <teythoon> I'm sorry, I've been very busy the last two weeks, but I've
+ plenty of time now
+ <braunr> great :)
+ <braunr> did you see youpi's mail ?
+ <braunr> i have the exact same question
+ <teythoon> I did
+ <braunr> it seems your code registers active translators
+ <braunr> but parent translators don't seem to register them when they're
+ created from passive translators
+ <braunr> or am i mistaken ?
+ <teythoon> I'll need a moment to get my hurd machine and myself up to
+ speed...
+ <teythoon> braunr: I concur with youpi, hooking into fshelp_fetch_root
+ should do just fine
+ <teythoon> I'll just try that
+ <braunr> ok
+ <braunr> how do you deal with mtab reporting itself ?
+ <teythoon> o_O does it do that?
+ <braunr> no, but it should
+ <braunr> when i set it as an active translator, i get a deadlock
+ <braunr> hm
+ <braunr> teythoon: before you change libfshelp, i'd like you to try
+ something else
+ <braunr> use more appropriate names for global variables in mtab.c
+ <braunr> in particular, the variable path clashes with local names
+ <teythoon> noted
+ <braunr> teythoon: as a side note (i'm not asking to rewrite anything)
+ <braunr> i strongly recommend a very explicit object oriented style of
+ coding
+ <braunr> (or data-oriented as it's sometimes called)
+ <braunr> use prefixes for all your interfaces so they can be made public if
+ needed (which acts as a namespace and avoids lots of collisions
+ naturally)
+ <braunr> use "constructors" and "destructors" (functions that both allocate
+ and initialize)
+ <braunr> this helps avoiding leaks a lot too
+ <teythoon> hm, I thought I did that, could you be more specific?
+ <braunr> ok didn't see the comment
+ <braunr> /* XXX split up */ error_t mtab_populate (...
+ <braunr> :)
+ <braunr> as a better example, see your code in libfshelp/translator-list.c
+ <braunr> struct translator should have been treated as an object
+ <braunr> this would probably have completely avoided any leaks in the first
+ place
+ <teythoon> braunr: right, I deviated from that style there
+ <braunr> teythoon: these are minor details, don't mind them too much, i
+ just find it helps me a lot
+ <teythoon> braunr: sure, I appreciate the feedback :)
+
+
+### IRC, freenode, #hurd, 2013-10-08
+
+ <teythoon> braunr: I'm on to the passive translator not getting registered
+ issue
+ <teythoon> however, removing them from the list if the active translator is
+ killed does not work as expected... I still need to fiddle with the
+ notifications to get this right
+ <braunr> ok
+
+
+### IRC, freenode, #hurd, 2013-10-16
+
+ <teythoon> braunr: btw, I fixed the 'passive translator not showing up in
+ proc/mounts'-issue
+ <teythoon> but 4 ports do leak each time a translator is killed and
+ reinstalled
+ <teythoon> this happens with passive ones as well as active ones
+ <braunr> teythoon: is that issue tied to your changed ?
+ <braunr> changes*
+ <teythoon> I'm not sure tbh, testing that is on my list of things to do
+ <braunr> ok
+ <braunr> first thing to know i guess
+ <teythoon> yes
+
+
+## Memory Leak in `translator_ihash_cleanup`
+
+### IRC, freenode, #hurd, 2013-10-04
+
+ <braunr> teythoon: isn't there a leak in translator_ihash_cleanup ?
+ <teythoon> braunr: looks like, yes
+ <teythoon> braunr: I probably forgot to add the free (element->name) when I
+ added the name field
+ <braunr> teythoon: ok
+ <braunr> teythoon: i let you fix that :p
+ <teythoon> braunr: sure ;)
diff --git a/hurd/translator/proc.mdwn b/hurd/translator/proc.mdwn
index d5e0960c..75bfb8fd 100644
--- a/hurd/translator/proc.mdwn
+++ b/hurd/translator/proc.mdwn
@@ -63,6 +63,35 @@ It is stated by `/hurd/init`.
something special
+## IRC, freenode, #hurd, 2013-09-25
+
+ <braunr> so nice to finally see proc in top :)
+ <braunr> hm cute, htop layout has become buggy, top just won't start
+ <teythoon> braunr: make sure your procfs knows the correct kernel pid
+ <teythoon> # showtrans /proc
+ <teythoon> /hurd/procfs -c -k 3
+ <teythoon> we could have handled this nicer if procfs were integrated
+ upstream
+ <teythoon> we should probably just update the default
+ <braunr> teythoon: mhm
+ <braunr> $ fsysopts /proc
+ <braunr> /hurd/procfs --stat-mode=444 --fake-self=1
+ <braunr> $ showtrans /proc
+ <braunr> /hurd/procfs -c
+ <pinotree> -c == --stat-mode=444 --fake-self=1
+ <braunr> better indeed
+ <braunr> teythoon: thanks
+
+
+## IRC, freenode, #hurd, 2013-10-24
+
+ <gg0> braunr: i'm using your repo and i can't see cpu percentage in htop
+ anymore, all zeroes, confirmed?
+ <braunr> gg0: no
+ <braunr> gg0: you probably need to reset procfs
+ <braunr> gg0: settrans /proc /hurd/procfs -c -k 3
+
+
# Process Discovery
## IRC, freenode, #hurd, 2013-08-26
diff --git a/hurd/translator/procfs/jkoenig/discussion.mdwn b/hurd/translator/procfs/jkoenig/discussion.mdwn
index fc071337..018db7b2 100644
--- a/hurd/translator/procfs/jkoenig/discussion.mdwn
+++ b/hurd/translator/procfs/jkoenig/discussion.mdwn
@@ -436,6 +436,72 @@ Also used in `[GCC]/intl/relocatable.c`:`find_shared_library_fullname` for
`#ifdef __linux__`.
+### IRC, freenode, #hurd, 2013-10-03
+
+ <camm`> what's the equivalent of cat /proc/self/maps on hurd?
+ <braunr> camm`: for now, /proc/self doesn't work as expected
+ <camm`> thanks, I just want to get a list of maps and protection status for
+ a running process -- how?
+ <braunr> vminfo
+ <camm`> thanks so much! I'm trying to debug an unexec failure on hurd when
+ a linker script is present. All works with the default script, but when
+ the text address is changed, unexec fails, running into a page with no
+ access in the middle of the executable: 0xc4b000[0x1000] (prot=0,
+ max_prot=RWX, offs=0xb55000)
+ <camm`> I get a segfault when trying to read from this page.
+ <braunr> unexec ?
+ <camm`> emacs/gcl/maxima/acl2/hol88/axiom use unexec to dump a running
+ image into a saved executable elf file.
+ <braunr> what is unexec ?
+ <braunr> ok looks like a dirty tool
+ <braunr> camm`: what is segfaulting, unexec or the resulting executable ?
+ <camm`> unexec opens the file from which the running program was originally
+ executed, finds its section start addresses, then writes a new file
+ replacing any data in the old file with possibly modified versions in
+ running memory. The reverse of 'exec'.
+ <camm`> the read from running memory delimited by the addresses in the
+ executable file is hitting a page which has been protected with *no*
+ access, and is segfaulting. Somehow, when the binary file is loaded,
+ hurd turning off all rights to this page.
+ <camm`> let me check the stack location ...
+ <camm`> ok I think I've got it -- hurd moves the sbrk(0) address away from
+ the end of .data (as reported by readelf) if the addresses are low,
+ presumably to avoid running into the stack.
+ <camm`> starting sbrk(0)!=.data+data_size on hurd
+ <braunr> i'm not sure there is anything like the heap on the hurd
+ <braunr> sbrk is probably implemented on top of mmap
+ <braunr> camm`: hm no, i'm wrong, glibc implements brk and sbrk mostly as
+ expected, but remapping the area isn't atomic
+ <braunr> "Now reallocate it with no access allowed"
+ <braunr> then, there is a call to vm_protect
+ <braunr> and no error checking
+ <braunr> ...
+ <camm`> ok, that's fine, but need to know -- in general there is no
+ relationship between the address returned by sbrk(0) and the .data
+ addresses reported by readelf on the file, (hurd only) yes?
+ <braunr> i don't know about that
+ <braunr> there should be ..
+ <camm`> Specific example: readelf -a -> [24] .data PROGBITS
+ 000f5580 0c4580 000328 00 WA 0 0 32
+ <camm`>
+ <camm`> sbrk(0)->(void *) 0x8021000
+ <braunr> camm`: is that on an executable or a shared object ?
+ <camm`> executable
+ <braunr> 000f5580 looks very low
+ <camm`> This is using a linker script. The default setup works just fine.
+ <camm`> I think it (might) make sense for hurd to silently do this give the
+ placement of the C stack, but the assumptions behind my algorithm need
+ changing (perhaps).
+ <camm`> (I probe in configure the allowable range of __executable_start,
+ and then choose a value to either ensure a large free signed range around
+ NULL, or a low data start to maximize heap)
+ <camm`> braunr: are there any guarantees of sbrk(0)==.data+size without a
+ linker script?
+ <braunr> camm`: i'm not sure at all
+ <braunr> sbrk isn't even posix
+ <camm`> thanks
+
+
# `/proc/[PID]/mem`
Needed by glibc's `pldd` tool (commit
@@ -471,3 +537,19 @@ Needed by glibc's `pldd` tool (commit
<braunr> both htop and top seem to have problems report the cpu time
<braunr> so i expect the problem to be in procfs
+
+
+# IRC, freenode, #hurd, 2013-10-03
+
+ <braunr> teythoon: any reason the static variable translator_exists isn't
+ protected by a lock in procfs/rootdir.c ?
+
+
+## IRC, freenode, #hurd, 2013-10-04
+
+ <braunr> teythoon: can you tell me why translator_exists isn't protected
+ from shared access in rootdir_mounts_exists ?
+ <teythoon> braunr: hm, dunno tbh, I probably thought the race was harmless
+ enough
+ <braunr> it probably is
+ <braunr> settrans -Rg doesn't work on procfs :(
diff --git a/hurd/translator/term.mdwn b/hurd/translator/term.mdwn
new file mode 100644
index 00000000..667677a7
--- /dev/null
+++ b/hurd/translator/term.mdwn
@@ -0,0 +1,207 @@
+[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+The *term* translator implements POSIX termios discipline.
+
+
+# Open Issues
+
+## [[open_issues/Term_Blocking]]
+
+## Leaks/Not Re-used/Not Terminating
+
+[[!tag open_issue_hurd]]
+
+
+### IRC, freenode, #hurd, 2013-10-14
+
+ <braunr> good news
+ <braunr> the terminal leak is related to privilege separation
+ <atheia> I love how, as an unknowing by-stander, that is somehow good news
+ :-)
+ <braunr> :)
+ <braunr> it's a good news because 1/ we have more knowledge about the issue
+ <braunr> and 2/ it may not even be a hurd bug
+ <braunr> but rather an openssh-on-hurd bug
+ <braunr> this explains why i didn't see the issue on anything else
+ (mach/hurd consoles, x terminals)
+ <braunr> and this will also indirectly solve the screen lockup issue
+ <teythoon> braunr: good catch :)
+ <braunr> s/a good news/good news/
+ <atheia> ah, yes, both definitely good news. Congrats on the progress.
+ <braunr> i remember we used to disable privilege separation in the past
+ <braunr> i'll have to dig what made us use it
+ <braunr> interesting, screen seems to be affected nonetheless
+ <braunr> so it's something common to both screen and ssh privsep
+ <braunr> apparently, what sshd+privse and screen have in common is a fifo
+ <braunr> so it's probably a tricky hurd bug actually
+
+
+### IRC, freenode, #hurd, 2013-10-16
+
+ <braunr> pflocal is leaking ports ..
+ <braunr> this might be what blocks terminals
+ * pinotree gives braunr a stick of glue
+ <braunr> thanks
+
+ <braunr> pflocal leaks struct sock ..
+ <braunr> grmbl
+
+ <braunr> hm nice, pflocal leaks each time a socket is bound and/or accepted
+ on
+ <braunr> looks like a simple ref mess
+ <pinotree> braunr: really?
+ <braunr> yes
+ <pinotree> a leak in pflocal feels strange, never noticed it taking lots of
+ memory (and it's used a lot)
+ <braunr> it's a port leak
+ <braunr> well
+ <braunr> no it's both a memory and port leak
+ <braunr> not sure which one is the root cause yet
+ <braunr> i guess server sockets aren't automatically unbound
+ <braunr> if you want to see the leak, just disable priv separation in ssh
+ (to avoid the terminal leak ....) and write a shell loop to start ssh
+ your_server echo hello
+ <braunr> google shows mails about the leak in the past
+ <braunr> i also hope it fixes the terminal leak, although i'm really not
+ sure :(
+
+
+### IRC, freenode, #hurd, 2013-10-17
+
+ <braunr> hm nice, apparently, there is no pflocal leak
+ <braunr> but a libdiskfs one !
+ <braunr> since ext2fs enables the ifsock shortcut
+ <braunr> seems like it leaks a reference on sock node deletion
+ <teythoon> braunr: have you looked at libdiskfs/dead-name.c?
+ <teythoon> braunr: I think I'm hunting a very similar problem
+ <braunr> i'm doing it now
+ <teythoon> I had the problem of dead name notifications not being delivered
+ <braunr> wow
+ <teythoon> b/c I held no reference to the ports_info thing, so the dead
+ name handler in libports could no longer find the pi struct, so the
+ notification was silently dropped
+ <braunr> i see
+ <braunr> but it looks like dropping a node makes sure the associated
+ sockaddr has been deleted if any
+ <teythoon> are you sure the node is dropped in the first place?
+ <braunr> no
+ <braunr> well
+ <braunr> i see something happenning at the pflocal side when removing the
+ node
+ <braunr> but there is still a send right lingering somewhere
+ <braunr> (see why we need a global lsof :p)
+ <teythoon> indeed
+ <braunr> i'll try portinfo with that option we talked about
+ <teythoon> yes
+ <braunr> 121 => 1682: send (refs: 1)
+ <braunr> yep, ext2fs still has it
+ <teythoon> (I wonder how portinfo does that...)
+ <braunr> i guess it imports rights from the target task
+ <braunr> and see if it gets the same name as a local right
+ <teythoon> makes sense
+ <braunr> easy to check
+ <teythoon> well, no, it cannot do that for receive rights
+ <braunr> it creates an empty task just for that purpose
+ <braunr> and uses mach_port_extract_right
+ <teythoon> but it works as you described, yes
+ <braunr> so yes it does work for receive rights too
+ <teythoon> yes
+ <teythoon> cool :)
+ <braunr> so it assumes identical port names are part of the ipc interface
+ <braunr> something neal said we shouldn't rely on
+ <braunr> iirc
+ <teythoon> yes, I remember something like that too
+ <braunr> here is the strange thing
+ <braunr> node->sockaddr is deallocated on a dead name notification
+ <braunr> drop_node checks that sockaddr is null
+ <braunr> so how can the dead name notification occur before the node is
+ dropped ?
+ <braunr> so maybe the node is still around indeed
+ <braunr> apparently, libdiskfs considers the address holds a reference on
+ the node
+ <braunr> on the other hand, the server socket won't get released unless the
+ address gets a no-sender notification ...
+ <braunr> this should probably be turned into a weak reference
+ <braunr> teythoon: indeed, the node is leaked
+
+ <braunr> pflocal crashes when removing correctly deallocating addresses and
+ removing server sockets :/
+
+ <braunr> ok, pflocal bug fixed
+ <braunr> still have to fix the libdiskfs leak
+ <braunr> and libdiskfs leak fixed too
+ <braunr> :)
+ <braunr> i'll build hurd packages with my changes to make sure i don't
+ break something before comitting
+ <braunr> and see if this fixes the term issue
+
+ <braunr> looks like my patches work just fine :)
+ <braunr> it doesn't solve the term issue though
+
+ <braunr> so, according to portinfo, pflocal has send rights to terminals oO
+
+ <braunr> mhhhmmmmmm
+ <braunr> openssh seems to pass terminal file descriptors through unix
+ sockets when using privilege separation
+ <pinotree> braunr: i a write(sock, &pid, sizeof int) (or the like)?
+ <pinotree> *ie
+ <braunr> not pid, file descriptors
+ <braunr> SCM_RIGHTS
+ <pinotree> ah ok
+ <braunr> the socket send/recv interface does support passing mach ports
+ <braunr> and the leaked ports do turn into dead names when i kill terminals
+ <pinotree> yes, we support with a patch pochu did few years ago
+ <braunr> so it seems the leak is related to libpipe this time
+ <braunr> ok got it :)
+ <braunr> pflocal used copy_send instead of move_send
+ <braunr> \o/
+ <braunr> that bug was such a pain
+ * braunr happy
+ <teythoon> :)
+ <pinotree> speaking of it, in pflocal' S_socket_recv is it correct the
+ "out_flags = 0;"?
+ <braunr> nice catch
+ <braunr> although i wonder why flags are returned
+ <braunr> it may have been set to null to tell us that we don't want to
+ return flags
+ <braunr> pfinet seems to use it
+ <pinotree> but you change a local variable anyway
+ <braunr> yes it's not useful
+ <braunr> hmm
+ <braunr> out_flags is what gets in struct msghdr -> msg_flags
+ <braunr> so i guess it makes sense to fix it to *out_flags = 0, just to be
+ safe
+ <braunr> pinotree: do you want me to push it tonight along with the others
+ ?
+ <pinotree> yes please
+ <braunr> ok
+ <pinotree> thanks!
+ <braunr> pflocal seems to not leak any memory or ports at all
+ <braunr> great :>
+
+ <braunr> there, patches pushed :)
+
+
+## `screen` Logout Hang
+
+[[!tag open_issue_hurd]]
+
+
+### IRC, freenode, #hurd, 2013-10-14
+
+ <braunr> i fixed term so that screen can shutdown properly
+ <braunr> read() wouldn't return EIO after terminal hangup
+
+
+### IRC, freenode, #hurd, 2013-10-17
+
+ <braunr> and the missing EOI prevented screen from correctly shutting down
+ windows
diff --git a/hurd/translator/tmpfs/discussion.mdwn b/hurd/translator/tmpfs/discussion.mdwn
index 20aba837..8c332d84 100644
--- a/hurd/translator/tmpfs/discussion.mdwn
+++ b/hurd/translator/tmpfs/discussion.mdwn
@@ -430,3 +430,40 @@ License|/fdl]]."]]"""]]
<youpi> ok
<youpi> but that indeed means writeback of ext2fs works, which is a good
sign :)
+
+
+# IRC, freenode, #hurd, 2013-10-04
+
+ <teythoon> btw, I noticed that fifos do not work on tmpfs
+ <braunr> teythoon: tmpfs seems limited, yes
+ <teythoon> that's annoying b/c /run is a tmpfs on Debian and sysvinit
+ creates a crontrol fifo there
+ <teythoon> I wonder why I didn't notice that before
+ <braunr> also, fifos, like symlinks, can be shortcircuited in libdiskfs
+ <braunr> i wonder if that has anything to do with the problem at hand
+
+[[mtab/discussion]], *Multiple mtab Translators Spawned*.
+
+ <teythoon> b/c this breaks reboot & friends
+ <teythoon> I do too
+ <teythoon> b/c I cannot find any shortcut related code in tmpfs
+ <braunr> well, it's optional normally
+ <braunr> so that's ok
+ <braunr> but has it really been tested when the option wasn't there ? :)
+ <teythoon> yes, but the tmpfs requests this by setting diskfs_shortcut_fifo
+ = 1;
+ <pinotree> hm i remember tmpfs was said to be working with
+ sockets/fifos/etc, back then when it was fixed
+ <braunr> teythoon: oh
+
+
+## IRC, freenode, #hurd, 2013-10-11
+
+ <teythoon> this will have to wait for the next hurd pkg unfortunately, b/c
+ I broke tmpfs by accident :-/
+ <pinotree> how so?
+ <teythoon> the dropping of privileges broke passive translators and mkfifo
+ <braunr> there actually is a reason why those are run as root or with the
+ privilege of their owner
+ <braunr> privileges should be decoupled from identity
+ <teythoon> yes
diff --git a/microkernel/mach/deficiencies.mdwn b/microkernel/mach/deficiencies.mdwn
index 8f47f61f..2e205a9a 100644
--- a/microkernel/mach/deficiencies.mdwn
+++ b/microkernel/mach/deficiencies.mdwn
@@ -2384,3 +2384,310 @@ In context of [[open_issues/multithreading]] and later [[open_issues/select]].
concurrently
<braunr> (which is another contention issue when using mach-like ipc, which
often do need to allocate/release virtual memory)
+
+
+## IRC, freenode, #hurd, 2013-09-28
+
+ <rah> braunr: http://git.sceen.net/rbraun/x15.git/blob/HEAD:/README
+ <rah> "X15 is a free microkernel."
+ <rah> braunr: what distinguishes it from existing microkernels?
+
+
+## IRC, freenode, #hurd, 2013-09-29
+
+ <braunr> rah: the next part maybe ?
+ <braunr> "Its purpose is to provide a foundation for a Hurd-like operating
+ system."
+ <rah> braunr: there are already microkernels that canbe used as the
+ foundatin for Hurd=like operating systems; why are you creating another
+ one?
+ <rah> braunr: what distinguishes your microkernel from existing
+ microkernels?
+ <tschwinge> rah:
+ http://www.gnu.org/software/hurd/microkernel/mach/deficiencies.html
+ <braunr> rah: it's better :)
+ <braunr> rah: and please, cite one suitable kernel for the hurd
+ <rah> tschwinge: those are deficiencies in Mach; I'm asking about x15
+ <rah> braunr: in what way is it better exactly?
+ <braunr> rah: more performant, more scalable
+ <rah> braunr: how?
+ <braunr> better algorithms, better interfaces
+ <braunr> for example, it supports smp
+ <rah> ah
+ <rah> it supports SMP
+ <rah> ok
+ <rah> that's one thing
+ <braunr> it implements lockless synchronization à la rcu
+ <rah> are there any others?
+ <rah> ok
+ <rah> lockless sync
+ <rah> anything else?
+ <braunr> it can scale from 4MB of physical memory up to several hundreds
+ GiB
+ <braunr> ipc is completely different, leading to simpler code, less data
+ involved, faster context switches
+ <braunr> (although there is no code for that yet)
+ <rah> how can it support larger memory while other microkernels can't?
+ <rah> how is the ipc "different"?
+ <braunr> others can
+ <braunr> gnumach doesn't
+ <rah> how can it support larger memory while gnumach can't?
+ <azeem_> because it's not the same code base?
+ <braunr> gnumach doesn't support temporary kernel mapping
+ <rah> ok, so x15 supports temporary kernel mapping
+ <braunr> not exactly
+ <braunr> virtual memory is completely different
+ <rah> how so?
+ <braunr> gnumach does the same as linux, physical memory is mapped in
+ kernel space
+ <braunr> so you can't have more physical memory than you have kernel space
+ <braunr> which is why gnumach can't handle more than 1.8G right now
+ <braunr> it's a 2/2 split
+ <braunr> in x15, the kernel maps what it needs
+ <braunr> and can map it from anywhere in physical memory
+ <tschwinge> rah: I think basically all this has already been discussed
+ before and captured on that page?
+ <braunr> it already supports i386/pae/amd64
+ <rah> I see
+ <braunr> the drawback is that it needs to update kernel page tables more
+ often
+ <braunr> on linux, a small part of the kernel space is reserved for
+ temporary mappings, which need page table updates too
+ <braunr> but most allocations don't use that
+ <braunr> it's complicated
+ <braunr> also, i plan to make virtual memory operations completely
+ concurrent on x15, similar to what is described in radixvm
+ <rah> ok
+ <braunr> which means mapping operations on non overlapping regions won't be
+ serialized
+ <braunr> a big advantage for microkernels which base their messaging
+ optimizations on mapping
+ <braunr> so simply put, better performance because of simpler ipc and data
+ structures, and better scalability because of improved data structure
+ algorithms and concurrency
+ <rah> tschwinge: yes but that page is no use to someone who wants a summary
+ of what distinguishes x15
+ <braunr> x15 is still far from complete, which is why i don't advertise it
+ other than here
+ <rah> "release early, release often"?
+ <braunr> give it a few more years :p
+ <braunr> release what ?
+ <braunr> something that doesn't work ?
+ <rah> software
+ <rah> yes
+ <braunr> this release early practice applies to maintenance
+ <rah> release something that doesn't work so that others can help make it
+ work
+ <braunr> not big developments
+ <braunr> i don't want that for now
+ <braunr> i have a specific idea of what i want, and both explaining and
+ defending it would take time, better spent in development itself
+ <braunr> just wait for a first prototype
+ <braunr> and then you'll see if you want to help or not
+ * rah does not count himself as one of the "others" who might help make it
+ work
+ <braunr> one big difference with other microkernels is that x15 is
+ specifically intended to run a unix like system
+ <braunr> a hurd like system providing a psoix interface more accurately
+ <braunr> and efficiently
+ <braunr> so for example, while many microkernels provide only sync ipc, x15
+ provides both sync ipc and signals
+ <braunr> and then, there are a lot of small optimizations, like port names
+ which will transparently identify as file descriptors
+ <braunr> light reference counting
+ <braunr> a restriction on ipc that only allows reliable transfers across
+ network to machines with same arch and endianness
+ <braunr> etc..
+ <rah> http://darnassus.sceen.net/~hurd-web/microkernel/x15/
+ <rah> please take note of the fact that this newly created page is not just
+ a dump of IRC logs
+
+
+## IRC, freenode, #hurd, 2013-09-30
+
+ <braunr> rah: i'm uncomfortable with a page about x15 on the wiki ...
+ <braunr> there is a reason i don't want to advertise it for now
+ <braunr> and you're just completely ignoring it
+ <rah> braunr: detailed information about x15 is already included elsewhere
+ in the wiki
+ <braunr> rah: really ?
+ <rah> braunr: there is a section named "X15" on this page:
+ http://www.gnu.org/software/hurd/microkernel/mach/deficiencies.html
+ <braunr> rah: oh ok, but it's still small and hard to find ;p
+ <rah> braunr: "small"?!
+ <rah> braunr: the X15 section starts at about 10% down the page and
+ finishes at the bottom of the page
+ <rah> braunr: and the page is huge
+ <braunr> rah: hm ok, but that's still listed as mach deficiencies, not as
+ x15 itself
+ <rah> braunr: I heard about x15
+ <rah> braunr: I wanted to learn about it
+ <rah> braunr: there was no easily accessible information for doing so
+ <rah> braunr: it's not unreasonable to want to learn about it, having heard
+ about it
+ <rah> braunr: others will want to learn about it
+ <azeem_> rah: please respect the developer's policy of how to advertise
+ their project
+ <rah> braunr: having learned about it myself, I've helped those who will
+ follow me by giving them the summary that I wanted
+ <rah> azeem_: I'm not disrespecting the developer's policy of how to
+ advertise their project; I'm not advertising their project
+ <azeem_> rah: maybe replace the wiki page by "If you would like to know
+ about X15, please contact <your email>"
+ <rah> azeem_: that's ridiculous
+ <braunr> rah: then ask me directly
+ <braunr> rah: don't make wiki pages
+ <rah> braunr: I don't understand what you mean
+ <rah> braunr: I have already asked you directly
+ <rah> braunr: I needed to ask you directly in order to make the wiki page
+ <azeem> rah: braunr does not like your wiki page, how hard is it to
+ understand?
+ <rah> azeem: my discussion is with braunr, not you
+ <braunr> rah: if someone wants to know more about x15, he can me directly,
+ no need for a wiki page
+
+
+## IRC, freenode, #hurd, 2013-10-01
+
+ <rah> braunr: that's hyperbole; there's no "need" for a wiki, or for x15 or
+ even for the Hurd
+ <rah> braunr: a wiki page is helpful
+ <rah> useful, even
+ <braunr> rah: as azeem said, i'm just not willing to advertise outside this
+ channel for now
+ <braunr> it makes sense to mention it in the defficiencies page, since this
+ page talks about what's lacking in gnumach
+ <braunr> and the wiki is about the hurd, including gnumach
+ <rah> braunr: why does it make sense to mention it in the deficiencies page
+ but not in a dedicated page?
+ <braunr> rah: because gnumach is a hurd project, x15 isn't
+ <rah> braunr: what do you mean by "a hurd project"?
+ <rah> braunr: you're saying that x15 differs from gnumach in some way, and
+ that this difference is the reason it doesn't make sense to have a wiki
+ page devoted to x15; the phrase you've used to descibe that difference is
+ "a hurd project" but it's not clear what, exactly, you mean by that
+ <rah> braunr: could you explain what you mean by that?
+ <azeem> rah: this is getting off-topic, please take this conversation
+ elsewhere
+ <rah> azeem: that's a very tenuous statement
+ <rah> azeem: I think this is the appropriate place to discuss the matter
+ <azeem> I leave that to braunr to decide
+ <rah> azeem: I think *you* don't want the conversation to be had at all and
+ are attempting to censor it using a tenuous excuse
+ <azeem> no no, I'm not censoring it - I am just saying you should take it
+ elsewhere
+ <braunr> let's take it elsewhere
+
+
+## IRC, freenode, #hurd, 2013-10-12
+
+ <zacts> braunr: are you still working on x15/propel?
+ * zacts checks the git logs
+ <braunr> zacts: taking a break for now, will be back on it when i have a
+ clearer view of the new vm system
+
+
+## IRC, freenode, #hurd, 2013-10-15
+
+ <gnufreex> braunr, few questions about x15. I was reading IRC logs on hurd
+ site, and in the latest part, you say (or I misunderstood) that x15 is
+ now hybrid kernel. So what made you change design... or did you?
+ <braunr> gnufreex: i always intended to go for a hybrid
+
+
+## IRC, freenode, #hurd, 2013-10-19
+
+ <zacts> braunr: when do you plan to start on x15/propel again?
+ <braunr> zacts: after i'm done with thread destruction on the hurd
+
+[[open_issues/libpthread/t/fix_have_kernel_resources]].
+
+ <zacts> and do you plan to actually run hurd on top of x15, or are you
+ still going to reimplement hurd as propel?
+ <braunr> and no, i don't intend to run the hurd on top of x15
+
+
+## IRC, freenode, #hurd, 2013-10-24
+
+ <neal> braunr: What is your Mach replacement doing?
+ <braunr> "what" ? :)
+ <braunr> you mean how i guess
+ <neal> Sure.
+ <braunr> well it's not a mach replacement any more
+ <braunr> and for now it's stalled while i'm working on the hurd
+ <neal> that could be positive :)
+ <braunr> it's in good shape
+ <neal> how did it diverge?
+ <braunr> sync ipc, with unix-like signals
+ <braunr> and qnx-like bare data messages
+ <neal> hmm, like okl5?
+ <braunr> (with scatter gather)
+ <neal> okl4
+ <braunr> yes
+ <braunr> btw, if you can find a document that explains this property of
+ okl4, i'm interested, since i can't find it again on my own :/
+ <braunr> basically, x15 has a much lighter ipc interface
+ <neal> capabilities?
+ <braunr> mach ports are mostly retained
+ <braunr> but reference counting will be simplified
+ <neal> hmm
+ <neal> I don't like the reference counting part
+ <braunr> port names will be plain integers, to directly be usable as file
+ descriptors and avoid a useless translation layer
+ <braunr> (same as in qnx)
+ <neal> this sounds like future tense
+ <braunr> there is no ipc code yet
+ <neal> so I guess this stuff is not implemented
+ <neal> ok.
+ <braunr> next step is virtual memory
+ <braunr> and i'm taking my time because i want it to be a killer feature
+ <neal> so if you don't IPC and you don't have VM, what do you have? :)
+ <braunr> i have multiprocessor multithreading
+ <neal> I see.
+ <braunr> mutexes, condition variables, rcu-like lockless synchronization,
+ work queues
+ <braunr> basic bsd-like virtual memory
+ <braunr> which i want to rework
+ <neal> I ignored all of that in Viengoos :)
+ <braunr> and since ipc will still depend on virtual memory for zero-copy, i
+ want the vm system to be right
+ <braunr> well, i'm more interested in the implementation than the
+ architecture
+ <braunr> for example, i have unpublished code that features a lockless
+ radix tree for vm_object lookups
+ <braunr> that's quite new for a microkernel based system, but the ipc
+ interface itself is very similar to what already exists
+ <braunr> your half-sync ipc are original :)
+ <neal> I'm considering getting back in the OS game.
+ <braunr> oh
+ <neal> But, I'm not going to write a kernel this time.
+ <braunr> did anyone here consider starting a company for such things, like
+ genode did ?
+ <neal> I was considering using genode as a base.
+ <braunr> neal: why genode ?
+ <neal> I want to build a secure system.
+ <neal> I think the best way to do that is using capabilities.
+ <neal> Genode runs on Fiasco.OC, for instance
+ <neal> and it provides a lot of infrastructure
+ <braunr> neal: why not l4re for example ?
+ <braunr> neal: how important is the ability to revoke capabilities ?
+
+In the discussion on [[community/gsoc/project_ideas/object_lookups]], *IRC,
+freenode, #hurd, 2013-10-24*:
+
+ <teythoon> and, with some effort, getting rid of the hash table lookup by
+ letting the kernel provide the address of the object (iirc neil knew the
+ proper term for that)
+ <braunr> teythoon: that is a big interface change
+ <teythoon> how so
+ <braunr> optimizing libihash and libpthread should already be a good start
+ <braunr> well how do you intend to add this information ?
+ <braunr> ok, "big" is overstatement, but still, it's a low level interface
+ change that would probably break a lot of things
+ <teythoon> store a pointer in the port structure in gnumach, make that
+ accessible somehow
+ <braunr> yes but how ?
+ <teythoon> interesting question indeed
+ <braunr> my plan for x15 is to make this "label" part of received messages
+ <braunr> which means you need to change the format of messages
+ <braunr> that is what i call a big change
diff --git a/microkernel/mach/gnumach/boot_trace.mdwn b/microkernel/mach/gnumach/boot_trace.mdwn
index 7b729c23..ea999a9b 100644
--- a/microkernel/mach/gnumach/boot_trace.mdwn
+++ b/microkernel/mach/gnumach/boot_trace.mdwn
@@ -227,3 +227,25 @@ License|/fdl]]."]]"""]]
>> vm\_pageout
>> Does not return.
+
+
+# IRC, freenode, #hurd, 2013-10-07
+
+ <cureOS> look, where should i dig or where from should i start from, if i
+ have desire to know how the kernel was written from baremetal? Can it be
+ ever done nowadays?
+ <youpi> cureOS: the boot entry of the kernel is i386/i386at/boothdr.S ,
+ boot_entry
+ <youpi> that's what grub jumps to
+ <youpi> then that jumps to c_boot_entry
+ <youpi> and everything else is C
+ <cureOS> grub loads it somehow. how does it prepare cpu and memoty, cpu
+ cache control if any... segments for stack..
+ <youpi> see the grub documentation
+ <youpi> basically it's all flat linear space
+ <cureOS> does kernel transform it after that?
+ <youpi> see the ldt/gdt initialization
+ <youpi> from i386at_init and children
+ <youpi> nothing much fancy, a kernel cs/ds, and user cs/ds
+ <braunr> and paging, naturally
+ <youpi> sure
diff --git a/open_issues/64-bit_port.mdwn b/open_issues/64-bit_port.mdwn
index b0c95612..edb2dccd 100644
--- a/open_issues/64-bit_port.mdwn
+++ b/open_issues/64-bit_port.mdwn
@@ -155,3 +155,10 @@ In context of [[mondriaan_memory_protection]].
<braunr> the problem is the interfaces themselves
<braunr> type widths
<braunr> as passed between userspace and kernel
+
+
+# IRC, OFTC, #debian-hurd, 2013-10-05
+
+ <dharc> and what about 64 bit support, almost done?
+ <youpi> kernel part is done
+ <youpi> MIG 32/64 trnaslation missing
diff --git a/open_issues/anatomy_of_a_hurd_system.mdwn b/open_issues/anatomy_of_a_hurd_system.mdwn
index ba72b00f..a3c55063 100644
--- a/open_issues/anatomy_of_a_hurd_system.mdwn
+++ b/open_issues/anatomy_of_a_hurd_system.mdwn
@@ -803,3 +803,11 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l
<braunr> and hoping it didn't corrupt something important like file system
caches before being flushed
<giuscri> kilobug, braunr : mhn, ook
+
+
+# IRC, freenode, #hurd, 2013-10-13
+
+ <ahungry> ahh, ^c isn't working to cancel a ping - is there alternative?
+ <braunr> ahungry: ctrl-c does work, you just missed something somewhere and
+ are running a shell directly on a console, without a terminal to handle
+ signals
diff --git a/open_issues/boehm_gc.mdwn b/open_issues/boehm_gc.mdwn
index 623dcb83..0a476d71 100644
--- a/open_issues/boehm_gc.mdwn
+++ b/open_issues/boehm_gc.mdwn
@@ -523,3 +523,22 @@ restults of GNU/Linux and GNU/Hurd look very similar.
<congzhang> hi, I am dotgnu work on hurd, and even winforms app
<congzhang> s/am/make
<congzhang> and maybe c# hello world translate another day :)
+
+
+## Leak Detection
+
+### IRC, freenode, #hurd, 2013-10-17
+
+ <teythoon> I spent the last two days integrating libgc - the boehm
+ conservative garbage collector - into hurd
+ <teythoon> it can be used in leak detection mode
+ <azeem> whoa, cool
+ <teythoon> and it actually kind of works, finds malloc leaks in translators
+ <braunr> i think there were problems with signal handling in libgc
+ <braunr> i'm not sure we support nested signal handling well
+ <teythoon> yes, I read about them
+ <teythoon> libgc uses SIGUSR1/2, so any program installing handlers on them
+ will break
+ <azeem> (which is not a problem on Linux, cause there some RT-signals or so
+ are used)
+ <teythoon> yes
diff --git a/open_issues/code_analysis/discussion.mdwn b/open_issues/code_analysis/discussion.mdwn
index 7ac3beb1..4cb03293 100644
--- a/open_issues/code_analysis/discussion.mdwn
+++ b/open_issues/code_analysis/discussion.mdwn
@@ -1,4 +1,5 @@
-[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2011, 2012, 2013 Free Software Foundation,
+Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -42,6 +43,8 @@ License|/fdl]]."]]"""]]
<braunr> i tried duma, and it crashes, probably because of cthreads :)
+# Static Analysis
+
## IRC, freenode, #hurd, 2012-09-08
<mcsim> hello. What static analyzer would you suggest (probably you have
@@ -49,3 +52,54 @@ License|/fdl]]."]]"""]]
<braunr> mcsim: if you find some good free static analyzer, let me know :)
<pinotree> a simple one is cppcheck
<mcsim> braunr: I'm choosing now between splint and adlint
+
+
+## IRC, freenode, #hurd, 2013-10-17
+
+ <teythoon> whoa, llvm kinda works, enough to make scan-build work :)
+ <braunr> teythoon: what is scan-build ?
+ <teythoon> braunr: clangs static analyzer
+ <braunr> ok
+ <teythoon> I'm doing a full build of the hurd using it, I will post the
+ report once it is finished
+ <teythoon> this will help spot many problems
+ <teythoon> well, here are the scan-build reports I got so far:
+ https://teythoon.cryptobitch.de/qa/2013-10-17/scan-build/
+ <teythoon> I noticed it finds problems in mig generated code, so there are
+ probably lot's of duplictaes for those kind of problems
+ <pinotree> what's a... better one to look at?
+ <teythoon> it's also good at spotting error handling errors, and can spot
+ leaks sometimes
+ <teythoon> hm
+ <teythoon>
+ https://teythoon.cryptobitch.de/qa/2013-10-17/scan-build/report-yVBHO1.html
+ <braunr> that's minor, the device always exist
+ <braunr> but that's still ugly
+ <teythoon>
+ https://teythoon.cryptobitch.de/qa/2013-10-17/scan-build/report-MtgWSa.html
+ <teythoon>
+ https://teythoon.cryptobitch.de/qa/2013-10-17/scan-build/report-QdsZIm.html
+ <teythoon> this could be important:
+ https://teythoon.cryptobitch.de/qa/2013-10-17/scan-build/report-PDMEbk.html
+ <teythoon> this is the issue it finds in mig generated server stubs:
+ https://teythoon.cryptobitch.de/qa/2013-10-17/scan-build/report-iU3soc.html
+ <braunr> this one is #if TypeCheck1
+ <braunr> the libports one looks weird indeed
+ <teythoon> but TypeCheck is 1 (the tooltip shows macro expansion)
+ <teythoon> it is defined in line 23
+ <braunr> oh
+ <teythoon> hmmm... clang does not support nested functions, that will limit
+ its usefulness for us :/
+ <braunr> yes
+ <braunr> one more reason not to use them
+
+
+### IRC, freenode, #hurd, 2013-10-18
+
+ <teythoon> more complete, now with index:
+ https://teythoon.cryptobitch.de/qa/2013-10-17/scan-build-2/
+
+
+# Leak Detection
+
+See *Leak Detection* on [[boehm_gc]].
diff --git a/open_issues/dbus.mdwn b/open_issues/dbus.mdwn
index a41515a1..4473fba0 100644
--- a/open_issues/dbus.mdwn
+++ b/open_issues/dbus.mdwn
@@ -253,3 +253,115 @@ See [[glibc]], *Missing interfaces, amongst many more*, *`SOCK_CLOEXEC`*.
to know how to find this sendmsg.c file?
<pinotree> (it's in glibc, but otherwise the remark is valid)
<pinotree> s/otherwise/anyway/
+
+
+# Emails
+
+# IRC, freenode, #hurd, 2013-10-16
+
+ <braunr> gnu_srs: how could you fail to understand credentials need to be
+ checked ?
+ <gnu_srs> braunr: If data is sent via sendmsg, no problem, right?
+ <braunr> gnu_srs: that's irrelevant
+ <gnu_srs> It's just to move the check to the receive side.
+ <braunr> and that is the whole problem
+ <braunr> it's not "just" doing it
+ <braunr> first, do you know what the receive side is ?
+ <braunr> do you know what it can be ?
+ <braunr> do you know where the corresponding source code is to be found ?
+ <gnu_srs> please, describe a scenario where receiving faulty ancillary data
+ could be a problem instead
+ <braunr> dbus
+ <braunr> a user starting privileged stuff although he's not part of a
+ privileged group of users for example
+ <braunr> gnome, kde and others use dbus to pass user ids around
+ <braunr> if you can't rely on these ids being correct, you can compromise
+ the whole system
+ <braunr> because dbus runs as root and can give root privileges
+ <braunr> or maybe not root, i don't remember but a system user probably
+ <pinotree> "messagebus"
+ <gnu_srs> k!
+ <braunr> see http://www.gnu.org/software/hurd/open_issues/dbus.html
+ <braunr> IRC, freenode, #hurd, 2013-07-17
+ <braunr> <teythoon> and the proper fix is to patch pflocal to query the
+ auth server and add the credentials?
+ <braunr> <pinotree> possibly
+ <braunr> <teythoon> that doesn't sound to bad, did you give it a try?
+
+
+# IRC, freenode, #hurd, 2013-10-22
+
+ <gnu_srs> I think I have a solution on the receive side for SCM_CREDS :)
+
+ <gnu_srs> A question related to SCM_CREDS: dbus use a zero data byte to get
+ credentials sent.
+ <gnu_srs> however, kfreebsd does not care which data (and credentials) is
+ sent, they report the credentials anyway
+ <gnu_srs> should the hurd implementation do the same as kfreebsd?
+ <youpi> gnu_srs: I'm not sure to understand: what happens on linux then?
+ <youpi> does it see zero data byte as being bogus, and refuse to send the
+ creds?
+ <gnu_srs> linux is also transparent, it sends the credentials independent
+ of the data (but data has to be non-null)
+ <youpi> ok
+ <youpi> anyway, what the sending application writes does not matter indeed
+ <youpi> so we can just ignore that
+ <youpi> and have creds sent anyway
+ <braunr> i think the interface normally requires at least a byte of data
+ for ancilliary data
+ <youpi> possibly, yes
+ <braunr> To pass file descriptors or credentials over a SOCK_STREAM,
+ you need to send or
+ <braunr> receive at least one byte of non-ancillary data in
+ the same sendmsg(2) or
+ <braunr> recvmsg(2) call.
+ <braunr> but that may simply be linux specific
+ <braunr> gnu_srs: how do you plan on implementing right checking ?
+ <gnu_srs> Yes, data has to be sent, at least one byte, I was asking about
+ e.g. sending an integer
+ <braunr> just send a zero
+ <braunr> well
+ <braunr> dbus already does that
+ <braunr> just don't change anything
+ <braunr> let applications pass the data they want
+ <braunr> the socket interface already deals with port rights correctly
+ <braunr> what you need to do is make sure the rights received match the
+ credentials
+ <gnu_srs> The question is to special case on a zero byte, and forbid
+ anything else, or allow any data.
+ <braunr> why would you forbid
+ <braunr> ?
+ <gnu_srs> linux and kfreebsd does not special case on a received zero byte
+ <braunr> same question, why would you want to do that ?
+ <gnu_srs> linux sends credentials data even if no SCM_CREDENTIALS structure
+ is created, kfreebsd don't
+ <braunr> i doubt that
+ <gnu_srs> To be specific:msgh.msg_control = NULL; msgh.msg_controllen = 0;
+ <braunr> bbl
+ <gnu_srs> see the test code:
+ http://lists.debian.org/debian-hurd/2013/08/msg00091.html
+ <braunr> back
+ <braunr> why would the hurd include groups when sending a zero byte, but
+ only uid when not ?
+ <gnu_srs> ?
+ <braunr> 1) Sent credentials are correct:
+ <braunr> no flags: Hurd: OK, only sent ids
+ <braunr> -z Hurd: OK, sent IDs + groups
+ <braunr> and how can it send more than one uid and gid ?
+ <braunr> "sent credentials are not honoured, received ones are created"
+ <gnu_srs> Sorry, the implementation is changed by now. And I don't special
+ case on a zero byte.
+ <braunr> what does this mean ?
+ <braunr> then why give me that link ?
+ <gnu_srs> The code still applies for Linux and kFreeBSD.
+ <gnu_srs> It means that whatever you send, the kernel emits does not read
+ that data: see
+ <gnu_srs> socket.h: before struct cmsgcred: the sender's structure is
+ ignored ...
+ <braunr> do you mean receiving on a socket can succeed with gaining
+ credentials, although the sender sent wrong ones ?
+ <gnu_srs> Looks like it. I don't have a kfreebsd image available right now.
+ <gnu_srs> linux returns EPERM
+ <braunr> anyway
+ <braunr> how do you plan to implement credential checking ?
+ <gnu_srs> I'll mail patches RSN
diff --git a/open_issues/debugging_gnumach_startup_qemu_gdb.mdwn b/open_issues/debugging_gnumach_startup_qemu_gdb.mdwn
index e3a6b648..3faa56fc 100644
--- a/open_issues/debugging_gnumach_startup_qemu_gdb.mdwn
+++ b/open_issues/debugging_gnumach_startup_qemu_gdb.mdwn
@@ -1,4 +1,5 @@
-[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2010, 2011, 2013 Free Software Foundation,
+Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -12,8 +13,22 @@ License|/fdl]]."]]"""]]
[[!tag open_issue_gdb open_issue_gnumach]]
+[[!toc]]
-# IRC, freenode, #hurd, 2011-07-14
+
+# Memory Map
+
+## IRC, freenode, #hurd, 2010-06 (?)
+
+ <jkoenig> is there a way to get gdb to map addresses as required when
+ debugging mach with qemu ?
+ <jkoenig> I can examine the data if I manually map the addresses th
+ 0xc0000000 but maybe there's an easier way...
+ <youpi> jkoenig: I haven't found a way
+ <youpi> I'm mostly using the internal kdb
+
+
+## IRC, freenode, #hurd, 2011-07-14
<mcsim> Hello. I have problem with debugging gnumach. I set 2 brakepoints
in file i386/i386at/model_dep.c on functions gdt_init and idt_init. Then
@@ -114,3 +129,18 @@ License|/fdl]]."]]"""]]
<antrik> oh, right, without GDB...
<antrik> though if that's what he meant, his statement was very misleading
at least
+
+
+# Multiboot
+
+See also discussion about *multiboot* on [[arm_port]].
+
+
+## IRC, freenode, #hurd, 2013-10-09
+
+ <matlea01> I was just wondering - once gnumach is compiled and I have the
+ gnumach elf, is that bootable? I.e. can I use something like
+ "qemu-system-i386 -kernel gnumach"?
+ <kilobug> matlea01: you need something with multiboot support (like grub)
+ to provide the various bootstrap modules to the kernel
+ <matlea01> Ah, I see
diff --git a/open_issues/emacs.mdwn b/open_issues/emacs.mdwn
index cdd1b10d..749649be 100644
--- a/open_issues/emacs.mdwn
+++ b/open_issues/emacs.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2009 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2009, 2013 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -1525,3 +1525,18 @@ perhaps prepared (I did not yet have a look), and re-tries again and again?
Why doesn't Mach page out some pages to make memory available?
This is stock GNU Mach from Git, no patches, configured for Xen domU usage.
+
+
+# IRC, freenode, #hurd, 2013-10-04
+
+ <pinotree> given you are an emacs user: could you please pick the build
+ patch from deb#725099, recompile emacs24 and test it with your daily
+ work?
+
+
+## IRC, freenode, #hurd, 2013-10-07
+
+ <gnu_srs> Wow! emacs24 runs in X:-D
+ <gnu_srs> pinotree: I've now built and installed emacs 24.3. So far so good
+ ^
+ <pinotree> good, keep testing and stressing
diff --git a/open_issues/exec_memory_leaks.mdwn b/open_issues/exec_memory_leaks.mdwn
index 67281bdc..1fc5a928 100644
--- a/open_issues/exec_memory_leaks.mdwn
+++ b/open_issues/exec_memory_leaks.mdwn
@@ -94,3 +94,28 @@ After running the libtool testsuite for some time:
8 39.5 0:15.60 28:48.57
9 0.0 0:04.49 10:24.12
10 12.8 0:08.84 19:34.45
+
+
+# IRC, freenode, #hurd, 2013-10-08
+
+ * braunr hunting the exec leak
+ <braunr> and i think i found it
+ <braunr> yes :>
+ <braunr> testing a bit more and committing the fix later tonight
+ <braunr> pinotree: i've been building glibc for 40 mins and exec is still
+ consuming around 1m memory
+ <pinotree> wow nice
+ <pinotree> i've been noticing exec leaking quite some time ago, then forgot
+ to pay more attention to that
+ <braunr> it's been more annoying since darnassus provides web access to
+ cgis
+ <braunr> automated tools make requests every seconds
+ <braunr> the leak occurred when starting a shell script or using system()
+ <braunr> youpi: not sure you saw it, i fixed the exec leak
+
+
+## IRC, freenode, #hurd, 2013-10-10
+
+ <gg0> braunr: http://postimg.org/image/jd764wfpp/
+ <braunr> exec 797M
+ <braunr> this should be fixed with the release of the next hurd packages
diff --git a/open_issues/ext2fs_libports_reference_counting_assertion.mdwn b/open_issues/ext2fs_libports_reference_counting_assertion.mdwn
index ff1c4c38..9ff43afa 100644
--- a/open_issues/ext2fs_libports_reference_counting_assertion.mdwn
+++ b/open_issues/ext2fs_libports_reference_counting_assertion.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -91,3 +91,14 @@ With that patch in place, the assertion failure is seen more often.
sure we can get that easily lol
[[automatic_backtraces_when_assertions_hit]].
+
+
+# IRC, freenode, #hurd, 2013-10-09
+
+ <braunr> mhmm, i may have an explanation for the weird assertions we
+ sometimes see in ext2fs
+ <braunr> glibc uses alloca to reserve memory for one reply port per thread
+ in abort_all_rpcs
+ <braunr> if this erases the thread-specific area, we can expect all kinds
+ of wreckage
+ <braunr> i'm not sure how to fix this though
diff --git a/open_issues/gdb_qemu_debugging_gnumach.mdwn b/open_issues/gdb_qemu_debugging_gnumach.mdwn
deleted file mode 100644
index d3105f50..00000000
--- a/open_issues/gdb_qemu_debugging_gnumach.mdwn
+++ /dev/null
@@ -1,19 +0,0 @@
-[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]]
-
-[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
-id="license" text="Permission is granted to copy, distribute and/or modify this
-document under the terms of the GNU Free Documentation License, Version 1.2 or
-any later version published by the Free Software Foundation; with no Invariant
-Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
-is included in the section entitled [[GNU Free Documentation
-License|/fdl]]."]]"""]]
-
-[[!tag open_issue_gdb open_issue_gnumach]]
-
-\#hurd, freenode, June (?) 2010
-
- <jkoenig> is there a way to get gdb to map addresses as required when debugging mach with qemu ?
- <jkoenig> I can examine the data if I manually map the addresses th 0xc0000000 but maybe there's an easier way...
- <youpi> jkoenig: I haven't found a way
- <youpi> I'm mostly using the internal kdb
-
diff --git a/open_issues/gdb_signal_handler.mdwn b/open_issues/gdb_signal_handler.mdwn
index 3084f7e3..5e27a099 100644
--- a/open_issues/gdb_signal_handler.mdwn
+++ b/open_issues/gdb_signal_handler.mdwn
@@ -401,3 +401,74 @@ License|/fdl]]."]]"""]]
<zyg> braunr: are you sure? there is minimal user-code run before the
signal is going into the handler.
<braunr> you "step out of the handler"
+
+
+# IRC, freenode, #hurd, 2013-10-24
+
+ <gnu_srs> how come some executables are not debuggable with gdb, e.g Cannot
+ access memory at address xxx. -fPIC flag?
+ <braunr> no
+ <braunr> i'm not sure but it's certainly not -fPIC
+ <gnu_srs> Another example is localedef: ./debian/tmp-libc/usr/bin/localedef
+ -i en_GB -c -f UTF-8 -A /usr/share/locale/locale.alias en_GB.UTF-8
+ segfailts
+ <gnu_srs> and in gdb hangs after creating a thread., after C-c no useful
+ info: stack ends with: Cannot access memory at address 0x8382c385
+ <braunr> if it's on the stack, it's probably a stack corruption
+ <nalaginrut> gnu_srs: are u using 'x' command or 'print' in GDB? IIRC
+ print may throw such message, but x may not
+ <gnu_srs> bt
+ <braunr> x may too
+ <braunr> what you're showing looks like an utf-8 string
+ <braunr> c385 is Å
+ <braunr> 83 is a special f
+ <braunr> 82 is a comma
+ <gnu_srs> so the stack is corrupted:-(
+ <braunr> probably
+ <braunr> well, certainly
+ <braunr> but gdb should show you where the program counter is
+ <gnu_srs> is that: ECX: the count register
+ <braunr> no
+ <braunr> eip
+ <braunr> program counter == instruction pointer
+ <gnu_srs> k!, the program counter is at first entry in bt: #0 0x01082612
+ in _hurd_intr_rpc_msg_in_trap () at intr-msg.c:133
+ <braunr> this is the hurd interruptible version of mach_msg
+ <braunr> so it probably means the corruption was made by a signal handler
+ <braunr> which is one of the reasons why gdb can't handle Ctrl-c
+ <gnu_srs> what to do in such a case, follow the source code
+ single-stepping?
+ <braunr> single stepping also uses signals
+ <braunr> and using printf will probably create an infinite recursion
+ <braunr> in those cases, i use mach_print
+ <braunr> as a first step, you could make sure a signal is actually received
+ <braunr> and which one
+ <braunr> hmm
+ <braunr> also, before rushing into conclusions, make sure you're looking at
+ the right thread
+ <braunr> i don't expect localedef to be multithreaded
+ <braunr> but gdb sometimes just doesn't get the thread where the segfault
+ actually occurred
+ <gnu_srs> two threads: 1095.4 and 1095.5 (created when starting localedef
+ in gdb)
+ <braunr> no, at the time of the crash
+ <braunr> the second thread is always the signal thread
+ <gnu_srs> OK,in gdb the program hangs, interrupted by C-c, outside it
+ segfaults
+ <braunr> when you use bt to get the corrupted stack, you can also use info
+ threads and thread apply all bt
+ <gnu_srs> I did: http://paste.debian.net/61170/
+ <braunr> ok so it confirms there is only one real application thread, the
+ main one
+ <braunr> and that the corruption probably occurs during signal handling
+ <gnu_srs> rpctrace (edited out non-printable characters):
+ http://paste.debian.net/61178/
+ <gnu_srs> Ah, have to do it again as root;-)
+ <braunr> yes .. :p
+ <gnu_srs> new last part: http://paste.debian.net/61181/
+ <braunr> so, there is a seek, then a stat, then a close perhaps (port
+ deallocation) and then a signal received (probably sigsegv)
+ <braunr> gnu_srs: when you try running it in gdb, do you get a sigkill ?
+ <braunr> damn, gdb on darnassus is bugged :-(
+ <gnu_srs> It hangs, interrupted with C-c.
+ <braunr> ok
diff --git a/open_issues/git-core-2.mdwn b/open_issues/git-core-2.mdwn
index cbf47bd2..a92b3ebb 100644
--- a/open_issues/git-core-2.mdwn
+++ b/open_issues/git-core-2.mdwn
@@ -61,6 +61,113 @@ Fixing this situation is easy enough:
Still seen.
+## IRC, freenode, #hurd, 2013-10-10
+
+ <sea`> Huh? I've cloned the 'hurd' repository and I'm attempting to compile
+ it, but the 'rtnetlink.h' header in
+ 'hurd/pfinet/linux-src/include/linux/' is just blank. (Which leads to an
+ error later down when a macro that's supposed to be defined in there is
+ first used)
+ <sea`> So I'm just wondering, is that file really blank? Or is this some
+ unexpected error of decompression?
+ <braunr> clone again and see
+ <braunr> the file is definitely not empty
+ <sea`> I cloned it twice, both have that file blank. BUT, I want to point
+ out that both clones do have some decompression errors. (Some files are
+ missing chunks in /both/ cloned repositories).
+ <braunr> where did you clone it from ?
+ <sea`> git.sv.gnu.org/hurd/hurd.git
+ <braunr> hum decompression errors ?
+ <braunr> can you paste them please ?
+ <sea`> Hmm, I can clone again and show you an example if I find one
+ <sea`> This was on the hurd. When I run: git clone $repo;, it seems to fail
+ almost randomly with "incorrect header check", but when it does succeed,
+ occasionally some files are missing chunks
+ <sea`> and apparently entire files can be blank
+ <braunr> http or git ?
+ <sea`> git.
+ <braunr> that's really weird
+ <braunr> actually i don't even have problems with http any more nowadays ..
+ <sea`> This is using the hurd image from sthibault
+ <sea`> So once I get it recompiled and shuffle in the new binaries, the
+ problem should probably go away
+ <braunr> no
+ <braunr> well maybe but
+ <braunr> don't recompile
+ <braunr> upgrade packages instead
+ <sea`> Alright, I'll do an upgrade instead. Why that path specifically?
+ <braunr> rebuilding is long
+ <braunr> i wonder if the image you got is corrupted
+ <braunr> compute the checksum
+ <braunr> we've had weird reports in the past about the images he provides
+ <braunr> well not the images themselves, but differences after dowloading
+ ..
+ <braunr> downloading*
+ <sea`> The MD5SUMS file on his site isn't including the values for the most
+ recent images.
+ <sea`> It stops at 2012-12-28
+ <braunr> hummm
+ <sea`> Anyway, let's see. git clone failed again:
+ <sea`> Receiving objects: 100% (50955/50955), 15.48 MiB | 42 KiB/s, done.
+ <sea`> error: inflate: date stream error (incorrect header check) <- This
+ is the interesting part
+ <sea`> fatal: serious inflate inconsistency
+ <sea`> fatal: index-pack failed
+ <braunr> not intereseting enough unfortunately
+ <braunr> but it might come from savannah too
+ <braunr> try the mirrors at
+ http://darnassus.sceen.net/gitweb/?a=project_list;pf=savannah_mirror
+ <sea`> Let's see..if I try: 'git clone
+ git://darnassus.sceen.net/gitweb/savannah_mirror/hurd.git', I get:
+ 'fatal: remote error: access denied or repository not exported:
+ /gitweb/savannah_mirror/hurd.git'
+ <braunr> my bad
+ <braunr> that's weird, it should work ..
+ <braunr> oh, stupid translation error
+ <sea`> translation? From one human language to another?
+ <braunr> not translation actually
+ <braunr> typo :)
+ <braunr> it's either
+ <braunr> git://darnassus.sceen.net/savannah_mirror/hurd.git
+ <braunr> or
+ <braunr> http://darnassus.sceen.net/gitweb/savannah_mirror/hurd.git
+ <braunr> copy paste the url exactly please
+ <braunr> /gitweb/ is only present in the http url
+ <sea`> Ah, right. Okay, I'll paste it exactly
+ <sea`> Ehm. The whole thing locked up badly. I'll reboot it and try again.
+ <braunr> are you sure it locked oO ?
+ <braunr> the hurd can easily become unresponsive when performing io
+ operations
+ <braunr> but you need more than such a git repository to reach that state
+ <sea`> Yeah, that happens occasionally. It's not related to git, but rather
+ it happens when I cancel some command.
+ <braunr> your image must be corrupted
+ <braunr> have you enabled host io caching btw ?
+ <sea`> By now it's corrupted for sure..everytime it crashes the filesystem
+ gets into a weird state.
+ <sea`> I'll unpack a fresh image, then update the packages, and then try
+ cloning this git repository.
+ <braunr> i'll get the image too so we can compare sums
+ <sea`> 957bb0768c9558564f0c3e0adb9b317e ./debian-hurd.img.tar.gz
+ <sea`> Which unpacks to: debian-hurd-20130504.img
+ <azeem_> the NSA might backdoor the Hurd, in anticipation of our scheduled
+ world-dominance
+ <braunr> for now they're doing it passively :
+ <braunr> :p
+ <braunr> sea`: same thing here
+ <braunr> sea`: if you still have problems, the image itself might be wrong
+ <braunr> in which case you should try with the debian network installer
+ <sea`> Ah, so if problems persist, try with the network installer. Okay
+ <sea`> Is there some recipe for constructing a hurd/mach minimal
+ environment?
+ <sea`> A system with only just enough tools and libraries to compile and
+ poke at things.
+ <braunr> not currently
+ <braunr> we all work in debian environments
+ <braunr> the reason being that a lot of patches are queued for integration
+ upstream
+
+
# 2010-11-17
A very similar issue. The working tree had a lot of
diff --git a/open_issues/glibc.mdwn b/open_issues/glibc.mdwn
index b453b44f..292c6256 100644
--- a/open_issues/glibc.mdwn
+++ b/open_issues/glibc.mdwn
@@ -330,6 +330,33 @@ Last reviewed up to the [[Git mirror's 0323d08657f111267efa47bd448fbf6cd76befe8
clearly not a priority
<nalaginrut> ok
+ IRC, freenode, #hurd, 2013-09-26:
+
+ <nalaginrut> if I want to have epoll/kqueue like things, where
+ should it dwell? kernel or some libs?
+ <braunr> libs
+ <pinotree> userland
+ <braunr> that would be a good project to work on, something i
+ intended to do (so i can help) but it requires a lot of work
+ <braunr> you basically need to add a way to explicitely install and
+ remove polling requests (instead of the currently way that
+ implicitely remove polling requests when select/poll returns)
+ <braunr> while keeping the existing way working for some time
+ <braunr> glibc implements select
+ <braunr> the hurd io interface shows the select interface
+ <braunr> servers such as pfinet/pflocal implement it
+ <braunr> glibc implements the client-side of the call
+ <nalaginrut> where's poll? since epoll just added edge-trigger in
+ poll
+ <braunr> both select and poll are implemented on top of the hurd io
+ select call (which isn't exactly select)
+ <braunr>
+ http://darnassus.sceen.net/gitweb/savannah_mirror/hurd.git/blob/HEAD:/hurd/io.defs
+ <braunr> this is the io interface
+ <braunr>
+ http://darnassus.sceen.net/gitweb/savannah_mirror/glibc.git/blob/refs/heads/tschwinge/Roger_Whittaker:/hurd/hurdselect.c
+ <braunr> this is the client side implementation
+
* `sys/eventfd.h`
* `sys/inotify.h`
@@ -854,6 +881,298 @@ Last reviewed up to the [[Git mirror's 0323d08657f111267efa47bd448fbf6cd76befe8
<braunr> to check where those locks are held and determine the
right order
+ IRC, OFTC, #debian-hurd, 2013-09-28:
+
+ <gg0_> now we'd just need tls
+ <gg0_> http://bugs.ruby-lang.org/issues/8937
+ <gg0_> well, it would pass makecheck at least. makecheckall would
+ keep hanging on threads/pipes tests i guess, unless tls/thread
+ destruction patches fix them
+
+ IRC, OFTC, #debian-hurd, 2013-10-05:
+
+ <youpi> so what is missing for ruby2.0, only disabling use of
+ context for now, no?
+ <pinotree> i'm not tracking it closely, gg0_ is
+ <gg0_> maybe terceiro would accept a patch which only disables
+ *context, "maybe" because he rightly said changes must go
+ upstream
+ <gg0_> anyway with or without *context, many many tests in
+ makecheckall fail by making it hang, first with and without
+ assertion you removed, now they all simply hang
+ <gg0_> youpi: what do we want to do? if you're about finishing tls
+ migration (as i thought a couple of weeks ago), i won't propose
+ anything upstream. otherwise i could but that will have to be
+ reverted upstream once you finish
+ <gg0_> about tests, current ruby2.0 doesn't run makecheckall, only
+ makecheck which succeeds on hurd (w/o context)
+ <gg0_> if anyone wants to give it a try:
+ http://paste.debian.net/plain/51089
+ <gg0_> first hunk makes makecheck (not makecheckall) succeed and
+ has been upstreamed, not packaged yet
+ <pinotree> what about makecheckall for ruby2.0?
+ <gg0_> 16:58 < gg0_> anyway with or without *context, many many
+ tests in makecheckall fail by making it hang, first with and
+ without assertion you removed, now they all simply hang
+ <pinotree> i for a moment thought it as for 1.9.1, ok
+ <pinotree> these hangs should be debugged, yes
+ <gg0_> nope, tests behavior doesn't change between 1.9 and 2.0. i
+ started suppressing tests onebyone on 2.0 as well and as happened
+ on 1.9, i gave up cause there were too many
+ <gg0_> yep a smart mind could start debugging them, starting from
+ patch above pasted by a lazy one owner
+ <gg0_> one problem is that one can't reproduce them by isolate
+ them, they don't fail. start makecheckall then wait for one fail
+ <gg0_> now after my stupid report, someone like pinotree could take
+ it over, play with it for half an hour/an hour (which equals to
+ half a gg0's year/a gg0's year
+ <gg0_> )
+ <gg0_> and fix them all
+
+ <gg0_> 17:05 < gg0_> youpi: what do we want to do? if you're about
+ finishing tls migration (as i thought a couple of weeks ago), i
+ won't propose anything upstream. otherwise i could but that will
+ have to be reverted upstream once you finish
+ <youpi> gg0_: I don't really know what to answer
+ <youpi> that's why I didn't answer :)
+ <gg0_> youpi: well then we could upstream context disable and keep
+ it disabled even if you fix tls. ruby won't be as fast as it
+ would be with context but i don't think anyone will complain
+ about that. then once packaged, if terceiro doesn't enable
+ makecheckall, we will have ruby2.0 in main
+ <youpi> that can be a plan yes
+ <gg0_> btw reverting it upstream should not be a problem eventually
+ <youpi> sure, the thing is remembering to do it
+ <gg0_> filed http://bugs.ruby-lang.org/issues/8990
+ <gg0_> please don't fix tls too soon :)
+ <gg0_> s/makecheck/maketest/g
+
+ IRC, OFTC, #debian-hurd, 2013-10-08:
+
+ <gg0_> ok. *context disabled http://bugs.ruby-lang.org/issues/8990
+
+ <gg0> bt full of an attached stuck ruby test
+ http://paste.debian.net/plain/53788/
+ <gg0> anything useful?
+ <youpi> uh, is that really all?
+ <youpi> there's not much interesting unfortunately
+ <youpi> did you run thread apply all bt full ?
+ <youpi> (not just bt full)
+ <gg0> no just bt full
+ <gg0> http://paste.debian.net/plain/53790/
+ <gg0> wait, there's a child
+ <gg0> damn ctrl-c'ing while it was loading symbols made it crash :/
+ <gg0> restarted testsuite
+ <gg0> isn't it interesting that failed tests fail only if testsuite
+ runs from beginning, whereas if run singularly, they succeed?
+ <gg0> as it got out of whatever resources
+ <gg0> youpi: http://paste.debian.net/plain/53798/
+ <youpi> the interesting part is actually right at the top
+ <youpi> it's indeed stuck in the critical section spinlock
+ <youpi> question being what is keeping it
+ <youpi> iirc I had already checked in the whole glibc code that all
+ paths which lock critical_section_lock actually release it in
+ all cases, but maybe I have missed some
+ <youpi> (I did find some missing paths, which I fixed)
+ <gg0> i guess the same check you and braunr talk about in
+ discussion just before this anchor
+ http://darnassus.sceen.net/~hurd-web/open_issues/glibc/#recvmmsg
+ <youpi> yes, but the issue we were discussing there is not what
+ happens here
+ <youpi> we would see another thread stuck in the other way roudn,
+ otherwise
+ <gg0> no way to get what is locking?
+ <youpi> no, that's not recorded
+ <gg0> and what about writing it somewhere right after getting the
+ lock?
+ <youpi> one will have to do that in all spots taking that lock
+ <youpi> but yes, that's the usual approach
+ <gg0> i would give it try but eglibc rebuild takes too much time,
+ that conflicts with my laziness
+ <gg0> i read even making locks timed would help
+
+ IRC, OFTC, #debian-hurd, 2013-10-09:
+
+ <gg0> so correct order would be:
+ <gg0> __spin_lock (&ss->lock); // locks sigstate
+ <gg0> __spin_lock (&ss->critical_section_lock);
+ <gg0> [do critical stuff]
+ <gg0> __spin_unlock (&ss->critical_section_lock);
+ <gg0> __spin_unlock (&ss->lock); // unlocks sigstate
+ <gg0> ?
+
+ <gg0> 21:44 < gg0> terceiro: backported to 2.0 (backport to 1.9 is
+ waiting) https://bugs.ruby-lang.org/issues/9000
+ <gg0> 21:46 < gg0> that means that if you take a 2.0 snapshot,
+ it'll build fine on hurd (unless you introduce maketestall as in
+ 1.9, that would make it get stuck like 1.9)
+ <gg0> 21:48 < terceiro> gg0: nice
+ <gg0> 21:48 < terceiro> I will try to upload a snapshot as soon as
+ I can
+ <gg0> 21:52 < gg0> no problem. you might break my "conditional
+ satisfaction" by adding maketestall. better if you do that on
+ next+1 upload so we'll have at least one 2.0 built :)
+
+ <gg0> would it be a problem granting me access to a porter box to
+ rebuild eglibc+ruby2.0?
+ <gg0> i'm already doing it on another vm but host often loses power
+ <pinotree> you cannot install random stuff on a porterbox though
+ <gg0> i know i'd just need build-deps of eglibc+ruby2.0 i guess
+ <gg0> (already accessed to porter machines in the past, account
+ lele, mips iirc)
+ <gg0> ldap should remember that
+ <gg0> don't want to disturb anyone else work btw. if it's not a
+ problem, nice. otherwise no problem
+ <pinotree> please send a request to admin@exodar.debian.net so it
+ is not forgotten
+ <gg0> following this one would be too "official"?
+ http://dsa.debian.org/doc/guest-account/
+ <pinotree> hurd is not a release architecture, so hurd machines are
+ not managed by DSA
+ <gg0> ok
+ <pinotree> the general procedure outlines is ok though, just need
+ to be sent to the address above
+ <gg0> sent
+ <gg0> (1st signed mail with mutt, in the worst case i've attached
+ passphrase :))
+ <youpi> gg0: could you send me an ssh key?
+ <pinotree> no alioth account?
+ <youpi> yes, but EPERM
+ <gg0> youpi: sent to youpi@
+ <youpi> youpi@ ?
+ <gg0> (... which doesn't exist :/)
+ <gg0> sthibault@
+ <youpi> please test gg0-guest@exodar.debian.net ?
+ <youpi> (I'd rather not adduser the ldap name, who knows what might
+ happen when you get your DD account)
+ <gg0> i'm in. thanks
+ <youpi> you're welcome
+ <gg0> ldap users need to be adduser'ed?
+ <youpi> I'm not getting your ldap user account from ud-replicate,
+ at least
+ <gg0> (btw i never planned to apply nm, i'd be honoured but i
+ simply think not to deserve it)
+ <youpi> never say never ;)
+ <gg0> bah i like failing. that would be a success. i can't :)
+ <gg0> gg0-guest@exodar:~$ dchroot
+ <gg0> E: Access not authorised
+ <gg0> I: You do not have permission to access the schroot service.
+ <gg0> I: This failure will be reported.
+ <youpi> ah, right, iirc I need to add you somewhere
+ <youpi> gg0: please retry?
+ <gg0> works
+ <youpi> good
+ <gg0> are there already eglibc+ruby2.0 build-deps?
+ <youpi> yes
+ <gg0> oh that means i should do something myself now :)
+ <youpi> yep, that had to happen at some point :)
+ <gg0> my laziness thanks: "at some point" is better than "now" :)
+
+ IRC, freenode, #hurd, 2013-10-10:
+
+ <gg0> ok just reproduced the
+ former. ../sysdeps/mach/hurd/jmp-unwind.c:53 waits
+ <braunr> 20:37 < braunr> gg0: does ruby create and destroy threads
+ ?
+ <gg0> no idea
+ <gg0> braunr: days ago you and youpi talked about locking order
+ (just before this anchor
+ http://darnassus.sceen.net/~hurd-web/open_issues/glibc/#recvmmsg)
+ <braunr> oh right
+ <gg0> <youpi> could you submit the fix for jmp-unwind.c to
+ upstream?
+ <braunr> it didn't made it in the todo list
+ <gg0> so correct order is in hurd_thread_cancel, right?
+ <braunr> sorry about that
+ <braunr> we need to make a pass to make sure it is
+ <gg0> that means locking first ss->critical_section_lock _then_
+ ss->lock
+ <gg0> correct?
+ <braunr> but considering how critical hurd_thread_cancel is, i
+ expect so
+
+ <gg0> i get the same deadlock by swapping locks
+ <gg0> braunr: youpi: fyi ^
+ <gg0> 20:51 < braunr> 20:37 < braunr> gg0: does ruby create and
+ destroy threads ?
+ <gg0> how could i check it?
+ <braunr> gg0: ps -eflw
+ <youpi> gg0: that's not surprising, since in the b acktrace you
+ posted there isn't another thread locked in the other order
+ <youpi> so it's really that somehow the thread is already in
+ critical sesction
+ <braunr> youpi: you mean there is ?
+ <braunr> ah, it's not the same bug
+ <youpi> no, in what he posted, no other thread is stuck
+ <youpi> so it's not a locking order
+ <youpi> just that the critical section is actually busy
+ <gg0> youpi: ack
+ <gg0> braunr: what's the other bug? ext2fs one?
+ <braunr> gg0: idk
+ <gg0> braunr: thanks. doesn't show threads (found -T for that) but
+ at least doesn't limit columns number if piped (thanks to -w)
+ <braunr> it does
+ <braunr> there is a TH column
+ <gg0> ok thread count. -T gives more info
+
+ IRC, freenode, #hurd, 2013-10-24:
+
+ <youpi> ruby2.0 builds fine with the to-be-uploaded libc btw
+ <gg0> youpi: without d-ports patches? surprise me :)
+ <youpi> gg0: plain main archive source
+ <gg0> you did it. surprised
+ <gg0> ah ok you just pushed your tls. great!
+ <braunr> tls will fix a lot of things
+
+ * `sigaltstack`
+
+ IRC, freenode, #hurd, 2013-10-09:
+
+ <gnu_srs1> Hi, is sigaltstack() really supported, even if it is
+ defined as well as SA_ONSTACK?
+ <braunr> probably not
+ <braunr> well,
+ <braunr> i don't know actually, mistaking with something else
+ <braunr> it may be supported
+ <pinotree> iirc no
+ <gnu_srs1> pinotree: are you sure?
+ <pinotree> this is what i remember
+ <pinotree> if you want to be sure that $foo works, just do the
+ usual way: test it yourself
+ <gnu_srs1> found it: hurd/TODO: *** does sigaltstack/sigstack
+ really work? -- NO
+ <pinotree> well TODO is old and there were signal-related patches
+ by jk in the meanwhile, although i don't think they would have
+ changed this lack
+ <pinotree> in any case, test it
+ <gnu_srs1> anybody fluent in assembly? Looks like this code
+ destroys the stack: http://paste.debian.net/54331/
+ <braunr> gnu_srs1: why would it ?
+ <braunr> it does something special with the stack pointer but it
+ just looks like it aligns it to 16 bytes, maybe because of sse2
+ restrictions (recent gcc align the stack already anyway)
+ <gnu_srs1> Well, in that case it is the called function:
+ http://paste.debian.net/54341/
+ <braunr> how do you know there is a problem with the stack in the
+ first place ?
+ <gnu_srs1> tracing up to here, everything is OK. then esp and ebp
+ are destroyed.
+ <gnu_srs1> and single stepping goes backward until it segfaults
+ <braunr> "destroyed" ?
+ <gnu_srs1> zero if I remember correctly now. the x86 version built
+ for is i586, should that be changed to i486?
+ <braunr> this shouldn't change anything
+ <braunr> and they shouldn't get to 0
+ <braunr> use gdb to determine exactly which instruction resets the
+ stack pointer
+ <gnu_srs1> how to step into the assembly part? using 's' steps
+ through the function since no line information:
+ <gnu_srs1> Single stepping until exit from function
+ wine_call_on_stack,
+ <gnu_srs1> which has no line number information.
+ <braunr> gnu_srs1: use break on the address
+ <gnu_srs1> how do i get the address of where the assembly starts?
+
* `recvmmsg`/`sendmmsg` (`t/sendmmsg`)
From [[!message-id "20120625233206.C000A2C06F@topped-with-meat.com"]],
diff --git a/open_issues/glibc/t/tls-threadvar.mdwn b/open_issues/glibc/t/tls-threadvar.mdwn
index 7ce36f41..40d1463e 100644
--- a/open_issues/glibc/t/tls-threadvar.mdwn
+++ b/open_issues/glibc/t/tls-threadvar.mdwn
@@ -116,3 +116,40 @@ dropped altogether, and `__thread` directly be used in glibc.
## IRC, OFTC, #debian-hurd, 2013-09-23
<youpi> yay, errno threadvar conversion success
+
+
+## IRC, OFTC, #debian-hurd, 2013-10-05
+
+ <gg0_> youpi: any ETA for tls?
+ <youpi> gg0_: one can't have an ETA for bugfixing
+ <gg0_> i don't call them bugs if there's something missing to implement btw
+ <youpi> no, here it's bugs
+ <youpi> the implementation is already in the glibc branches in our
+ repository
+ <youpi> it just makes some important regressions
+
+
+## IRC, OFTC, #debian-hurd, 2013-10-07
+
+ <youpi> about tls, I've made some "progress": now I'm wondering how raise()
+ has ever been working before :)
+
+
+## IRC, OFTC, #debian-hurd, 2013-10-15
+
+ <youpi> good, reply_port tls is now ok
+ <youpi> last but not least, sigstate
+
+
+## IRC, OFTC, #debian-hurd, 2013-10-21
+
+ <youpi> started testsuite with threadvars dropped completely
+ <youpi> so far so good
+
+
+## IRC, OFTC, #debian-hurd, 2013-10-24
+
+ <youpi> ok, hurd boots with full-tls libc, no threadvars at all any more
+ <gg0> \o/
+ <gg0> good bye threadvars bugs, welcome tls ones ;)
+ <youpi> now I need to check that threads can really use another stack :)
diff --git a/open_issues/gnumach_page_cache_policy.mdwn b/open_issues/gnumach_page_cache_policy.mdwn
index 5e93887e..77e52ddb 100644
--- a/open_issues/gnumach_page_cache_policy.mdwn
+++ b/open_issues/gnumach_page_cache_policy.mdwn
@@ -811,3 +811,63 @@ License|/fdl]]."]]"""]]
<braunr> have*
<braunr> and even if laggy, it doesn't feel much more than the usual lag of
a network (ssh) based session
+
+
+# IRC, freenode, #hurd, 2013-10-08
+
+ <braunr> hmm i have to change what gnumach reports as being cached memory
+
+
+## IRC, freenode, #hurd, 2013-10-09
+
+ <braunr> mhmm, i'm able to copy files as big as 256M while building debian
+ packages, using a gnumach kernel patched for maximum memory usage in the
+ page cache
+ <braunr> just because i used --sync=30 in ext2fs
+ <braunr> a bit of swapping (around 40M), no deadlock yet
+ <braunr> gitweb is a bit slow but that's about it
+ <braunr> that's quite impressive
+ <braunr> i suspect thread storms might not even be the cataclysmic event
+ that we thought it was
+ <braunr> the true problem might simply be parallel fs synces
+
+
+## IRC, freenode, #hurd, 2013-10-10
+
+ <braunr> even with the page cache patch, memory filled, swap used, and lots
+ of cached objects (over 200k), darnassus is impressively resilient
+ <braunr> i really wonder whether we fixed ext2fs deadlock
+
+ <braunr> youpi: fyi, darnassus is currently running a patched gnumach with
+ the vm cache changes, in hope of reproducing the assertion errors we had
+ in the past
+ <braunr> i increased the sync interval of ext2fs to 30s like we discussed a
+ few months back
+ <braunr> and for now, it has been very resilient, failing only because of
+ the lack of kernel map entries after several heavy package builds
+ <gg0> wait the latter wasn't a deadlock it resumed after 1363.06 s
+ <braunr> gg0: thread storms can sometimes (rarely) fade and let the system
+ resume "normally"
+ <braunr> which is why i increased the sync interval to 30s, this leaves
+ time between two intervals for normal operations
+ <braunr> otherwise writebacks are queued one after the other, and never
+ processed fast enough for that queue to become empty again (except
+ rarely)
+ <braunr> youpi: i think we should consider applying at least the sync
+ interval to exodar, since many DDs are just unaware of the potential
+ problems with large IOs
+ <youpi> sure
+
+ <braunr> 222k cached objects (1G of cached memory) and darnassus is still
+ kicking :)
+ <braunr> youpi: those lock fixing patches your colleague sent last year
+ must have helped somewhere
+ <youpi> :)
+
+
+## IRC, freenode, #hurd, 2013-10-13
+
+ <youpi> braunr: how are your tests going with the object cache?
+ <braunr> youpi: not so good
+ <braunr> youpi: it failed after 2 days of straight building without a
+ single error output :/
diff --git a/open_issues/hurd_101.mdwn b/open_issues/hurd_101.mdwn
index 574a03ec..25822512 100644
--- a/open_issues/hurd_101.mdwn
+++ b/open_issues/hurd_101.mdwn
@@ -60,3 +60,41 @@ Not the first time that something like this is proposed...
<neal> how ipc works
<neal> and understand exactly what state is stored where
<zacts> ok
+
+
+# IRC, freenode, #hurd, 2013-10-12
+
+ <ahungry> Hi all, can anyone expand on
+ https://www.gnu.org/software/hurd/contributing.html - if I proceed with
+ the quick start and have the system running in a virtual image, how do I
+ go from there to being able to start tweaking the source (and recompiling
+ ) in a meaningful way?
+ <ahungry> Would I modify the source, compile within the VM and then what
+ would be the next step to actually test my new changes?
+ <braunr> ahungry: we use debian
+ <braunr> i suggest formatting your changes into patches, importing them
+ into debian packages, rebuilding those packages, and installing them over
+ the upstream ones
+ <ahungry> what about modifications to mach itself? or say I wanted to try
+ to work on the wifi drives - I would build the translator or module or
+ whatever and just add to the running instance of hurd?
+ <ahungry> s/drives/drivers
+ <braunr> same thing
+ <braunr> although
+ <braunr> during development, it's obviously a bit too expensive to rebuild
+ complete packages each time
+ <braunr> you can use the hurd on top of a gnumach kernel built completely
+ from upstream sources
+ <braunr> you need a few debian patches for the hurd itself
+ <braunr> a lot of them for glibc
+ <braunr> i usually create a temporary local branch with the debian patches
+ i need to make my code run
+ <braunr> and then create the true development branch itself from that one
+ <braunr> drivers are a a dark corner of the hurd
+ <braunr> i wouldn't recommend starting there
+ <braunr> but if you did, yes, you'd write a server to run drivers, and
+ start it
+ <braunr> you'd probably write a translator (which is a special kind of
+ server), yes
+ <ahungry> braunr: thanks for all the info, hittin the sack now but ill have
+ to set up a box and try to contribute
diff --git a/open_issues/hurd_init.mdwn b/open_issues/hurd_init.mdwn
index b0b58a70..cc06935c 100644
--- a/open_issues/hurd_init.mdwn
+++ b/open_issues/hurd_init.mdwn
@@ -214,3 +214,11 @@ License|/fdl]]."]]"""]]
<teythoon> I've been hacking on init/startup, I've looked into cleaning it
up
+
+
+## IRC, freenode, #hurd, 2013-10-07
+
+ <teythoon> braunr: btw, what do you think of my /hurd/startup proposal?
+ <braunr> i haven't read it in detail yet
+ <braunr> it's about separating init right ?
+ <teythoon> yes
diff --git a/open_issues/libpthread/t/fix_have_kernel_resources.mdwn b/open_issues/libpthread/t/fix_have_kernel_resources.mdwn
index 6f09ea0d..feea7c0d 100644
--- a/open_issues/libpthread/t/fix_have_kernel_resources.mdwn
+++ b/open_issues/libpthread/t/fix_have_kernel_resources.mdwn
@@ -413,3 +413,67 @@ Address problem mentioned in [[/libpthread]], *Threads' Death*.
<braunr> oh, git is multithreaded
<braunr> great
<braunr> so i've actually tested my libpthread patch quite a lot
+
+
+## IRC, freenode, #hurd, 2013-09-25
+
+ <braunr> on a side note, i was able to build gnumach/libc/hurd packages
+ with thread destruction
+ <teythoon> nice :)
+ <braunr> they boot and work mostly fine, although they add their own issues
+ <braunr> e.g. the comm field of the root ext2fs is empty
+ <braunr> ps crashes when trying to display threads
+ <braunr> but thread destruction actually works, i.e. servers (those that
+ are configured that away at least) go away after some time, and even
+ heavily used servers such as ext2fs dynamically scale over time :)
+
+
+## IRC, freenode, #hurd, 2013-10-10
+
+ <braunr> concerning threads, i think i figured out the last bugs i had with
+ thread destruction
+ <braunr> it should be well on its way to be merged by the end of the year
+
+
+## IRC, freenode, #hurd, 2013-10-11
+
+ <gg0> braunr: is your thread destruction patch ready for testing?
+ <braunr> gg0: there are packages at my repository, yes
+ <braunr> but i still have hurd fixes to do before i polish it
+ <braunr> in particular, posix says returning from main() stops the entire
+ process and all other threads
+ <braunr> i didn't check that during the switch to pthreads, and ext2fs (and
+ maybe others) actually return from main but expect other threads to live
+ on
+ <braunr> this creates problems when the main thread is actually destroyed,
+ but not the process
+ <teythoon> braunr: tmpfs does something like that, but calls pthread_exit
+ at the end of main
+ <braunr> same effect
+ <braunr> this was fine with cthreads, but must be changed with pthreads
+ <braunr> and libpthread must be fixed to enforce it
+ <braunr> (or libc)
+
+ <braunr> diskfs_startup_diskfs should probably be changed to reuse the main
+ thread instead of returning
+
+
+## IRC, freenode, #hurd, 2013-10-19
+
+ <zacts> I know what threads are, but what is 'thread destruction'?
+ <braunr> the hurd currently never destroys individual threads
+ <braunr> they're destroyed when tasks are destroyed
+ <braunr> if the number of threads in a task peaks at a high number, say
+ thousands of them, they'll remain until the task is terminated
+ <braunr> such tasks are usually file systems, normally never restarted (and
+ in the case of the root file system, not restartable)
+ <braunr> this results in a form of leak
+ <braunr> another effect of this leak is that servers which should go away
+ because of inactivity still remain
+ <braunr> since thread destruction doesn't actually work, the debian package
+ uses a patch to prevent worker threads from timeouting
+ <braunr> and to finish with, since thread destruction actually doesn't
+ work, normal (unpatched) applications that destroy threads are certainly
+ failing bad
+ <braunr> i just need to polish a few things, wait for youpi to finish his
+ work on TLS to resolve conflicts, and that will be all
diff --git a/open_issues/lsof.mdwn b/open_issues/lsof.mdwn
index 2cbf2302..2651932d 100644
--- a/open_issues/lsof.mdwn
+++ b/open_issues/lsof.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -11,3 +11,41 @@ License|/fdl]]."]]"""]]
We don't have a `lsof` tool. Perhaps we could cook something with having a
look at which ports are open at the moment (as [[`portinfo`|hurd/portinfo]]
does, for example)?
+
+
+# IRC, freenode, #hurd, 2013-10-16
+
+ <teythoon> braunr: there's something I've been working on, it's not yet
+ finished but usable
+ <teythoon> http://paste.debian.net/58266/
+ <teythoon> it graphs port usage
+ <teythoon> it's a bit heavy on the dependency-side though...
+ <braunr> but
+ <braunr> is it able to link rights from different ipc spaces ?
+ <teythoon> no
+ <teythoon> what do you mean exactly?
+ <braunr> know that send right 123 in task 1 refers to receive right 321 in
+ task 2
+ <braunr> basically, lsof
+ <braunr> i'm not sure it's possible right now, and that's what we'd really
+ need
+ <teythoon> does the kernel hand out this information?
+ <braunr> ^
+ <teythoon> right, I'm not sure it's possible either
+ <braunr> but a graph maker in less than 300 is cute :)
+ <braunr> 300 lines*
+ <teythoon> well, it leverages pymatplotlib or something, it needs half of
+ the pythonverse ;)
+ <braunr> lsof and pmap and two tools we really lack on the hurd
+ <teythoon> what does portinfo --translate=PID do?
+ <braunr> i guess it asks proc so that ports that refer to task actually
+ give useful info
+ <braunr> hml
+ <braunr> no
+ <braunr> doesn't make sense to give a pid in this case
+ <braunr> teythoon: looks like it does what we talked about
+ <teythoon> :)
+ <braunr> teythoon: the output looks a bit weird anyway, i think we need to
+ look at the code to be sure
+ <teythoon> braunr: this is what aptitude update looks like:
+ https://teythoon.cryptobitch.de/portmonitor/aptitude_portmonitor.svg
diff --git a/open_issues/mach-defpager_swap.mdwn b/open_issues/mach-defpager_swap.mdwn
index 7d3b001c..6e4dc088 100644
--- a/open_issues/mach-defpager_swap.mdwn
+++ b/open_issues/mach-defpager_swap.mdwn
@@ -18,3 +18,24 @@ License|/fdl]]."]]"""]]
<lifeng> I allocated a 5GB partition as swap, but hurd only found 1GB
<youpi> use 2GiB swaps only, >2Gib are not supported
<youpi> (and apparently it just truncates the size, to be investigated)
+
+## IRC, freenode, #hurd, 2013-10-25
+
+ <C-Keen> mkswap truncated the swap partiton to 2GB
+ <teythoon> :/
+ <teythoon> have you checked with 'free' ?
+ <teythoon> I have a 4gb swap partition on one of my boxes
+ <C-Keen> how did you create it?
+ <C-Keen> 2gig swap alright
+ <C-Keen> according to free
+
+
+# Swap Files
+
+## IRC, freenode, #hurd, 2013-10-25
+
+ <braunr> C-Keen: swapfiles are not to work very badly on the hurd
+ <braunr> swapfiles cause recursion and reservation problems on every system
+ but on the hurd, we just never took the time to fix the swap code
+
+Same issues as we generally would have with `hurd-defpager`?
diff --git a/open_issues/multiprocessing.mdwn b/open_issues/multiprocessing.mdwn
index 0ac7f195..eaaa2289 100644
--- a/open_issues/multiprocessing.mdwn
+++ b/open_issues/multiprocessing.mdwn
@@ -17,7 +17,7 @@ for applying multiprocessing. That is, however, only true from a first and
inexperienced point of view: there are many difficulties.
-IRC, freenode, #hurd, August / September 2010
+# IRC, freenode, #hurd, August / September 2010
<marcusb> silver_hook: because multi-server systems depend on inter-process
communication, and inter-process communication is many times more
@@ -32,7 +32,7 @@ IRC, freenode, #hurd, August / September 2010
serious research challenges
-IRC, freenode, #hurd, 2011-07-26
+# IRC, freenode, #hurd, 2011-07-26
< braunr> 12:03 < CTKArcher> and does the hurd take more advantages in a
multicore architecture than linux ?
@@ -57,7 +57,7 @@ IRC, freenode, #hurd, 2011-07-26
< braunr> (here, thread migration means being dispatched on another cpu)
-debian-hurd list
+# debian-hurd list
On Thu, Jan 02, 2003 at 05:40:00PM -0800, Thomas Bushnell, BSG wrote:
> Georg Lehner writes:
diff --git a/open_issues/performance.mdwn b/open_issues/performance.mdwn
index ae05e128..772fd865 100644
--- a/open_issues/performance.mdwn
+++ b/open_issues/performance.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2010, 2011, 2012 Free Software Foundation,
+[[!meta copyright="Copyright © 2010, 2011, 2012, 2013 Free Software Foundation,
Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
@@ -44,6 +44,8 @@ call|/glibc/fork]]'s case.
* [[metadata_caching]]
+ * [[community/gsoc/project_ideas/object_lookups]]
+
---
diff --git a/open_issues/performance/io_system/read-ahead.mdwn b/open_issues/performance/io_system/read-ahead.mdwn
index cd39328f..05a58f2e 100644
--- a/open_issues/performance/io_system/read-ahead.mdwn
+++ b/open_issues/performance/io_system/read-ahead.mdwn
@@ -3031,3 +3031,13 @@ License|/fdl]]."]]"""]]
<mcsim> so, add?
<braunr> if that's what you want to do, ok
<braunr> i'll think about your initial question tomorrow
+
+
+## IRC, freenode, #hurd, 2013-09-30
+
+ <antrik> talking about which... did the clustered I/O work ever get
+ concluded?
+ <braunr> antrik: yes, mcsim was able to finish clustered pageins, and it's
+ still on my TODO list
+ <braunr> it will get merged eventually, now that the large store patch has
+ also been applied
diff --git a/open_issues/performance/microkernel_multi-server.mdwn b/open_issues/performance/microkernel_multi-server.mdwn
index 111d2b88..0382c835 100644
--- a/open_issues/performance/microkernel_multi-server.mdwn
+++ b/open_issues/performance/microkernel_multi-server.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -12,7 +12,8 @@ License|/fdl]]."]]"""]]
Performance issues due to the microkernel/multi-server system architecture?
-IRC, freenode, #hurd, 2011-07-26
+
+# IRC, freenode, #hurd, 2011-07-26
< CTKArcher> I read that, because of its microkernel+servers design, the
hurd was slower than a monolithic kernel, is that confirmed ?
@@ -45,3 +46,181 @@ IRC, freenode, #hurd, 2011-07-26
< braunr> but in 95, processors weren't that fast compared to other
components as they are now
< youpi> while disk/mem haven't evovled so fast
+
+
+# IRC, freenode, #hurd, 2013-09-30
+
+ <snadge> ok.. i noticed when installing debian packages in X, the mouse
+ lagged a little bit
+ <snadge> that takes me back to classic linux days
+ <snadge> it could be a side effect of running under virtualisation who
+ knows
+ <braunr> no
+ <braunr> it's because of the difference of priorities between server and
+ client tasks
+ <snadge> is it simple enough to increase the priority of the X server?
+ <snadge> it does remind me of the early linux days.. people were more
+ interested in making things work, and making things not crash.. than
+ improving the desktop interactivity or responsiveness
+ <snadge> very low priority :P
+ <braunr> snadge: actually it's not the difference in priority, it's the
+ fact that some asynchronous processing is done at server side
+ <braunr> the priority difference just gives more time overall to servers
+ for that processing
+ <braunr> snadge: when i talk about servers, i mean system (hurd) servers,
+ no x
+ <snadge> yeah.. linux is the same.. in the sense that, that was its
+ priority and focus
+ <braunr> snadge: ?
+ <snadge> servers
+ <braunr> what are you talking about ?
+ <snadge> going back 10 years or so.. linux had very poor desktop
+ performance
+ <braunr> i'm not talking about priorities for developers
+ <snadge> it has obviously improved significantly
+ <braunr> i'm talking about things like nice values
+ <snadge> right.. and some of the modifications that have been done to
+ improve interactivity of an X desktop, are not relevant to servers
+ <braunr> not relevant at all since it's a hurd problem, not an x problem
+ <snadge> yeah.. that was more of a linux problem too, some time ago was the
+ only real point i was making.. a redundant one :p
+ <snadge> where i was going with that.. was desktop interactivity is not a
+ focus for hurd at this time
+ <braunr> it's not "desktop interactivity"
+ <braunr> it's just correct scheduling
+ <snadge> is it "correct" though.. the scheduler in linux is configurable,
+ and selectable
+ <snadge> depending on the type of workload you expect to be doing
+ <braunr> not really
+ <snadge> it can be interactive, for desktop loads.. or more batched, for
+ server type loads.. is my basic understanding
+ <braunr> no
+ <braunr> that's the scheduling policy
+ <braunr> the scheduler is cfs currently
+ <braunr> and that's the main difference
+ <braunr> cfs means completely fair
+ <braunr> whereas back in 2.4 and before, it was a multilevel feedback
+ scheduler
+ <braunr> i.e. a scheduler with a lot of heuristics
+ <braunr> the gnumach scheduler is similar, since it was the standard
+ practice from unix v6 at the time
+ <braunr> (gnumach code base comes from bsd)
+ <braunr> so 1/ we would need a completely fair scheduler too
+ <braunr> and 2/ we need to remove asynchronous processing by using mostly
+ synchronous rpc
+ <snadge> im just trying to appreciate the difference between async and sync
+ event processing
+ <braunr> on unix, the only thing asynchronous is signals
+ <braunr> on the hurd, simply cancelling select() can cause many
+ asynchronous notifications at the server to remove now unneeded resources
+ <braunr> when i say cancelling select, i mean one or more fds now have
+ pending events, and the others must be cleaned
+ <snadge> yep.. thats a pretty fundamental change though isnt it? .. if im
+ following you, you're talking about every X event.. so mouse move,
+ keyboard press etc etc etc
+ <snadge> instead of being handled async.. you're polling for them at some
+ sort of timing interval?
+ <snadge> never mind.. i just read about async and sync with regards to rpc,
+ and feel like a bit of a noob
+ <snadge> async provides a callback, sync waits for the result.. got it :p
+ <snadge> async is resource intensive on hurd for the above mentioned
+ reasons.. makes sense now
+ <snadge> how about optimising the situation where a select is cancelled,
+ and deferring the signal to the server to clean up resources until a
+ later time?
+ <snadge> so like java.. dont clean up, just make a mess
+ <snadge> then spend lots of time later trying to clean it up.. sounds like
+ my life ;)
+ <snadge> reuse stale objects instead of destroying and recreating them, and
+ all the problems associated with that
+ <snadge> but if you're going to all these lengths to avoid sending messages
+ between processes
+ <snadge> then you may as well just use linux? :P
+ <snadge> im still trying to wrap my head around how converting X to use
+ synchronous rpc calls will improve responsiveness
+ <pinotree> what has X to do with it?
+ <snadge> nothing wrong with X.. braunr just mentioned that hurd doesnt
+ really handle the async calls so well
+ <snadge> there is more overhead.. that it would be more efficient on hurd,
+ if it uses sync rpc instead
+ <snadge> and perhaps a different task scheduler would help also
+ <snadge> ala cfs
+ <snadge> but i dont think anyone is terribly motivated in turning hurd into
+ a desktop operating system just yet.. but i could be wrong ;)
+ <braunr> i didn't say that
+ <snadge> i misinterpreted what you said then .. im not surprised, im a
+ linux sysadmin by trade.. and have basic university OS understanding (ie
+ crap all) at a hobbyist level
+ <braunr> i said there is asynchronous processing (i.e. server still have
+ work to do even when there is no client)
+ <braunr> that processing mostly comes from select requests cancelling what
+ they installed
+ <braunr> ie.e. you select fd 1 2 3, even on 2, you cancel on 1 and 3
+ <braunr> those cancellations aren't synchronous
+ <braunr> the client deletes ports, and the server asynchronously receives
+ dead name notifications
+ <braunr> since servers have a greater priority, these notifications are
+ processed before the client can continue
+ <braunr> which is what makes you feel lag
+ <braunr> X is actually a client here
+ <braunr> when i say server, i mean hurd servers
+ <braunr> the stuff implementing sockets and files
+ <braunr> also, you don't need to turn the hurd into a desktop os
+ <braunr> any correct way to do fair scheduling will do
+ <snadge> can the X client be made to have a higher priority than the hurd
+ servers?
+ <snadge> or perhaps something can be added to hurd to interface with X
+ <azeem_> well, the future is wayland
+ <snadge> ufs .. unfair scheduling.. give priority to X over everything else
+ <snadge> hurd almost seams ideal for that idea.. since the majority of the
+ system is seperated from the kernel
+ <snadge> im likely very wrong though :p
+ <braunr> snadge: the reason we elevated the priority of servers is to avoid
+ delaying the processing of notifications
+ <braunr> because each notification can spawn a server thread
+ <braunr> and this lead to cases where processing notifications was so slow
+ that spawning threads would occur more frequently, leading to the server
+ exhausting its address space because of thread stacks
+ <snadge> cant it wait for X though? .. or does it lead to that situation
+ you just described
+ <braunr> we should never need such special cases
+ <braunr> we should remove async notifications
+ <snadge> my logic is this.. if you're not running X then it doesnt
+ matter.. if you are, then it might.. its sort of up to you whether you
+ want priority over your desktop interface or whether it can wait for more
+ important things, which creates perceptible lag
+ <braunr> snadge: no it doesn't
+ <braunr> X is clearly not the only process involved
+ <braunr> the whole chain should act synchronously
+ <braunr> from the client through the server through the drivers, including
+ the file system and sockets, and everything that is required
+ <braunr> it's a general problem, not specific to X
+ <snadge> right.. from googling around, it looks like people get very
+ excited about asyncronous
+ <snadge> there was a move to that for some reason.. it sounds great in
+ theory
+ <snadge> continue processing something else whilst you wait for a
+ potentially time consuming process.. and continue processing that when
+ you get the result
+ <snadge> its also the only way to improve performance with parallelism?
+ <snadge> which is of no concern to hurd at this time
+ <braunr> snadge: please don't much such statements when you don't know what
+ you're talking about
+ <braunr> it is a concern
+ <braunr> and yes, async processing is a way to improve performance
+ <braunr> but don't mistake async rpc and async processing
+ <braunr> async rpc simply means you can send and receive at any time
+ <braunr> sync means you need to recv right after send, blocking until a
+ reply arrives
+ <braunr> the key word here is *blocking*ù
+ <snadge> okay sure.. that makes sense
+ <snadge> what is the disadvantage to doing it that way?
+ <snadge> you potentially have more processes that are blocking?
+ <braunr> a system implementing posix such as the hurd needs signals
+ <braunr> and some event handling facility like select
+ <braunr> implementing them synchronously means a thread ready to service
+ these events
+ <braunr> the hurd currently has such a message thread
+ <braunr> but it's complicated and also a scalability concern
+ <braunr> e.g. you have at least two thread per process
+ <braunr> bbl
diff --git a/open_issues/pthread_atfork.mdwn b/open_issues/pthread_atfork.mdwn
index 1b656f05..06b9d6c6 100644
--- a/open_issues/pthread_atfork.mdwn
+++ b/open_issues/pthread_atfork.mdwn
@@ -18,3 +18,89 @@ can probably be borrowed from `nptl/sysdeps/unix/sysv/linux/register-atfork.c`.
<pinotree> SRCDIR/opal/mca/memory/linux/arena.c:387: warning: warning:
pthread_atfork is not implemented and will always fail
+
+
+# Samuel's implementation
+
+TODO.
+
+
+## IRC, OFTC, #debian-hurd, 2013-10-08
+
+ <pinotree> youpi: if you need/want to test your pthread_atfork
+ implementation, you can check libposix-atfork-perl and its test suite
+ (whose test 004 hangs now, with eglibc -93)
+ <youpi> while it failed previously indeed
+ <youpi> we might simply need to rebuild perl against it
+ <youpi> (I see ifdef pthread_atfork in perl)
+
+
+## IRC, freenode, #hurd, 2013-10-16
+
+ <teythoon> tschwinge: I'd love to try your cross-gnu tool, the wiki page
+ suggests that the list of required source packages is outdated. can you
+ give me some hints?
+ <teythoon> tschwinge: I got this error running cross-gnu:
+ http://paste.debian.net/58303/
+ make[4]: Leaving directory `/home/teythoon/repos/hurd/cross/src/glibc/setjmp'
+ make subdir=string -C ../string ..=../ objdir=/home/teythoon/repos/hurd/cross/obj/glibc -f Makefile -f ../elf/rtld-Rules rtld-all rtld-modules='rtld-strchr.os rtld-strcmp.os rtld-strcpy.os rtld-strlen.os rtld-strnlen.os rtld-memchr.os rtld-memcmp.os rtld-memmove.os rtld-memset.os rtld-mempcpy.os rtld-stpcpy.os rtld-memcpy.os rtld-rawmemchr.os rtld-argz-count.os rtld-argz-extract.os rtld-stpncpy.os'
+ make[4]: Entering directory `/home/teythoon/repos/hurd/cross/src/glibc/string'
+ make[4]: Leaving directory `/home/teythoon/repos/hurd/cross/src/glibc/string'
+ make[4]: Entering directory `/home/teythoon/repos/hurd/cross/src/glibc/string'
+ make[4]: Nothing to be done for `rtld-all'.
+ make[4]: Leaving directory `/home/teythoon/repos/hurd/cross/src/glibc/string'
+ make[3]: Leaving directory `/home/teythoon/repos/hurd/cross/src/glibc/elf'
+ i686-pc-gnu-gcc -shared -static-libgcc -Wl,-O1 -Wl,-z,defs -Wl,-dynamic-linker=/lib/ld.so.1 -B/home/teythoon/repos/hurd/cross/obj/glibc/csu/ -Wl,--version-script=/home/teythoon/repos/hurd/cross/obj/glibc/libc.map -Wl,-soname=libc.so.0.3 -Wl,-z,combreloc -Wl,-z,relro -Wl,--hash-style=both -nostdlib -nostartfiles -e __libc_main -L/home/teythoon/repos/hurd/cross/obj/glibc -L/home/teythoon/repos/hurd/cross/obj/glibc/math -L/home/teythoon/repos/hurd/cross/obj/glibc/elf -L/home/teythoon/repos/hurd/cross/obj/glibc/dlfcn -L/home/teythoon/repos/hurd/cross/obj/glibc/nss -L/home/teythoon/repos/hurd/cross/obj/glibc/nis -L/home/teythoon/repos/hurd/cross/obj/glibc/rt -L/home/teythoon/repos/hurd/cross/obj/glibc/resolv -L/home/teythoon/repos/hurd/cross/obj/glibc/crypt -L/home/teythoon/repos/hurd/cross/obj/glibc/mach -L/home/teythoon/repos/hurd/cross/obj/glibc/hurd -Wl,-rpath-link=/home/teythoon/repos/hurd/cross/obj/glibc:/home/teythoon/repos/hurd/cross/obj/glibc/math:/home/teythoon/repos/hurd/cross/obj/glibc/elf:/home/teythoon/repos/hurd/cross/obj/glibc/dlfcn:/home/teythoon/repos/hurd/cross/obj/glibc/nss:/home/teythoon/repos/hurd/cross/obj/glibc/nis:/home/teythoon/repos/hurd/cross/obj/glibc/rt:/home/teythoon/repos/hurd/cross/obj/glibc/resolv:/home/teythoon/repos/hurd/cross/obj/glibc/crypt:/home/teythoon/repos/hurd/cross/obj/glibc/mach:/home/teythoon/repos/hurd/cross/obj/glibc/hurd -o /home/teythoon/repos/hurd/cross/obj/glibc/libc.so -T /home/teythoon/repos/hurd/cross/obj/glibc/shlib.lds /home/teythoon/repos/hurd/cross/obj/glibc/csu/abi-note.o /home/teythoon/repos/hurd/cross/obj/glibc/elf/soinit.os /home/teythoon/repos/hurd/cross/obj/glibc/libc_pic.os /home/teythoon/repos/hurd/cross/obj/glibc/elf/sofini.os /home/teythoon/repos/hurd/cross/obj/glibc/elf/interp.os /home/teythoon/repos/hurd/cross/obj/glibc/elf/ld.so /home/teythoon/repos/hurd/cross/obj/glibc/mach/libmachuser-link.so /home/teythoon/repos/hurd/cross/obj/glibc/hurd/libhurduser-link.so -lgcc
+ /home/teythoon/repos/hurd/cross/obj/glibc/libc_pic.os: In function `__fork':
+ /home/teythoon/repos/hurd/cross/src/glibc/posix/../sysdeps/mach/hurd/fork.c:70: undefined reference to `__start__hurd_atfork_prepare_hook'
+ /home/teythoon/repos/hurd/cross/lib/gcc/i686-pc-gnu/4.8.1/../../../../i686-pc-gnu/bin/ld: /home/teythoon/repos/hurd/cross/obj/glibc/libc_pic.os: relocation R_386_GOTOFF against undefined hidden symbol `__start__hurd_atfork_prepare_hook' can not be used when making a shared object
+ /home/teythoon/repos/hurd/cross/lib/gcc/i686-pc-gnu/4.8.1/../../../../i686-pc-gnu/bin/ld: final link failed: Bad value
+ collect2: error: ld returned 1 exit status
+ make[2]: *** [/home/teythoon/repos/hurd/cross/obj/glibc/libc.so] Error 1
+ make[2]: Leaving directory `/home/teythoon/repos/hurd/cross/src/glibc/elf'
+ make[1]: *** [elf/subdir_lib] Error 2
+ make[1]: Leaving directory `/home/teythoon/repos/hurd/cross/src/glibc'
+ make: *** [all] Error 2
+ + rm -f /home/teythoon/repos/hurd/cross/sys_root/lib/ld.so
+ + exit 100
+
+ binutils-2.23.2,
+ gcc-4.8.1,
+ everything else is from git as specified in the wiki.
+
+
+## IRC, freenode, #hurd, 2013-10-24
+
+ <AliciaC> in recent glibc commits (tschwinge/Roger_Whittaker branch) there
+ are references to _hurd_atfork_* symbols in sysdeps/mach/hurd/fork.c, and
+ some _hurd_fork_* symbols, some of the _hurd_fork_* symbols seem to be
+ defined in Hurd's boot/frankemul.ld (mostly guessing by their names being
+ mentioned, I don't know linker script syntax), but those _hurd_atfork_*
+ symbols don't seem to be defined there, are they supposed to be defined
+ elsewhere or is th
+ <AliciaC> does anyone know where the _hurd_atfork_* group of symbols
+ referenced in glibc are defined (if anywhere)?
+ <youpi> AliciaC: it's the DEFINE_HOOK (_hurd_atfork_prepare_hook, (void));
+ in glibc/sysdeps/mach/hurd/fork.c
+ <AliciaC> hm, is that not just a declaration?
+ <youpi> no, it's a definition, as its name suggests :
+ <AliciaC> (despite the macro name)
+ <youpi> :)
+ <AliciaC> ok
+ <AliciaC> I should look into it more, I could have sworn I was getting
+ undefined references, but maybe the symbol names used are different from
+ those defined, but that'd be odd as well, in the same file and all
+ <AliciaC> I mean, I do get undefined references, but question is if it's to
+ things that should have been defined or not
+ <youpi> what undefined references do you gaT?
+ <youpi> s/gaT/get
+ <AliciaC> I'll get back to you once I have that system up again
+ <AliciaC> youpi: sysdeps/mach/hurd/fork.c:70: undefined reference to
+ `__start__hurd_atfork_prepare_hook'
+ <AliciaC> fork.c:70: 'RUN_HOOK (_hurd_atfork_prepare_hook, ());'
+ <AliciaC> DEFINE_HOOK (_hurd_atfork_prepare_hook, (void)); is higher up in
+ the file
+ <AliciaC> though there is also this message: build/libc_pic.os: relocation
+ R_386_GOTOFF against undefined hidden symbol
+ `__start__hurd_atfork_prepare_hook' can not be used when making a shared
+ object
diff --git a/open_issues/smp.mdwn b/open_issues/smp.mdwn
index a45a1e22..89474d25 100644
--- a/open_issues/smp.mdwn
+++ b/open_issues/smp.mdwn
@@ -37,3 +37,11 @@ See also the [[FAQ entry|faq/smp]].
## Richard, 2013-03-20
This task actually looks too big for a GSoC project.
+
+
+## IRC, freenode, #hurd, 2013-09-30
+
+ <braunr> also, while the problem with hurd is about I/O, it's actually a
+ lot more about caching, and even with more data cached in, the true
+ problem is contention, in which case having several processors would
+ actually slow things down even more
diff --git a/open_issues/strict_aliasing.mdwn b/open_issues/strict_aliasing.mdwn
index b7d39805..0e59f796 100644
--- a/open_issues/strict_aliasing.mdwn
+++ b/open_issues/strict_aliasing.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -29,3 +29,16 @@ License|/fdl]]."]]"""]]
issues (if gcc catches them all)
<tschwinge> The strict aliasing things should be fixed, yes. Some might be
from MIG.
+
+
+# IRC, freenode, #hurd, 2013-10-17
+
+ <braunr> we should build gnumach and the hurd with -fno-strict-aliasing
+ <pinotree> aren't the mig-generated stubs the only issues related to that?
+ <braunr> no
+ <teythoon> b/c we often have pointers of different type pointing to the
+ same address? for example code using libports?
+ <braunr> the old linux code, including pfinet, and even the hurd libraries,
+ use techniques that assume aliasing
+ <braunr> exactly
+ <teythoon> right, I agree
diff --git a/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn b/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn
index 7159551d..f40e0455 100644
--- a/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn
+++ b/open_issues/thread-cancel_c_55_hurd_thread_cancel_assertion___spin_lock_locked_ss_critical_section_lock.mdwn
@@ -50,3 +50,5 @@ IRC, unknown channel, unknown date:
result in others trying to take it...
<youpi> nope: look at the code :)
<youpi> or maybe the cancel_hook, but I really doubt it
+
+See discussion about *`critical_section_lock`* on [[glibc]].
diff --git a/open_issues/time.mdwn b/open_issues/time.mdwn
index 367db872..d9f1fa1d 100644
--- a/open_issues/time.mdwn
+++ b/open_issues/time.mdwn
@@ -837,3 +837,17 @@ not get a define for `HZ`, which is then defined with a fallback value of 60.
<nalaginrut> braunr: Guile2 works smoothly now, let me try something cool
with it
<braunr> nalaginrut: nice
+
+
+### IRC, OFTC, #debian-hurd, 2013-09-29
+
+ <pinotree> youpi: is the latest glibc carrying the changes related to
+ timing? what about gb guile-2.0 with it?
+ <youpi> it does
+ <youpi> so that was the only issue with guile?
+ <youpi> well at least we'll see
+ <pinotree> iirc yes
+ <pinotree> according to nalaginrut and the latest build log, it'd seem so
+ <youpi> started
+ <youpi> yay, guile-2.0 :)
+ <pinotree> yay
diff --git a/open_issues/wine.mdwn b/open_issues/wine.mdwn
index 65e6c584..f8bb469b 100644
--- a/open_issues/wine.mdwn
+++ b/open_issues/wine.mdwn
@@ -1,4 +1,5 @@
-[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2010, 2011, 2013 Free Software Foundation,
+Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -21,7 +22,7 @@ requirements Wine has: only libc / POSIX / etc., or if there are
allocation. There is kernel support for this,* however.
-IRC, freenode, #hurd, 2011-08-11
+# IRC, freenode, #hurd, 2011-08-11
< arethusa> I've been trying to make Wine work inside a Debian GNU/Hurd VM,
and to that end, I've successfully compiled the latest sources from Git
@@ -67,3 +68,13 @@ IRC, freenode, #hurd, 2011-08-11
< youpi> yes
< pinotree> (but that patch is lame)
+
+
+# IRC, freenode, #hurd, 2013-10-02
+
+ <gnu_srs> youpi: I've come a little further with wine, see debian bug
+ #724681 (same problem).
+ <gnu_srs> Now the problem is probably due to the specific address space
+ and stack issues to be
+ <gnu_srs> fixed for wine to run as braunr pointed out some months ago
+ (IRC?) when we discussed wine.
diff --git a/unix/process.mdwn b/unix/process.mdwn
index 21fbfc69..82034751 100644
--- a/unix/process.mdwn
+++ b/unix/process.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -8,13 +8,11 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
-A *UNIX process* is TODO.
+A *UNIX process* is a program in execution, that is, an instance running in an
+execution context.
Generally, especially in [[microkernel]]-based systems, the [[kernel]]'s idea
of a task is not as encompassing as a UNIX process, and will use additional
effort to enhance the kernel's primitive to a full-fledged UNIX model.
-
-A [[Mach task|microkernel/mach/task]] implements a part of a UNIX process.
-
In the GNU/Hurd, processes are based on [[Mach task|microkernel/mach/task]]s,
but are [[enhanced by the glibc|glibc/process]].