summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
Diffstat
-rw-r--r--hurd/debugging/rpctrace.mdwn37
-rw-r--r--hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn37
-rw-r--r--hurd/virtual_file_system/discussion.mdwn39
-rw-r--r--microkernel/mach/gnumach/memory_management.mdwn22
-rw-r--r--open_issues/anatomy_of_a_hurd_system.mdwn28
-rw-r--r--open_issues/ext2fs_page_cache_swapping_leak.mdwn88
-rw-r--r--open_issues/gnumach_memory_management.mdwn202
-rw-r--r--open_issues/libmachuser_libhurduser_rpc_stubs.mdwn50
-rw-r--r--open_issues/mig_portable_rpc_declarations.mdwn58
-rw-r--r--open_issues/mission_statement.mdwn12
-rw-r--r--open_issues/page_cache.mdwn73
-rw-r--r--open_issues/perl.mdwn50
-rw-r--r--open_issues/robustness.mdwn64
-rw-r--r--open_issues/syslog.mdwn27
-rw-r--r--open_issues/translator_stdout_stderr.mdwn32
15 files changed, 815 insertions, 4 deletions
diff --git a/hurd/debugging/rpctrace.mdwn b/hurd/debugging/rpctrace.mdwn
index f7136056..fd24f081 100644
--- a/hurd/debugging/rpctrace.mdwn
+++ b/hurd/debugging/rpctrace.mdwn
@@ -52,6 +52,43 @@ See `rpctrace --help` about how to use it.
<antrik> note that there is a number of known bugs in rpctrace, for which zhengda has sent patches... though I haven't reviewed all of them I think
<antrik> there are some nasty Mach operations that are really hard to proxy -- but I don't think the auth mechanism needs any of these...
+* IRC, freenode, #hurd, 2011-11-04
+
+ [[!taglink open_issue_documentation]]
+
+ <mcsim> hello. Are there any documentation about understanding output
+ of rpctrace?
+ <braunr> no
+ <braunr> you should read the source code, best doc available
+ <braunr> if you have too many numbers and almost no symbolc names,
+ you're lacking rpc definition lists
+ <braunr> check that the gnumach-common package is installed, as it
+ provides the gnumach definitions
+ <braunr> (the glibc ones are almost always available)
+ <braunr> with those two, you should be fine for the beginning
+ <mcsim> gnumach-common is installed. And what is the name for glibc
+ package for gnumach definitions.
+ <mcsim> Also I'm using libraries specified by LD_LIBRARY_PATH. Does it
+ make influence on absence of symbolic names?
+ <braunr> no
+ <braunr> rpctrace --help
+ <braunr> see the --rpc-list=FILE option
+ <braunr> the default lists are in /usr/share/msgids/, with the .msgids
+ extension
+ <braunr> $ dpkg -S msgids
+ <braunr> gnumach-common: /usr/share/msgids/gnumach.msgids
+ <braunr> hurd: /usr/share/msgids/hurd.msgids
+ <braunr> ok, glibc has none, it's the hurd
+ <braunr> for more details about the output, read the source code
+ <braunr> it shouldn't be that hard to grasp
+ <mcsim> -I /usr/share/msgids helped
+ <mcsim> thank you
+ <braunr> it shouldn't have, it's the default path
+ <mcsim> but symbolic names appeared
+ <braunr> well, that's weird :)
+ <pinotree> braunr: the output of rpctrace --help should tell the
+ default dir for msgids
+
# See Also
diff --git a/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn
index a9317c21..5228515f 100644
--- a/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn
+++ b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn
@@ -233,3 +233,40 @@ See also: [[open_issues/resource_management_problems/pagers]].
I have never looked at it.
[[open_issues/mach-defpager_vs_defpager]].
+
+
+# IRC, freenode, #hurd, 2011-11-08
+
+ <mcsim> who else uses defpager besides tmpfs and kernel?
+ <braunr> normally, nothing directly
+ <mcsim> than why tmpfs should use defpager?
+ <braunr> it's its backend
+ <braunr> backign store rather
+ <braunr> the backing store of most file systems are partitions
+ <braunr> tmpfs has none, it uses the swap space
+ <mcsim> if we allocate memory for tmpfs using vm_allocate, will it be able
+ to use swap partition?
+ <braunr> it should
+ <braunr> vm_allocate just maps anonymous memory
+ <braunr> anonymous memory uses swap space as its backing store too
+ <braunr> but be aware that this part of the vm system is known to have
+ deficiencies
+ <braunr> which is why all mach based implementations have rewritten their
+ default pager
+ <mcsim> what kind of deficiencies?
+ <braunr> bugs
+ <braunr> and design issues, making anonymous memory fragmentation horrible
+ <antrik> mcsim: vm_allocate doesn't return a memory object; so it can't be
+ passed to clients for mmap()
+ <mcsim> antrik: I use vm_allocate in pager_read_page
+ <antrik> mcsim: well, that means that you have to actually implement a
+ pager yourself
+ <antrik> also, when the kernel asks the pager to write back some pages, it
+ expects the memory to become free. if you are "paging" to ordinary
+ anonymous memory, this doesn't happen; so I expect it to have a very bad
+ effect on system performance
+ <antrik> both can be avoided by just passing a real anonymous memory
+ object, i.e. one provided by the defpager
+ <antrik> only problem is that the current defpager implementation can't
+ really handle that...
+ <antrik> at least that's my understanding of the situation
diff --git a/hurd/virtual_file_system/discussion.mdwn b/hurd/virtual_file_system/discussion.mdwn
new file mode 100644
index 00000000..9e12d01e
--- /dev/null
+++ b/hurd/virtual_file_system/discussion.mdwn
@@ -0,0 +1,39 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_documentation]]
+
+IRC, freenode, #hurd, 2011-11-12:
+
+ <sea4ever> So hurd implements a 'transparent translator' somewhere which
+ just passes all IO calls to the posix IO I'm used to? (i.e. read, write,
+ open, close, etc.?)
+ <youpi> it's the normal way of operation
+ <youpi> glibc's read() doesn't do a system call, it always does an RPC to
+ the underlying translator
+ <youpi> be it ext2fs for /, or your foobarfs for your node
+ <sea4ever> Ok that makes sense. How does one program know which translator
+ it should refer to though?
+ <sea4ever> the read() call magically knows which process to invoke?
+ <youpi> the / translator is always known
+ <youpi> and then you ask /'s translator about /home, then /home/you, then
+ /home/you/foobar
+ <youpi> it tells you which other translator tyou have to contact
+ <youpi> that's on open
+ <sea4ever> It's a tree! Ok.
+ <youpi> the notion of fd is then simply knowing the translator
+ <sea4ever> Right. 'file descriptor' is now 'translator address descriptor'
+ maybe.
+ <youpi> it's glibc which knows about FDs, nothing else knows
+ <youpi> yes
+ <youpi> actually an RPC port, simply
+ <sea4ever> I want to try out the new RPC mechanism that mach implements
+ <youpi> err, which "new" RPC ?
+ <youpi> mach's RPCs are very old actually :)
diff --git a/microkernel/mach/gnumach/memory_management.mdwn b/microkernel/mach/gnumach/memory_management.mdwn
index 43b99d83..ca2f42c4 100644
--- a/microkernel/mach/gnumach/memory_management.mdwn
+++ b/microkernel/mach/gnumach/memory_management.mdwn
@@ -80,3 +80,25 @@ IRC, freenode, #hurd, 2011-06-09
<braunr> wow
<braunr> i remember the linux support for 4G/4G split when there was enough
RAM to fill the kernel space with struct page entries
+
+
+IRC, freenode, #hurd, 2011-11-12
+
+ <youpi> well, the Hurd doesn't "artificially" limits itself to 1.5GiB
+ memory
+ <youpi> i386 has only 4GiB addressing space
+ <youpi> we currently chose 2GiB for the kernel and 2GiB for the userspace
+ <youpi> since kernel needs some mappings, that leaves only 1.5GiB usable
+ physical memory
+ <sea4ever`> Hm? 2GiB for kernel, 2GiB for userspace, 500MiB are used for
+ what?
+ <youpi> for mappings
+ <youpi> such as device iomap
+ <youpi> contiguous buffer allocation
+ <youpi> and such things
+ <sea4ever`> Ah, ok. You map things in kernel space into user space then.
+ <youpi> linux does the same without the "bigmem" support
+ <youpi> no, just in kernel space
+ <youpi> kernel space is what determines how much physical memory you can
+ address
+ <youpi> unless using the linux-said-awful "bigmem" support
diff --git a/open_issues/anatomy_of_a_hurd_system.mdwn b/open_issues/anatomy_of_a_hurd_system.mdwn
index 46526641..13599e19 100644
--- a/open_issues/anatomy_of_a_hurd_system.mdwn
+++ b/open_issues/anatomy_of_a_hurd_system.mdwn
@@ -87,7 +87,7 @@ RPC stubs.
More stuff like [[hurd/IO_path]].
---
+---
IRC, freenode, #hurd, 2011-10-18:
@@ -96,3 +96,29 @@ IRC, freenode, #hurd, 2011-10-18:
<antrik> short version: grub loads mach, ext2, and ld.so/exec; mach starts
ext2; ext2 starts exec; ext2 execs a few other servers; ext2 execs
init. from there on, it's just standard UNIX stuff
+
+---
+
+IRC, OFTC, #debian-hurd, 2011-11-02:
+
+ <sekon_> is __dir_lookup a RPC ??
+ <sekon_> where can i find the source of __dir_lookup ??
+ <sekon_> grepping most gives out rvalue assignments
+ <sekon_> -assignments
+ <sekon_> but in hurs/fs.h it is used as a function ??
+ <pinotree> it should be the mig-generated function for that rpc
+ <sekon_> how do i know how its implemented ??
+ <sekon_> is there any way to delve deeprer into mig-generated functions
+ <tschwinge> sekon_: The MIG-generated stuff will either be found in the
+ package's build directory (if it's building it for themselves), or in the
+ glibc build directory (libhurduser, libmachuser; which are all the
+ available user RPC stubs).
+ <tschwinge> sekon_: The implementation can be found in the various Hurd
+ servers/libraries.
+ <tschwinge> sekon_: For example, [hurd]/libdiskfs/dir-lookup.c.
+ <tschwinge> sekon_: What MIG does is provide a function call interface for
+ these ``functions'', and the Mach microkernel then dispatches the
+ invocation to the corresponding server, for example a /hurd/ext2fs file
+ system (via libdiskfs).
+ <tschwinge> sekon_: This may help a bit:
+ http://www.gnu.org/software/hurd/hurd/hurd_hacking_guide.html
diff --git a/open_issues/ext2fs_page_cache_swapping_leak.mdwn b/open_issues/ext2fs_page_cache_swapping_leak.mdwn
index c0d0867b..075533e7 100644
--- a/open_issues/ext2fs_page_cache_swapping_leak.mdwn
+++ b/open_issues/ext2fs_page_cache_swapping_leak.mdwn
@@ -12,7 +12,10 @@ License|/fdl]]."]]"""]]
There is a [[!FF_project 272]][[!tag bounty]] on this task.
-IRC, OFTC, #debian-hurd, 2011-03-24
+[[!toc]]
+
+
+# IRC, OFTC, #debian-hurd, 2011-03-24
<youpi> I still believe we have an ext2fs page cache swapping leak, however
<youpi> as the 1.8GiB swap was full, yet the ld process was only 1.5GiB big
@@ -24,7 +27,7 @@ IRC, OFTC, #debian-hurd, 2011-03-24
<youpi> yes
<youpi> the disk content, basicallyt :)
-IRC, freenode, #hurd, 2011-04-18
+# IRC, freenode, #hurd, 2011-04-18
<antrik> damn, a cp -a simply gobbles down swap space...
<braunr> really ?
@@ -173,3 +176,84 @@ IRC, freenode, #hurd, 2011-04-18
backing store of memory objects created from its pager
<braunr> so you can view swap as the file system for everything that isn't
an external memory object
+
+
+# IRC, freenode, #hurd, 2011-11-15
+
+ <braunr> hm, now my system got unstable
+ <braunr> swap is increasing, without any apparent reason
+ <antrik> you mean without any load?
+ <braunr> with load, yes
+ <braunr> :)
+ <antrik> well, with load is "normal"...
+ <antrik> at least for some loads
+ <braunr> i can't create memory pressure to stress reclaiming without any
+ load
+ <antrik> what load are you using?
+ <braunr> ftp mirrorring
+ <antrik> hm... never tried that; but I guess it's similar to apt-get
+ <antrik> so yes, that's "normal". I talked about it several times, and also
+ wrote to the ML
+ <braunr> antrik: ok
+ <antrik> if you find out how to fix this, you are my hero ;-)
+ <braunr> arg :)
+ <antrik> I suspect it's the infamous double swapping problem; but that's
+ just a guess
+ <braunr> looks like this
+ <antrik> BTW, if you give me the exact command, I could check if I see it
+ too
+ <braunr> i use lftp (mirror -Re) from a linux git repository
+ <braunr> through sftp
+ <braunr> (lots of small files, big content)
+ <antrik> can't you just give me the exact command? I don't feel like
+ figuring it out myself
+ <braunr> antrik: cd linux-stable; lftp sftp://hurd_addr/
+ <braunr> inside lftp: mkdir linux-stable; cd linux-stable; mirror -Re
+ <braunr> hm, half of physical memory just got freed
+ <braunr> our page cache is really weird :/
+ <braunr> (i didn't delete any file when that happened)
+ <antrik> hurd_addr?
+ <braunr> ssh server ip address
+ <braunr> or name
+ <braunr> of your hurd :)
+ <antrik> I'm confused. you are mirroring *from* the Hurd box?
+ <braunr> no, to it
+ <antrik> ah, so you login via sftp and then push to it?
+ <braunr> yes
+ <braunr> fragmentation looks very fine
+ <braunr> even for the huge pv_entry cache and its 60k+ entries
+ <braunr> (and i'm running a kernel with the cpu layer enabled)
+ <braunr> git reset/status/diff/log/grep all work correctly
+ <braunr> anyway, mcsim's branch looks quite stable to me
+ <antrik> braunr: I can't reproduce the swap leak with ftp. free memory
+ idles around 6.5 k (seems to be the threshold where paging starts), and
+ swap use is constant
+ <antrik> might be because everything swappable is already present in swap
+ from previous load I guess...
+ <antrik> err... scratch that. was connected to the wrong host, silly me
+ <antrik> indeed swap gets eaten away, as expected
+ <antrik> but only if free memory actually falls below the
+ threshold. otherwise it just oscillates around a constant value, and
+ never touches swap
+ <antrik> so this seems to confirm the double swapping theory
+ <youpi> antrik: is that "double swap" theory written somewhere?
+ <youpi> (no, a quick google didn't tell me)
+
+
+## IRC, freenode, #hurd, 2011-11-16
+
+ <antrik> youpi:
+ http://lists.gnu.org/archive/html/l4-hurd/2002-06/msg00001.html talks
+ about "double paging". probably it's also the term others used for it;
+ however, the term is generally used in a completely different meaning, so
+ I guess it's not really suitable for googling either ;-)
+ <antrik> IIRC slpz (or perhaps someone else?) proposed a solution to this,
+ but I don't remember any details
+ <youpi> ok so it's the same thing I was thinking about with swap getting
+ filled
+ <youpi> my question was: is there something to release the double swap,
+ once the ext2fs pager managed to recover?
+ <antrik> apparently not
+ <antrik> the only way to free the memory seems to be terminating the FS
+ server
+ <youpi> uh :/
diff --git a/open_issues/gnumach_memory_management.mdwn b/open_issues/gnumach_memory_management.mdwn
index 9a4418c1..c9c3e64f 100644
--- a/open_issues/gnumach_memory_management.mdwn
+++ b/open_issues/gnumach_memory_management.mdwn
@@ -1810,3 +1810,205 @@ There is a [[!FF_project 266]][[!tag bounty]] on this task.
<braunr> etenil: but mcsim's work is, for one, useful because the allocator
code is much clearer, adds some debugging support, and is smp-ready
+
+
+# IRC, freenode, #hurd, 2011-11-14
+
+ <braunr> i've just realized that replacing the zone allocator removes most
+ (if not all) static limit on allocated objects
+ <braunr> as we have nothing similar to rlimits, this means kernel resources
+ are actually exhaustible
+ <braunr> and i'm not sure every allocation is cleanly handled in case of
+ memory shortage
+ <braunr> youpi: antrik: tschwinge: is this acceptable anyway ?
+ <braunr> (although IMO, it's also a good thing to get rid of those limits
+ that made the kernel panic for no valid reason)
+ <youpi> there are actually not many static limits on allocated objects
+ <youpi> only a few have one
+ <braunr> those defined in kern/mach_param.h
+ <youpi> most of them are not actually enforced
+ <braunr> ah ?
+ <braunr> they are used at zinit() time
+ <braunr> i thought they were
+ <youpi> yes, but most zones are actually fine with overcoming the max
+ <braunr> ok
+ <youpi> see zone->max_size += (zone->max_size >> 1);
+ <youpi> you need both !EXHAUSTIBLE and FIXED
+ <braunr> ok
+ <pinotree> making having rlimits enforced would be nice...
+ <pinotree> s/making//
+ <braunr> pinotree: the kernel wouldn't handle many standard rlimits anyway
+
+ <braunr> i've just committed my final patch on mcsim's branch, which will
+ serve as the starting point for integration
+ <braunr> which means code in this branch won't change (or only last minute
+ changes)
+ <braunr> you're invited to test it
+ <braunr> there shouldn't be any noticeable difference with the master
+ branch
+ <braunr> a bit less fragmentation
+ <braunr> more memory can be reclaimed by the VM system
+ <braunr> there are debugging features
+ <braunr> it's SMP ready
+ <braunr> and overall cleaner than the zone allocator
+ <braunr> although a bit slower on the free path (because of what's
+ performed to reduce fragmentation)
+ <braunr> but even "slower" here is completely negligible
+
+
+# IRC, freenode, #hurd, 2011-11-15
+
+ <mcsim> I enabled cpu_pool layer and kentry cache exhausted at "apt-get
+ source gnumach && (cd gnumach-* && dpkg-buildpackage)"
+ <mcsim> I mean kernel with your last commit
+ <mcsim> braunr: I'll make patch how I've done it in a few minutes, ok? It
+ will be more specific.
+ <braunr> mcsim: did you just remove the #if NCPUS > 1 directives ?
+ <mcsim> no. I replaced macro NCPUS > 1 with SLAB_LAYER, which equals NCPUS
+ > 1, than I redefined macro SLAB_LAYER
+ <braunr> ah, you want to make the layer optional, even on UP machines
+ <braunr> mcsim: can you give me the commands you used to trigger the
+ problem ?
+ <mcsim> apt-get source gnumach && (cd gnumach-* && dpkg-buildpackage)
+ <braunr> mcsim: how much ram & swap ?
+ <braunr> let's see if it can handle a quite large aptitude upgrade
+ <mcsim> how can I check swap size?
+ <braunr> free
+ <braunr> cat /proc/meminfo
+ <braunr> top
+ <braunr> whatever
+ <mcsim> total used free shared buffers
+ cached
+ <mcsim> Mem: 786368 332296 454072 0 0
+ 0
+ <mcsim> -/+ buffers/cache: 332296 454072
+ <mcsim> Swap: 1533948 0 1533948
+ <braunr> ok, i got the problem too
+ <mcsim> braunr: do you run hurd in qemu?
+ <braunr> yes
+ <braunr> i guess the cpu layer increases fragmentation a bit
+ <braunr> which means more map entries are needed
+ <braunr> hm, something's not right
+ <braunr> there are only 26 kernel map entries when i get the panic
+ <braunr> i wonder why the cache gets that stressed
+ <braunr> hm, reproducing the kentry exhaustion problem takes quite some
+ time
+ <mcsim> braunr: what do you mean?
+ <braunr> sometimes, dpkg-buildpackage finishes without triggering the
+ problem
+ <mcsim> the problem is in apt-get source gnumach
+ <braunr> i guess the problem happens because of drains/fills, which
+ allocate/free much more object than actually preallocated at boot time
+ <braunr> ah ?
+ <braunr> ok
+ <braunr> i've never had it at that point, only later
+ <braunr> i'm unable to trigger it currently, eh
+ <mcsim> do you use *-dbg kernel?
+ <braunr> yes
+ <braunr> well, i use the compiled kernel, with the slab allocator, built
+ with the in kernel debugger
+ <mcsim> when you run apt-get source gnumach, you run it in clean directory?
+ Or there are already present downloaded archives?
+ <braunr> completely empty
+ <braunr> ah just got it
+ <braunr> ok the limit is reached, as expected
+ <braunr> i'll just bump it
+ <braunr> the cpu layer drains/fills allocate several objects at once (64 if
+ the size is small enough)
+ <braunr> the limit of 256 (actually 252 since the slab descriptor is
+ embedded in its slab) is then easily reached
+ <antrik> mcsim: most direct way to check swap usage is vmstat
+ <braunr> damn, i can't live without slabtop and the amount of
+ active/inactive cache memory any more
+ <braunr> hm, weird, we have active/inactive memory in procfs, but not
+ buffers/cached memory
+ <braunr> we could set buffers to 0 and everything as cached memory, since
+ we're currently unable to communicate the purpose of cached memory
+ (whether it's used by disk servers or file system servers)
+ <braunr> mcsim: looks like there are about 240 kernel map entries (i forgot
+ about the ones used in kernel submaps)
+ <braunr> so yes, addin the cpu layer is what makes the kernel reach the
+ limit more easily
+ <mcsim> braunr: so just increasing limit will solve the problem?
+ <braunr> mcsim: yes
+ <braunr> slab reclaiming looks very stable
+ <braunr> and unfrequent
+ <braunr> (which is surprising)
+ <pinotree> braunr: "unfrequent"?
+ <braunr> pinotree: there isn't much memory pressure
+ <braunr> slab_collect() gets called once a minute on my hurd
+ <braunr> or is it infrequent ?
+ <braunr> :)
+ <pinotree> i have no idea :)
+ <braunr> infrequent, yes
+
+
+# IRC, freenode, #hurd, 2011-11-16
+
+ <braunr> for those who want to play with the slab branch of gnumach, the
+ slabinfo tool is available at http://git.sceen.net/rbraun/slabinfo.git/
+ <braunr> for those merely interested in numbers, here is the output of
+ slabinfo, for a hurd running in kvm with 512 MiB of RAM, an unused swap,
+ and a short usage history (gnumach debian packages built, aptitude
+ upgrade for a dozen of packages, a few git commands)
+ <braunr> http://www.sceen.net/~rbraun/slabinfo.out
+ <antrik> braunr: numbers for a long usage history would be much more
+ interesting :-)
+
+
+## IRC, freenode, #hurd, 2011-11-17
+
+ <braunr> antrik: they'll come :)
+ <etenil> is something going on on darnassus? it's mighty slow
+ <braunr> yes
+ <braunr> i've rebooted it to run a modified kernel (with the slab
+ allocator) and i'm building stuff on it to stress it
+ <braunr> (i don't have any other available machine with that amount of
+ available physical memory)
+ <etenil> ok
+ <antrik> braunr: probably would be actually more interesting to test under
+ memory pressure...
+ <antrik> guess that doesn't make much of a difference for the kernel object
+ allocator though
+ <braunr> antrik: if ram is larger, there can be more objects stored in
+ kernel space, then, by building something large such as eglibc, memory
+ pressure is created, causing caches to be reaped
+ <braunr> our page cache is useless because of vm_object_cached_max
+ <braunr> it's a stupid arbitrary limit masking the inability of the vm to
+ handle pressure correctly
+ <braunr> if removing it, the kernel freezes soon after ram is filled
+ <braunr> antrik: it may help trigger the "double swap" issue you mentioned
+ <antrik> what may help trigger it?
+ <braunr> not checking this limit
+ <antrik> hm... indeed I wonder whether the freezes I see might have the
+ same cause
+
+
+## IRC, freenode, #hurd, 2011-11-19
+
+ <braunr> http://www.sceen.net/~rbraun/slabinfo.out <= state of the slab
+ allocator after building the debian libc packages and removing all files
+ once done
+ <braunr> it's mostly the same as on any other machine, because of the
+ various arbitrary limits in mach (most importantly, the max number of
+ objects in the page cache)
+ <braunr> fragmentation is still quite low
+ <antrik> braunr: actually fragmentation seems to be lower than on the other
+ run...
+ <braunr> antrik: what makes you think that ?
+ <antrik> the numbers of currently unused objects seem to be in a similar
+ range IIRC, but more of them are reclaimable I think
+ <antrik> maybe I'm misremembering the other numbers
+ <braunr> there had been more reclaims on the other run
+
+
+# IRC, freenode, #hurd, 2011-11-25
+
+ <braunr> mcsim: i've just updated the slab branch, please review my last
+ commit when you have time
+ <mcsim> braunr: Do you mean compilation/tests?
+ <braunr> no, just a quick glance at the code, see if it matches what you
+ intended with your original patch
+ <mcsim> braunr: everything is ok
+ <braunr> good
+ <braunr> i think the branch is ready for integration
diff --git a/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn b/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn
index 93055b77..80fc9fcd 100644
--- a/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn
+++ b/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn
@@ -54,3 +54,53 @@ License|/fdl]]."]]"""]]
compatibility with eventual 3rd party users is not broken
<pinotree> but those using them, other than hurd itself, won't compile
anymore, so you fix them progressively
+
+
+# IRC, freenode, #hurd, 2011-11-16
+
+ <braunr> is the mach_debug interface packaged in debian ?
+ <antrik> what mach_debug interface?
+ <braunr> include/include/mach_debug/mach_debug.defs in gnumach
+ <braunr> include/mach_debug/mach_debug.defs in gnumach
+ <antrik> what exactly is supposed to be packaged there?
+ <braunr> i'm talking about the host_*_info client code
+ <antrik> braunr: you mean MIG-generated stubs?
+ <braunr> antrik: yes
+ <braunr> i wrote a tiny slabinfo tool, and rpctrace doesn't show the
+ host_slab_info call
+ <braunr> do you happen to know why ?
+ <antrik> braunr: doesn't show it at all, or just doesn't translate?
+ <braunr> antrik: doesn't at all, the msgids file contains what's needed to
+ translate
+ <braunr> btw, i was able to build the libc0.3 packages with a kernel using
+ the slab allocator today, while monitoring it with the aforementioned
+ slabinfo tool, everything went smoothly
+ <antrik> great :-)
+ <braunr> i'll probably add a /proc/slabinfo entry some day
+ <braunr> and considering the current state of our beloved kernel, i'm
+ wondering why host_*_info rpcs are considered debugging calls
+ <braunr> imo, they should always be included by default
+ <braunr> and part of the standard mach interface
+ <braunr> (if the rpc is missing, an error is simply returned)
+ <antrik> I guess that's been inherited from original Mach
+ <antrik> so you think the stubs should be provided by libmachuser?
+ <braunr> i'm not sure
+ <antrik> actually, it's a bit arguable. if interfaces are not needed by
+ libc itself, it isn't really necessary to build them as part of the libc
+ build...
+ <braunr> i don't know the complete list of potential places for such calls
+ <antrik> OTOH, as any updates will happen in sync with other Mach updates,
+ it makes sense to keep them in one place, to reduce transition pain
+ <braunr> and i didn't want to imply they should be part of libc
+ <braunr> on the contrary, libmachuser seems right
+ <antrik> libmachuser is part of libc
+ <braunr> ah
+ <braunr> :)
+ <braunr> why so ?
+ <antrik> well, for one, libc needs the Mach (and Hurd) stubs itself
+ <antrik> also, it's traditionally the role of libc to provide the call
+ wrappers for syscalls... so it makes some sense
+ <braunr> sure, but why doesn't it depend on an external libmachuser instead
+ of embedding it ?
+ <braunr> right
+ <antrik> now that's a good question... no idea TBH :-)
diff --git a/open_issues/mig_portable_rpc_declarations.mdwn b/open_issues/mig_portable_rpc_declarations.mdwn
new file mode 100644
index 00000000..084d7454
--- /dev/null
+++ b/open_issues/mig_portable_rpc_declarations.mdwn
@@ -0,0 +1,58 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_mig]]
+
+
+# IRC, freenode, #hurd, 2011-11-14
+
+ <braunr> also, what's the best way to deal with types such as
+ <braunr> type cache_info_t = struct[23] of integer_t;
+ <braunr> whereas cache_info_t contains longs, which are obviously not
+ integer-wide on 64-bits processors
+ <braunr> ?
+ <youpi> you mean, to port mach to 64bit?
+ <braunr> no, to make the RPC declaration portable
+ <braunr> just in case :)
+ <youpi> refine integer_t into something more precise
+ <youpi> such as size_t, off_t, etc.
+ <braunr> i can't use a single line then
+ <braunr> struct cache_info contains ints, vm_size_t, longs
+ <braunr> should i just use the maximum size it can get ?
+ <braunr> or declare two sizes depending on the word size ?
+ <youpi> well, I'd say three
+ <braunr> youpi: three ?
+ <youpi> the ints, the vm_size_ts, and the longs
+ <braunr> youpi: i don't get it
+ <braunr> youpi: how would i write it in mig language ?
+ <youpi> I don't know the mig language
+ <braunr> me neither :)
+ <youpi> but I'd say don't lie
+ <braunr> i just see struct[23] of smething
+ <braunr> the original zone_info struct includes both integer_t and
+ vm_size_t, and declares it as
+ <braunr> type zone_info_t = struct[9] of integer_t;
+ <braunr> in its mig defs file
+ <braunr> i don't have a good example to reuse
+ <youpi> which is lying
+ <braunr> yes
+ <braunr> which is why i was wondering if mach architects themselves
+ actually solved that problem :)
+ <braunr> "There is no way to specify the fields of a
+ <braunr> C structure to MIG. The size and type-desc are just used to
+ give the size of
+ <braunr> the structure.
+ <braunr> "
+ <braunr> well, this sucks :/
+ <braunr> well, i'll do what the rest of the code seems to do, and let it
+ rot until a viable solution is available
+ <antrik> braunr: we discussed the problem of expressing structs with MIG in
+ the libburn thread
+ <antrik> (which I still need to follow up on... [sigh])
diff --git a/open_issues/mission_statement.mdwn b/open_issues/mission_statement.mdwn
index 212d65e7..d136e3a8 100644
--- a/open_issues/mission_statement.mdwn
+++ b/open_issues/mission_statement.mdwn
@@ -10,7 +10,10 @@ License|/fdl]]."]]"""]]
[[!tag open_issue_documentation]]
-IRC, freenode, #hurd, 2011-10-12:
+[[!toc]]
+
+
+# IRC, freenode, #hurd, 2011-10-12
<ArneBab> we have a mission statement: http://hurd.gnu.org
<Gorodish> yes
@@ -37,3 +40,10 @@ IRC, freenode, #hurd, 2011-10-12:
ceases to amaze me
<Gorodish> I agree that the informational, factual, results oriented
documentation is the primary objective of documenting
+
+
+# IRC, freenode, #hurd, 2011-11-25
+
+ <antrik> heh, nice: http://telepathy.freedesktop.org/wiki/Rationale
+ <antrik> most of this could be read as a rationale for the Hurd just as
+ well ;-)
diff --git a/open_issues/page_cache.mdwn b/open_issues/page_cache.mdwn
new file mode 100644
index 00000000..062fb8d6
--- /dev/null
+++ b/open_issues/page_cache.mdwn
@@ -0,0 +1,73 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_gnumach]]
+
+IRC, freenode, #hurd, 2011-11-28:
+
+ <braunr> youpi: would you find it reasonable to completely disable the page
+ cache in gnumach ?
+ <braunr> i'm wondering if it wouldn't help make the system more stable
+ under memory pressure
+ <youpi> assuming cache=writeback in gnumach?
+ <youpi> because disabling the page cache will horribly hit performance
+ <braunr> no, it doesn't have anything to do with the host
+ <braunr> i'm not so sure
+ <braunr> while observing the slab allocator, i noticed our page cache is
+ not used that often
+ <youpi> eeh?
+ <youpi> apart from the damn 4000 limitation, I've seen it used
+ <youpi> (and I don't why it wouldn't be used)
+ <youpi> (e.g. for all parts of libc)
+ <youpi> ah, no, libc would be kept open by ext2fs
+ <braunr> taht's precisely because of the 4k limit
+ <youpi> but e.g. .o file emitted during make
+ <braunr> well, no
+ <youpi> well, see the summary I had posted some time ago, the 4k limit
+ makes it completely randomized
+ <youpi> and thus you lose locality
+ <braunr> yes
+ <youpi> but dropping the limit would just fix it
+ <braunr> that's my point
+ <youpi> which I had tried to do, and there were issues, you mentioned why
+ <youpi> and (as usual), I haven't had anyu time to have a look at the issue
+ again
+ <braunr> i'm just trying to figure out the pros and cons for having teh
+ current page cache implementation
+ <braunr> but are you saying you tried with a strict limit of 0 ?
+ <youpi> non, I'm saying I tried with no limit
+ <youpi> but then memory fills up
+ <braunr> yes
+ <youpi> so trying to garbage collect
+ <braunr> i tried that too, the system became unstable very quickly
+ <youpi> but refs don't falldown to 0, you said
+ <braunr> did i ?
+ <youpi> or maybe somebody else
+ <youpi> see the list archives
+ <braunr> that's possible
+ <braunr> i'd imagine someone like sergio lopez
+ <youpi> possibly
+ <youpi> somebody that knows memory stuff way better than me in any case
+ <braunr> youpi: i'm just wondering how much we'd loose by disabling the
+ page cache, and if we actually gain more stability (and ofc, if it's
+ worth it)
+ <youpi> no idea, measures will tell
+ <youpi> fixing the page cache shouldn't be too hard I believe, however
+ <youpi> you just need to know what you are doing, which I don't
+ <youpi> I do believe the cache is still at least a bit useful
+ <youpi> even if dumb because of randomness
+ <youpi> e.g. running make lib in the glibc tree gets faster on second time
+ <youpi> because the cache wouldbe filled at least randomly with glibc tree
+ stuff
+ <braunr> yes, i agree on that
+ <youpi> braunr: btw, the current stability is fine for the buildds
+ <youpi> restarting them every few days is ok
+ <youpi> so I'd rather keep the performance :)
+ <braunr> ok
diff --git a/open_issues/perl.mdwn b/open_issues/perl.mdwn
index 45680328..48343e3e 100644
--- a/open_issues/perl.mdwn
+++ b/open_issues/perl.mdwn
@@ -36,6 +36,56 @@ First, make the language functional, have its test suite pass without errors.
[[!inline pages=community/gsoc/project_ideas/perl_python feeds=no]]
+
+## IRC, OFTC, #debian-hurd, 2011-11-08
+
+ <Dom> pinotree: so, with your three fixes applied to 5.14.2, there are
+ still 9 tests failing. They don't seem to be regressions in perl, since
+ they also fail when I build 5.14.0 (even though the buildd managed it).
+ <Dom> What do you suggest as the way forward?
+ <Dom> (incidentally I'm trying on strauss's sid chroot to see how that
+ compares)
+ <pinotree> Dom: samuel makes buildds build perl with nocheck (otherwise
+ we'd have no perl at all)
+ <pinotree> which tests still fail?
+ <Dom> ../cpan/Sys-Syslog/t/syslog.t ../cpan/Time-HiRes/t/HiRes.t
+ ../cpan/autodie/t/recv.t ../dist/IO/t/io_pipe.t ../dist/threads/t/libc.t
+ ../dist/threads/t/stack.t ../ext/Socket/t/socketpair.t io/pipe.t
+ op/sigdispatch.t
+ <Dom> buildds> I see
+ <pinotree> ah ok, those that are failing for me even with my patches
+ <Dom> I hadn't spotted that the builds were done with nocheck.
+ <Dom> (but only sometimes...)
+ <Dom> Explains a lot
+ <pinotree> syslog is kind of non-working on hurd, and syslog.t succeeds in
+ buildds (as opposted to crash the machine...) because there's no /var/log
+ in chroots
+ <pinotree> libc.t appears to succeed too in buildds
+ * Dom notices how little memory strauss has, and cancels the build, now
+ that he *knows* that running out of memory caused the crahses
+ <pinotree> iirc HiRes.t, io_pipe.t , pipe.t and sigdispatch.t fails because
+ of trobules we have with posix signals
+ <pinotree> socketpair.t is kind of curious, it seems to block on
+ socketpair()...
+ * Dom wonders if a wiki page tracking this would be worthwhile
+ <pinotree> stack.t fails because we cannot set a different size for pthread
+ stacks, yet (similar failing test cases are also in the python test
+ suite)
+ <Dom> if there are problems which aren't going to get resolved any time
+ soon, it may be worth a few SKIPs conditional on architecture, depending
+ on how serious the issue is
+ <Dom> then we'd get better visibility of other/new issues
+ <Dom> (problems which aren't bugs in perl, that is)
+ <pinotree> understandable, yes
+ <pinotree> i think nobody digged much deeper in the failing ones yet, to
+ actually classify them as due to glibc/hurd/mach
+ <pinotree> (eg unlike the pipe behaviour in sysconf.t i reported)
+
+### 2011-11-26
+
+ <pinotree> cool, my recvfrom() fix also makes the perl test recv.t pass
+
+
---
diff --git a/open_issues/robustness.mdwn b/open_issues/robustness.mdwn
new file mode 100644
index 00000000..d32bd509
--- /dev/null
+++ b/open_issues/robustness.mdwn
@@ -0,0 +1,64 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_documentation open_issue_hurd]]
+
+
+# IRC, freenode, #hurd, 2011-11-18
+
+ <nocturnal> I'm learning about GNU Hurd and was speculating with a friend
+ who is also a computer enthusiast. I would like to know if Hurds
+ microkernel can recover services should they crash? and if it can, does
+ that recovery code exist in multiple services or just one core kernel
+ service?
+ <braunr> nocturnal: you should read about passive translators
+ <braunr> basically, there is no dedicated service to restore crashed
+ servers
+ <etenil> Hi everyone!
+ <braunr> services can crash and be restarted, but persistence support is
+ limited, and rather per serivce
+ <braunr> actually persistence is more a side effect than a designed thing
+ <braunr> etenil: hello
+ <etenil> braunr: translators can also be spawned on an ad-hoc basis, for
+ instance when accessing a particular file, no?
+ <braunr> that's what being passive, for a translator, means
+ <etenil> ah yeah I thought so :)
+
+
+# IRC, freenode, #hurd, 2011-11-19
+
+ <chromaticwt> will hurd ever have the equivalent of a rs server?, is that
+ even possible with hurd?
+ <youpi> chromaticwt: what is an rs server ?
+ <chromaticwt> a reincarnation server
+ <youpi> ah, like minix. Well, the main ground issue is restoring existing
+ information, such as pids of processes, etc.
+ <youpi> I don't know how minix manages it
+ <antrik> chromaticwt: I have a vision of a session manager that could also
+ take care of reincarnation... but then, knowing myself, I'll probably
+ never imlement it
+ <youpi> we do get proc crashes from times to times
+ <youpi> it'd be cool to see the system heal itself :)
+ <braunr> i need a better description of reincarnation
+ <braunr> i didn't think it would make core servers like proc able to get
+ resurrected in a safe way
+ <antrik> depends on how it is implemented
+ <antrik> I don't know much about Minix, but I suspect they can recover most
+ core servers
+ <antrik> essentially, the condition is to make all precious state be
+ constantly serialised, and held by some third party, so the reincarnated
+ server could restore it
+ <braunr> should it work across reboots ?
+ <antrik> I haven't thought about the details of implementing it for each
+ core server; but proc should be doable I guess... it's not necessary for
+ the system to operate, just for various UNIX mechanisms
+ <antrik> well, I'm not aware of the Minix implementation working across
+ reboots. the one I have in mind based on a generic session management
+ infrastructure should though :-)
diff --git a/open_issues/syslog.mdwn b/open_issues/syslog.mdwn
index 5fec38b1..2e902698 100644
--- a/open_issues/syslog.mdwn
+++ b/open_issues/syslog.mdwn
@@ -43,3 +43,30 @@ IRC, freenode, #hurd, 2011-08-08
< youpi> shm should work with the latest libc
< youpi> what won't is sysv sem
< youpi> (i.e. semget)
+
+
+IRC, OFTC, #debian-hurd, 2011-11-02:
+
+ * pinotree sighs at #645790 :/
+ <tschwinge> pinotree: W.r.t. 645790 -- yeah, ``someone'' should finally
+ figure out what's going on with syslog.
+ http://lists.gnu.org/archive/html/bug-hurd/2008-07/msg00152.html
+ <tschwinge> pinotree: And this...
+ http://lists.gnu.org/archive/html/bug-hurd/2007-02/msg00042.html
+ <pinotree> tschwinge: i did that 20 invocations tests recently, and
+ basically none of them has been logged
+ <pinotree> tschwinge: when i started playing with logger more, as result i
+ had some server that started taking all the cpu, followed by other
+ servers and in the end my ssh connection were dropped and i had nothing
+ to do (not even login from console)
+ <tschwinge> pinotree: Sounds like ``fun''. Hopefully we can manage to
+ understand (and fix the underlying issue) why a simple syslog()
+ invocation can make the whole system instable.
+ <pinotree> tschwinge: to be honest, i got havoc in the system when i told
+ syslog to manually look for /dev/log (-u /dev/log), possibly alao when
+ telling to use a datagram socket (-d)
+ <pinotree> but even if a normal syslog() invocation does not cause havoc,
+ there's still the "lost messages" issue
+ <tschwinge> Yep. What I've been doing ever since, is deinstall all
+ *syslog* packages.
+ <tschwinge> This ``fixed'' all syslog() hangs.
diff --git a/open_issues/translator_stdout_stderr.mdwn b/open_issues/translator_stdout_stderr.mdwn
index 11793582..14ea1c6d 100644
--- a/open_issues/translator_stdout_stderr.mdwn
+++ b/open_issues/translator_stdout_stderr.mdwn
@@ -11,11 +11,43 @@ License|/fdl]]."]]"""]]
[[!tag open_issue_hurd]]
+There have been several discussions and proposals already, about adding a
+suitable logging mechanism to translators, for example.
+
+
Decide / implement / fix that (all?) running (passive?) translators' output
should show up on the (Mach / Hurd) console / syslog.
+
[[!taglink open_issue_documentation]]: [[!message-id
"87oepj1wql.fsf@becket.becket.net"]]
+
[[!taglink open_issue_documentation]]: Neal once had written an email on this
topic.
+
+
+IRC, freenode, #hurd, 2011-11-06
+
+ <youpi> about CLI_DEBUG, you can use #define CLI_DEBUG(fmt, ...) {
+ fprintf(stderr, fmt, ## __VA_ARGS__); fflush(stderr); }
+ <tschwinge> Isn't stderr in auto-flush mode by default?
+ <tschwinge> man setbuf: The standard error stream stderr is always
+ unbuffered by default.
+ <youpi> tschwinge: "by default" is the important thing here
+ <youpi> in the case of translators iirc stderr is buffered
+ <tschwinge> youpi: Oh!
+ <tschwinge> That sounds wrong.
+
+
+IRC, freenode, #hurd, 2011-11-23
+
+ <braunr> we'd need a special logging task for hurd servers
+ <pinotree> if syslog would work, that could be used optionally
+ <braunr> no, it relies on services provided by the hurd
+ <braunr> i'm thinking of something using merely the mach interface
+ <braunr> e.g. using mach_msg to send log messages to a special port used to
+ reference the logging service
+ <braunr> which would then store the messages in a circular buffer in ram
+ <braunr> maybe sending to syslog if the service is available
+ <braunr> the hurd system buffer if you want
generated by cgit v1.2.3 (git 2.39.1) at 2024-11-26 07:27:50 +0000