From be49aa7ddec52e121d562e14d4d93fd301b05fbb Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 4 Nov 2011 19:19:35 +0100 Subject: IRC. --- glibc.mdwn | 2 + glibc/ioctl.mdwn | 52 +++++ hurd.mdwn | 1 + hurd/libdiskfs.mdwn | 42 ++++ hurd/libstore/examples/ramdisk/discussion.mdwn | 72 ++++++ hurd/running/virtualbox.mdwn | 5 + hurd/translator/tmpfs/notes_various.mdwn | 8 + open_issues/64-bit_port.mdwn | 36 +++ open_issues/anatomy_of_a_hurd_system.mdwn | 14 +- .../benefits_of_a_native_hurd_implementation.mdwn | 2 +- open_issues/bsd_compatibility.mdwn | 34 +++ open_issues/dde.mdwn | 25 ++- open_issues/dde/137784 | 43 ++++ open_issues/dev_zero_size.mdwn | 21 ++ open_issues/device_drivers_and_io_systems.mdwn | 4 +- open_issues/fifo_O_RDWR.mdwn | 25 +++ open_issues/glibc.mdwn | 2 + open_issues/glibc/t/tls-threadvar.mdwn | 24 ++ open_issues/gnumach_memory_management.mdwn | 33 +++ open_issues/gnumach_memory_management_2.mdwn | 246 +++++++++++++++++++++ .../gnumach_memory_management_physical_memory.mdwn | 46 ++++ open_issues/issue_tracking.mdwn | 78 +++++++ open_issues/libpthread_assertion_thread_prevp.mdwn | 52 +++++ .../libpthread_pthread_key_create_reuse.mdwn | 37 +++- open_issues/libpthread_set_stack_size.mdwn | 25 +++ open_issues/mission_statement.mdwn | 39 ++++ .../performance/io_system/binutils_ld_64ksec.mdwn | 4 + open_issues/pfinet_vs_system_time_changes.mdwn | 20 +- open_issues/screen_dead_session.mdwn | 30 ++- open_issues/user-space_device_drivers.mdwn | 4 +- open_issues/xen_lseek.mdwn | 35 +++ source_repositories/discussion.mdwn | 193 ++++++++++++++++ 32 files changed, 1239 insertions(+), 15 deletions(-) create mode 100644 glibc/ioctl.mdwn create mode 100644 hurd/libdiskfs.mdwn create mode 100644 hurd/libstore/examples/ramdisk/discussion.mdwn create mode 100644 open_issues/64-bit_port.mdwn create mode 100644 open_issues/bsd_compatibility.mdwn create mode 100644 open_issues/dde/137784 create mode 100644 open_issues/dev_zero_size.mdwn create mode 100644 open_issues/fifo_O_RDWR.mdwn create mode 100644 open_issues/glibc/t/tls-threadvar.mdwn create mode 100644 open_issues/gnumach_memory_management_2.mdwn create mode 100644 open_issues/gnumach_memory_management_physical_memory.mdwn create mode 100644 open_issues/libpthread_assertion_thread_prevp.mdwn create mode 100644 open_issues/libpthread_set_stack_size.mdwn create mode 100644 open_issues/mission_statement.mdwn create mode 100644 open_issues/xen_lseek.mdwn create mode 100644 source_repositories/discussion.mdwn diff --git a/glibc.mdwn b/glibc.mdwn index 8eaff6ad..78d44df0 100644 --- a/glibc.mdwn +++ b/glibc.mdwn @@ -58,6 +58,8 @@ fact simply forwarded to/implemented as [[system_call]]s. * [[signal]] + * [[ioctl]] + ## Individual functions diff --git a/glibc/ioctl.mdwn b/glibc/ioctl.mdwn new file mode 100644 index 00000000..e1a86f32 --- /dev/null +++ b/glibc/ioctl.mdwn @@ -0,0 +1,52 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_documentation]] + +IRC, freenode, #hurd, 2011-10-31: + + is there some example of translator replying to custom ioctl's? + let's say you define some ioctl (those which can be represented) + using the _IOW etc macros; how would a translator (or something else) + "register" and reply to them? + it's not an easy thing + see hurd/hurd/tioctl.defs for instance + that's where the 't' ioctls end up + ('t' being the group in the _IOW macro) + it's not that hard either + youpi: so you "roll" the ioctl to an ipc call with proper + parameters? + yes + ah ok, i thought there was some way to hook new ioctl's, and + have libc send the whole stuff at once + and the proper number (with a clear name) + hm + for many ioctls, you don't have to change libc + yes, there's a script which produces the .defs from _IOW calls, + iirc + or something like this + there's also a hook thing in glibc, but for "sane" ioctls, that's + not needed + (_hurd_lookup_ioctl_handler called by ioctl()) + yes, see the rules in hurd/hurd/Makefile + "The following rules assist in creating an `Xioctl.defs' file to + define RPCs that are sent primarily by ioctl commands." + well, you can have perfectly sane ioctl()s that still can't be + expressed within the constraints of the IO* macros... but admittedly + that's rather uncommon + (unless you consider passing of structs generally insane...) + I didn't want to spend time on finding an appropriate adjective + instaed of "sane" + while I knew he would understand what I meant (and you did) + (though maybe not actually) + by "sane", I mean, which use _IOW properly + i.e. with a group, proper numbers, etc. + (the imposed contraints on the parameters is obviously a flaw in + the hurdish ioctl design, and not insanity from structures) diff --git a/hurd.mdwn b/hurd.mdwn index 7bf98388..255cd3d7 100644 --- a/hurd.mdwn +++ b/hurd.mdwn @@ -92,6 +92,7 @@ in the *unstable* branch of the Debian archive. * [[libchannel]] * [[libtrivfs]] * [[libnetfs]] -- short introductory material + * [[libdiskfs]] * [[libihash]] * [[libpthread]] * [[IO_Path]] diff --git a/hurd/libdiskfs.mdwn b/hurd/libdiskfs.mdwn new file mode 100644 index 00000000..dd499785 --- /dev/null +++ b/hurd/libdiskfs.mdwn @@ -0,0 +1,42 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + + +# Paging + +In the course of Maksym's [[translator/tmpfs]] work: + +IRC, freenode, #hurd, 2011-10-24: + + I've compared the way pagers are handled in ext2fs and found out + that for every file new pager is created when occurs reading or writing + to this file. Is it necessary? And can one pager handle several memory + objects? + mcsim: yes, this in necessary. one pager port corresponds to one + memory object + mcsim: note that a pager, from the kernel's point of view, is + essentially just the port used to communicated with the process + responsible for paging the object. how your process manages multiple + pager ports is up to you + so, how can I attach those pager_* functions, which are declared + now in pager-stubs.c to new pager? + or is it done automatically with all pagers, which I create, If + only I'm not using default one? + I'm not sure how libpager works; but I suspect it's based on + libports. you probably need a port class for the pager ports, and add the + port for each new pager your create to that class + (of course you also need to add it to some port bucket. if you use + a single dispatcher for everything, this would be the default bucket; if + you want a separate thread for pager handling, you'd have to create an + extra bucket for the pagers) + +This is the `diskfs_get_filemap` function that a `libdiskfs` client has to +provide; used in `libdiskfs/rdwr-internal.c:_diskfs_rdwr_internal`, which in +turn is used by the [[interface/io_read]]/[[interface/io_write]] RPCs. diff --git a/hurd/libstore/examples/ramdisk/discussion.mdwn b/hurd/libstore/examples/ramdisk/discussion.mdwn new file mode 100644 index 00000000..d73bf903 --- /dev/null +++ b/hurd/libstore/examples/ramdisk/discussion.mdwn @@ -0,0 +1,72 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + + +# IRC, freenode, #hurd, 2011-10-15 + + youpi: I'm not at all talking about ordinary tmpfs. I'm talking + about the proposed variant using a separate backing store + youpi: and as you might remember, I once came up with a crazy + passive translator command line (based on another crazy passive + translator command line from tschwinge) that can automatically do the + mkfs + so there is really very little benefit in using something else + than ext2fs when not paging to the swap partition + real tmpfs IMHO is mostly useful precisely because it uses the + ordinary swap, and doesn't have an explicit size limit... + well, it is still quite a waste to bounce data betwen page cache + and memory storage + or is ext2fs able to map the store data directly? + then there's only the medata bounce which is spurious + and still, even a one-liner settrans doesn't fit with the "is just + an fs alternative for the existing tmpfs-mounting scripts" + youpi: well, if the invocation is the major concern, it would be + trivial to write a tiny wrapper binary or script that acts like a + "normal" FS... + antrik: could you write it then? + you mean a shell script that uses ext2fs on a memory store to act + like a "proper" tmpfs? + I mean whatever that permits to just run mount none /tmp -t tmpfs + and just works already nowadays + which we could e.g. ship instead of our currently-completely bugged + tmpfs + I suspect the mount script just looks for /hurd/tmpfs in this + case? if so, that should indeed be pretty trivial. let's see if I can dig + up my crazy command line -- turning that into a "proper" script should be + quite easy I hope... + hm... I digged up + http://lists.gnu.org/archive/html/bug-hurd/2007-04/msg00013.html ; but I + wonder how much of it is really necessary for a generic pseudo-tmpfs... + the major complication seems to be the chmod, which I guess we + don't need for most use cases... + I actually don't see why it's inlined there + doesn't the caller do it if it needs it? + ah, well, here there is no caller, it's just a passive entry + is it a problem that this solution needs an extra node for the + store? + yes + because you need to say where it resides + and there's no safe place + since such safe place would typically be a mounted tmpfs + I feared that much... + I suspect we could work around this by not attaching the store to + any node; but this a) doesn't work in a shell script, and b) is much more + involved... + hm... can we assume /dev/fd to be present? I have a vague crazy + idea... + yes + I consider hacking settrans so it grows an option which allows + passing the port to the translator as an FD, instead of attaching it to + any node... this way, we could work with anonymous translators in shell + scripts :-) + (of course that's not less work than just doing the wrapper in + C... but it could be useful in other cases) diff --git a/hurd/running/virtualbox.mdwn b/hurd/running/virtualbox.mdwn index d9be09f2..f57fcbc3 100644 --- a/hurd/running/virtualbox.mdwn +++ b/hurd/running/virtualbox.mdwn @@ -39,3 +39,8 @@ To convert the image you need the VirtualBox package properly installed with a V If [[QEMU with KVM|qemu]] is not available, VirtualBox reportedly has better performance. + +IRC, freenode, #hurd, 2011-10-31: + + I don't know what virtualbox does with hardware emulation, but + gnumach is awfully slow to probe things there diff --git a/hurd/translator/tmpfs/notes_various.mdwn b/hurd/translator/tmpfs/notes_various.mdwn index d16210ca..d1c5cf62 100644 --- a/hurd/translator/tmpfs/notes_various.mdwn +++ b/hurd/translator/tmpfs/notes_various.mdwn @@ -212,3 +212,11 @@ License|/fdl]]."]]"""]] seems something more broke in the meantime :-( ah, right... but I the main problem was some other change (or maybe it never really worked, not sure anymore) + +--- + +IRC, freenode, #hurd, 2011-10-11: + + There is no patch for "tmpfs crashes on filling an empty file". For + second bug there is Zheng Da's patch, but it wasn't applied (at least I + didn't found). diff --git a/open_issues/64-bit_port.mdwn b/open_issues/64-bit_port.mdwn new file mode 100644 index 00000000..797d540f --- /dev/null +++ b/open_issues/64-bit_port.mdwn @@ -0,0 +1,36 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_gnumach open_issue_mig]] + +IRC, freenode, #hurd, 2011-10-16: + + it'd be really good to have a 64bit kernel, no need to care about + addressing space :) + yes a 64 bits kernel would be nice + i guess it wouldn't be too hard to have a special mach kernel for + 64 bits processors, but 32 bits userland only + well, it means tinkering with mig + like old sparc systems :p + to build the 32bit interface, not the 64bit one + ah yes + hm + i'm not sure + mig would assume a 32 bits kernel, like now + and you'll have all kinds of discrepancies in vm_size_t & such + yes + the 64 bits type should be completely internal + types* + but it would be far less work than changing all the userspace bits + for 64 bit (ofc we'll do that some day but in the meanwhile ..) + yes + and it'd boost userland addrespace to 4GiB + yes + leaving time for a 64bit userland :) diff --git a/open_issues/anatomy_of_a_hurd_system.mdwn b/open_issues/anatomy_of_a_hurd_system.mdwn index 64af3d00..46526641 100644 --- a/open_issues/anatomy_of_a_hurd_system.mdwn +++ b/open_issues/anatomy_of_a_hurd_system.mdwn @@ -13,7 +13,7 @@ License|/fdl]]."]]"""]] A bunch of this should also be covered in other (introductionary) material, like Bushnell's Hurd paper. All this should be unfied and streamlined. -IRC, freenode, #hurd, 2011-03-08 +IRC, freenode, #hurd, 2011-03-08: I've a question on what are the "units" in the hurd project, if you were to divide them into units if they aren't, and what are the @@ -40,7 +40,7 @@ IRC, freenode, #hurd, 2011-03-08 --- -IRC, freenode, #hurd, 2011-03-12 +IRC, freenode, #hurd, 2011-03-12: when mach first starts up, does it have some basic i/o or fs functionality built into it to start up the initial hurd translators? @@ -86,3 +86,13 @@ RPC stubs. --- More stuff like [[hurd/IO_path]]. + +-- + +IRC, freenode, #hurd, 2011-10-18: + + what happens @ boot. and which translators are started in what + order? + short version: grub loads mach, ext2, and ld.so/exec; mach starts + ext2; ext2 starts exec; ext2 execs a few other servers; ext2 execs + init. from there on, it's just standard UNIX stuff diff --git a/open_issues/benefits_of_a_native_hurd_implementation.mdwn b/open_issues/benefits_of_a_native_hurd_implementation.mdwn index d796bf6b..afdcfb73 100644 --- a/open_issues/benefits_of_a_native_hurd_implementation.mdwn +++ b/open_issues/benefits_of_a_native_hurd_implementation.mdwn @@ -27,7 +27,7 @@ IRC, #hurd, August / September 2010 runs Linux in user space (just flip the two) for the drivers ArneBab: that is what the L4 people did with the DDE -([[DDE]]) +[[/DDE]]. ArneBab: so, with these different cuts, there are different opportunities. on the one end, you can run Linux as normal and get some diff --git a/open_issues/bsd_compatibility.mdwn b/open_issues/bsd_compatibility.mdwn new file mode 100644 index 00000000..5c916908 --- /dev/null +++ b/open_issues/bsd_compatibility.mdwn @@ -0,0 +1,34 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_documentation]] + +IRC, freenode, #hurd, 2011-10-12: + + hm, why our sys/param.h #define's BSD? + pinotree: because we're evil + is that correct? + (the define, not the evilness) + pinotree: i think it's because the Hurd is closer to the BSD + interfaces, probably because of Mach (which is itself derived from + BSD4.2) + braunr: but mach being bsd-ish won't make the userland (glibc) + interfaces bsd-ish, will it? + pinotree: no + braunr: so...? :) + pinotree: i guesse there are bsdisms in the glibc hurd specific + code, possibly because of mach + or they used to be bsdisms, before being standardized + e.g. mmap + braunr: hrmm... + braunr: the BSDisms are there on purpose... Hurd was originally + even meant to be binary-compatible with BSD + nothing to do with Mach really + antrik: ok diff --git a/open_issues/dde.mdwn b/open_issues/dde.mdwn index ef319a5c..9e2ec742 100644 --- a/open_issues/dde.mdwn +++ b/open_issues/dde.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -8,4 +8,25 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!meta redir=/dde]] +[[General Information|/dde]]. + + +# IRC, freenode, #hurd, 2011-10-18 + +[[!tag open_issue_hurd]] + + [DDE crash, or similar] + it's fake_local_irq_disable_flags(), then raw_local_irq_disable(), + then raw_local_irq_restore(), which *does not* release the cli_lock + the "prove it" comment is (as I expected) completely wrong + npnth: http://paste.debian.net/137784/ + +[[137784]] + + could you try this patch ? + (I've not even tried to build it) + youpi: speaking of which, it still seems to hang :/ I'll 1) double + check it applied correctly and 2) get a gdb output if it did + npnth: could you add printing the value of unlock_refcnt? + so we can check what's happening + unlock_refcnt is at 0, interesting diff --git a/open_issues/dde/137784 b/open_issues/dde/137784 new file mode 100644 index 00000000..1529465b --- /dev/null +++ b/open_issues/dde/137784 @@ -0,0 +1,43 @@ +diff --git a/libdde_linux26/lib/src/arch/l4/cli_sti.c b/libdde_linux26/lib/src/arch/l4/cli_sti.c +index 051f259..6a8c460 100644 +--- a/libdde_linux26/lib/src/arch/l4/cli_sti.c ++++ b/libdde_linux26/lib/src/arch/l4/cli_sti.c +@@ -4,6 +4,8 @@ + + /* IRQ lock reference counter */ + static atomic_t _refcnt = ATOMIC_INIT(0); ++/* Refcnt value at which unlocking the cli_lock (it's not always 0) */ ++static int unlock_refcnt; + static ddekit_lock_t cli_lock; + + /* Check whether IRQs are currently disabled. +@@ -57,9 +59,6 @@ void fake_local_irq_restore(unsigned long flags) + /* Store the current flags state. + * + * This is done by returning the current refcnt. +- * +- * XXX: Up to now, flags was always 0 at this point and +- * I assume that this is always the case. Prove? + */ + unsigned long __raw_local_save_flags(void) + { +@@ -82,7 +81,7 @@ void raw_local_irq_restore(unsigned long flags) + { + Assert(cli_lock != NULL); + atomic_set(&_refcnt, flags); +- if (flags == 0) ++ if (flags == unlock_refcnt) + ddekit_lock_unlock(&cli_lock); + } + +@@ -95,7 +94,9 @@ void raw_local_irq_disable(void) + if (cli_lock == NULL) + ddekit_lock_init_unlocked(&cli_lock); + +- nested_lock(cli_lock); ++ if (nested_lock(cli_lock)) ++ /* Tell the corresponding restorer to release cli_lock */ ++ unlock_refcnt = atomic_read(&_refcnt); + atomic_inc(&_refcnt); + } + diff --git a/open_issues/dev_zero_size.mdwn b/open_issues/dev_zero_size.mdwn new file mode 100644 index 00000000..20c5147e --- /dev/null +++ b/open_issues/dev_zero_size.mdwn @@ -0,0 +1,21 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +IRC, freenode, #hurd, 2011-10-18: + + i guess it is not normal for /dev/zero have a size of + (size_t)-1, no? + (it isn't (size_t)-1, but (long long)-1) + +2011-10-19: + + see the size you get with `stat /dev/zero` diff --git a/open_issues/device_drivers_and_io_systems.mdwn b/open_issues/device_drivers_and_io_systems.mdwn index ce50d93e..5bda0213 100644 --- a/open_issues/device_drivers_and_io_systems.mdwn +++ b/open_issues/device_drivers_and_io_systems.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -68,7 +68,7 @@ Also see [[user-space device drivers]]. # External Projects - * [[DDE]] + * [[/DDE]] * [Building Linux Device Drivers on FreeBSD](http://info.iet.unipi.it/~luigi/FreeBSD/linux_bsd_kld.html) diff --git a/open_issues/fifo_O_RDWR.mdwn b/open_issues/fifo_O_RDWR.mdwn new file mode 100644 index 00000000..730e5c6f --- /dev/null +++ b/open_issues/fifo_O_RDWR.mdwn @@ -0,0 +1,25 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="fifo O_RDWR"]] + +[[!tag open_issue_hurd]] + +[[!debbug 646016]] + +IRC, OFTC, #debian-hurd, 2011-10-19: + + pinotree: Nice! And that open(FIFO, O_RDWR) hanging issue may + warrant investigation on the Hurd side, too -- if the other systems + behave differently, we should probably have convincing reasons for our + behavior. + i gue somebody working on hurd never cared about that case - i + guess it falls back to one of O_RDONLY or O_WRONLY + *guess diff --git a/open_issues/glibc.mdwn b/open_issues/glibc.mdwn index e48dde9c..d3f88673 100644 --- a/open_issues/glibc.mdwn +++ b/open_issues/glibc.mdwn @@ -59,6 +59,8 @@ Last reviewed up to the [[Git mirror's 187da0aedcd9d0a2fb34477bef41549681ba1273 * `TLS_INIT_TP_EXPENSIVE` is unused; Hurd def. can be removed. + * [[t/tls-threadvar]] + * t/verify.h People didn't like this too much. diff --git a/open_issues/glibc/t/tls-threadvar.mdwn b/open_issues/glibc/t/tls-threadvar.mdwn new file mode 100644 index 00000000..f6a81590 --- /dev/null +++ b/open_issues/glibc/t/tls-threadvar.mdwn @@ -0,0 +1,24 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_glibc open_issue_libpthread]] + +IRC, freenode, #hurd, 2011-10-23: + + youpi: If we want to replace threadvars with TLS, there is one + problem: the threadvars interface is publically exported: + /usr/include/hurd/threadvar.h. + youpi: But I am somewhat inclined to say that the only user of + this is libthreads/libpthread. Do you think differently? + tschwinge: that's very probable + so I think we can just drop it + (people should use TLS anyway) + +[[libpthread_set_stack_size]]. diff --git a/open_issues/gnumach_memory_management.mdwn b/open_issues/gnumach_memory_management.mdwn index fb3d6895..9a4418c1 100644 --- a/open_issues/gnumach_memory_management.mdwn +++ b/open_issues/gnumach_memory_management.mdwn @@ -1777,3 +1777,36 @@ There is a [[!FF_project 266]][[!tag bounty]] on this task. an idea for when there's time eh hehe :-) + + +# IRC, freenode, #hurd, 2011-10-13 + + mcsim: what's the current state of your gnumach branch ? + I've merged it with master in September + yes i've seen that, but does it build and run fine ? + I've tested it on gnumach from debian repository, but for building + I had to make additional change in device/ramdisk.c, as I mentioned. + mcsim: why ? + And it runs fine for me. + mcsim: why did you need to make other changes ? + because there is a patch which comes with from-debian-repository + kernel and it addes some code, where I have to make changes. Earlier + kernel_map was a pointer to structure, but I change that and now + kernel_map is structure. So handling to it should be by taking the + address (&kernel_map) + why did you do that ? + or put it another way: what made you do that type change on + kernel_map ? + Earlier memory for kernel_map was allocating with zalloc. But now + salloc can't allocate memory before it's initialisation + that's not a good reason + a simple workaround for your problem is this : + static struct vm_map kernel_map_store; + vm_map_t kernel_map = &kernel_map_store; + braunr: Ok. I'll correct this. + + +# IRC, freenode, #hurd, 2011-11-01 + + etenil: but mcsim's work is, for one, useful because the allocator + code is much clearer, adds some debugging support, and is smp-ready diff --git a/open_issues/gnumach_memory_management_2.mdwn b/open_issues/gnumach_memory_management_2.mdwn new file mode 100644 index 00000000..64aae2a4 --- /dev/null +++ b/open_issues/gnumach_memory_management_2.mdwn @@ -0,0 +1,246 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_gnumach]] + +IRC, freenode, #hurd, 2011-10-16: + + braunr: I realize that kmem_alloc_wired maps the allocated pages in + the kernel map + it's a bit of a waste when my allocation is exactly a page size + is there a proper page allocation which would simply return its + physical address? + pages returned by vm_page_grab may get swapped out, right? + so could it be a matter of calling vm_page_alloc then vm_page_wire + (with lock_queues held) ? + s/alloc/grab/ + vm_page_grab() is only used at boot iirc + youpi: mapping allocated memory in the kernel map is normal, even + if it's only a page + the allocated area usually gets merged with an existing + vm_map_entry + youpi: also, i'm not sure about what you're trying to do here, so + my answers may be out of scope :p + saving addressing space + with that scheme we're using twice as much addressing space for + kernel buffers + kernel or user task ? + kernl + hm are there so many wired areas ? + several MiBs, yes + there are also the zalloc areas + that's pretty normal + which I've recently incrased + hm forget what i've just said about vm_page_grab() + youpi: is there a particular problem caused by kernel memory + exhaustion ? + I currently can't pass the dh_strip stage of iceweasel due to this + it can not allocate a stack + a kernel thread stack ? + yes + that's surprising + it'd be good to have a kernel memory profile + vminfo is able to return the kernel map + well, it's not suprising if the kernel map is full + but it doesn't tell what allocates which p ieces + that's what i mean, it's surprising to have a full kernel map + (that's what profiling is about) + right + well, it's not really surprising, considering that the krenel map + size is arbitrarily chosen + youpi: 2 GiB is really high enough + it's not 2GiB, precisely + there is much of the 2GiB addr space which is spent on physical + memory mapping + then there is the virtual mapping + are you sure we're talking about the kernel map, or e.g. the kmem + map + which is currently only 192MiB + the kmem_map part of kernel_map + ok, the kmem_map submap then + netbsd has used 128 MiB for yeas with almost no issue + mach uses more kernel objects so it's reasonable to have a bigger + map + but that big .. + I've made the zalloc areas quite big + err, not only zalloc area + kernel stacks are allocated directly from the kernel map + kalloc to 64MiB, zalloc to 64MiB + ipc map size to 8MiB + youpi: it could be the lack of kernel map entries + and the device io map to 16MiB + do you have the exact error message ? + no more room for vm_map_find_entry in 71403294 + no more rooom for kmem_alloc_aligned in 71403294 + ah, aligned + for a stack + which is 4 pages only + memory returned by kmem functions always return pages + hum + kmem functions always return memory in page units + and my xen driver is allocating 1MiB memory for the network buffer + 4 pages for kernel stacks ? + through kmem_alloc_wire + that seems a lot + that's needed for xen page updates + without having to split the update in several parts + ok + but are there any alignment requirements ? + I guess mach uses the alignment trick to find "self" + anyway, an alignment on 4pages shouldn't be a problem + i think kmem_alloc_aligned() is the generic function used both for + requests with and without alignment constraints + so I was thinking about at least moving my xen net driver to + vm_grab_page instead of kmem_alloc + and along this, maybe others + but you only get a vm_page, you can't access the memory it + describes + non, a lloc_aligned always aligns + why? + because it's not mapped + there's even vm_grab_page_physical_addr + it is, in the physical memory map + ah, you mean using the direct mapped area + yes + then yes + i don't know that part much + what I'm afraid of is the wiring + why ? + because I don't want to see my page swapped out :) + or whatever might happen if I don't wire it + oh i'm almost sure you won't + why? + why some people need to wire it, and I won't? + because in most mach vm derived code i've seen, you have to + explicitely tell the vm your area is pageable + ah, mach does such thing indeed + wiring can be annoying when you're passing kernel data to external + tasks + you have to make sure the memory isn't wired once passed + but that's rather a security/resource management problem + in the net driver case, it's not passed to anything else + I'm seeing 19MiB kmem_alloc_wired atm + looks ok to me + be aware that the vm_resident code was meant for single page + allocations + what does this mean? + there is no buddy algorithm or anything else decent enough wrt + performance + vm_page_grab_contiguous_pages() can be quite slow + err, ok, but what is the relation with the question at stake ? + you need 4 pages of direct mapped memory for stacks + those pages need to be physically contiguous if you want to avoid + the mapping + allocating physically contiguous pages in mach is slow + :) + I didn't mean I wanted to avoid the mapping for stacks + for anything more than a page, kmem mapping should be fine + I'm concerned with code which allocates only page per page + which thus really doesn't need any mapping + i don't know the mach details but in derived implementations, + there is almost no overhead when allocating single pages + except for the tlb programming + well, there is: twice as much addressing space + well + netbsd doesn't directly map physical memory + and for others, like freebsd + the area is just one vm_map_entry + and on modern mmus, 4 MiBs physical mappings are used in pmap + again, I don't care about tlb & performance + just about the addressing space + hm + you say "twice" + which is short when you're trying to link crazy stuff like + iceweasel & co + yes + ok, the virtual space is doubled + yes + but the resources consume to describe them aren't + even on mach + since you have both the physical mapping and the kmem mapping + I don't care much about the resources + but about addressing space + well there are a limited numbers of solutions + the space it takes and has to be taken from something else, that + is, here physical memory available to Mach + reduce the physical mapping + increase the kmem mapping + or reduce kernel memory consumption + and instead of taking the space from physical mapping, we can as + well avoid doubling the space consumption when it's trivial lto + yes, the hird + +t + that's what I'm asking from the beginning :) + 18:21 < youpi> I don't care much about the resources + actually, you care :) + yes and no + i understand what you mean + not in the sense "it takes a page_t to allocate a page" + you want more virtual space, and aren't that much concerned about + the number of objects used + yes + then it makes sense + but in this case, it could be a good idea to generalize it + have our own kmalloc/vmalloc interfaces + maybe a gsoc project :) + err, don't we have them already? + I mean, what exactly do you want to generalize? + mach only ever had vmalloc + we already have a hell lot of allocators :) + and it's a pain to distribute the available space to them + yes + but what you basically want is to introduce something similar to + kmalloc for single pages + or just patch the few cases that need it into just grabbing a page + there are probably not so many + ok + i've just read vm_page_grab() + it only removes a page from the free list + other functions such as vm_page_alloc insert them afterwards + if a page is in no list, it can't be paged out + so i think it's probably safe to assume it's naturally wired + you don't even need a call to vm_page_wire or a flag of some sort + ok + although considering the low amount of work done by + vm_page_wire(), you could, for clarity + braunr: I was also wondering about the VM_PAGE_FREE_RESERVED & such + constants + they're like 50 pages + is this still reasonable nowadays? + that's a question i'm still asking myself quite often :) + also, the BURST_{MAX,MIN} & such in vm_pageout.c are probably out + of date? + i didn't study the pageout code much + k + but correct handling of low memory thresholds is a good point to + keep in mind + youpi: i often wonder how linux can sometimes have so few free + pages left and still be able to work without any visible failure + well, as long as you have enough pages to be able to make progress, + you're fine + that' the point of the RESERVED pages in mach I guess + youpi: yes but, obviously, hard values are *bad* + linux must adjust it, depending on the number of processors, the + number of pdflush threads, probably other things i don't have in mind + i don't know what should make us adjust that value in mach + which value? + the size of the reserved pool + I don't think it's adjusted + that's what i'm saying + i guess there is an #ifndef line for arch specific definitions + err, you just said linux must adjust it : + )) + there is none + linux adjusts it dynamically + well ok + that's another way to say it + we don't have code to get rid of this macro + but i don't even know how we, as maintainers, are supposed to + guess it diff --git a/open_issues/gnumach_memory_management_physical_memory.mdwn b/open_issues/gnumach_memory_management_physical_memory.mdwn new file mode 100644 index 00000000..58fefe1f --- /dev/null +++ b/open_issues/gnumach_memory_management_physical_memory.mdwn @@ -0,0 +1,46 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_gnumach]] + +IRC, freenode, #hurd, 2011-10: + + antrik: about our physical memory limitations, i told you some + time ago that part of it was due to the linux drivers + and i mentioned the paper concerning the integration of the linux + drivers written at the time + it does indeed tell that mach, which used the common 3G->4G area + for the kernel space had to be adapted + because linux used segmentation so that kernel addresses matched + physical addresses + and it looks like some (many) drivers require that + our current gnumach actually does this (which i found surprising + when i first found it) + and i believe the easy solution to exceed this limitation is to + use a strategy similar to what linux still does on i386 + some highmem support + we could alter the vm_resident module so that, by default, it + still looks for pages in the low 0-800 (or 0-1800 on debian patched + kernels) area + but for everything else than the kernel, e.g. all user processes + we could use a flag or a specialized function that would first + look in the highmem pool for available physical pages to map + the only thing i'm not yet sure of is about user/kernel transfers + if virtual addresses and copies are always cleanly done, then it's + ok + and i really hope our linux drivers do so :) + (i mean ,the glue code ofc) + +2011-10-23: + + braunr: I believe, like Linus, that highmem support is a nightmare + braunr: uhm... the drivers want virtual addressses to match + physical ones? I guess that means switching address spaces before any + driver code is executed?... diff --git a/open_issues/issue_tracking.mdwn b/open_issues/issue_tracking.mdwn index 55c7b87b..72bb3b77 100644 --- a/open_issues/issue_tracking.mdwn +++ b/open_issues/issue_tracking.mdwn @@ -102,6 +102,84 @@ IRC, freenode, #hurd, 2011-08-31: RT is not the easiest thing to set up, but works pretty well once it's running. +IRC, freenode, #hurd, 2011-10-19: + + tschwinge: BTW, what happened to the plan of killing help-hurd? + (and possibly some other lists) + antrik: That plan got stalled, obviously. ;-) + antrik: Now, I had proposed to use hurd-dev for development, + and turn bug-hurd into a debbugs bug reportling list. That proposal has + not heard any supportive/unsupportive votes yet. + hurd-devel. That's the name. + And turn off hurd-devel-readers. And turn off help-hurd. + And web-hurd. + Keep l4-hurd. + yeah, I haven't replied regarding bug-hurd vs. hurd-devel, as I'm + not quite sure myself + on one hand, a dedicated bug list can be convenient; on the other + hand, this kind of splits always causes unnecessary overhead IMHO + also, hurd-devel would obviously be *only* for development, so in + this scenario we actually would *need* to keep something like help-hurd + as well... + I think I'd prefer the non-exclusive mode for debbugs... would + have to check again how it works exactly though :-) + antrik: I quite liked that exclusive mode for it automatically + archives discussions grouped by threads for easy reference. + antrik: And, the very most of bug-hurd emails are ``issues'' of + some sort: bug report, patch (that needs to be tracked until it is + applied, etc. + tschwinge: exclusive mode would just mean that people would take + most of these discussion elsewhere, and the bug list would only be used + when someone explicitly wants something tracked as a bug... + ideally, the bug tracker should only track things if explicitly + CCed. ideally, it should be possible to forward mails that have been + posted without CC, so they can be tracked retroactively... + antrik: Why do you think that people would take discussions + elsewhere? + because most people don't consider it useful to put every random + question or remark in an issue tracker + IMHO it should be easy to turn messages into tickets/followups; + but it should not happen automatically + What if people wouldn't even notice that their issues is kept + in a tracker, too? + It might send a notification of some sort? + I once posted to a list with RT in exclusive mode, and quite + frankly, I considered it rather strange to get a ticket created for my + message :-) + tschwinge: that would only be useful if you always close tickets + for irrelevant or finished discussions, mark duplicates etc. -- and this + would have to happen silently, without noise for most other people + following the list... + tschwinge: are you sure you want to do that?... :-) + Yes. + Because that way we don't lose so much stuff as we currently + do. + well, the decision is up to you in that case... + In fact, probably less than manually archiving the content, as + I'm doing now, partially. + antrik: Well, I'm just out for getting some comments. + it would further reduce our bus factor though :-( + That already is low enough that it doesn't matter anymore... + antrik: So, to sum up, you'd use non-exclusive mode, but are + not actively opposed to exclusive mode as long as it doesn't too much + disturbe any procedures you're currently using? + well, if it happens mostly in the background, I don't see why + anyone should be opposed... + just make sure people posting to the list don't get a "ticket + created" message in response :-) + it would make it harder though for people to explicitly track + issue they are interested in I fear + when using non-exclusive mode, and people explicitly CC things to + the tracker, which sends a notice about a ticket being created, everyone + sees that and can act accordingly. if everything happens in the + background, few people would even think about it... + so non-exclusive mode probably needs more effort to keep in order; + but it would be more useful too... + Well, but with exclusive mode, people don't lose anything + compared to the current state, do they? + tschwinge: probably not compared to the current state... but + possibly compared to a well-used non-exclusive mode :-) + # Further Systems diff --git a/open_issues/libpthread_assertion_thread_prevp.mdwn b/open_issues/libpthread_assertion_thread_prevp.mdwn new file mode 100644 index 00000000..1418bea1 --- /dev/null +++ b/open_issues/libpthread_assertion_thread_prevp.mdwn @@ -0,0 +1,52 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="libpthread: __pthread_enqueue: Assertion `thread->prevp == 0' +failed"]] + +[[!tag open_issue_libpthread]] + +IRC, OFTC, #debian-hurd, 2011-10-21: + + [Python testsuite] + [169/340/1] test_logging + python: + /home/pino/sources/hurd/hurd-20110519/./libpthread/pthread/pt-internal.h:109: + __pthread_enqueue: Assertion `thread->prevp == 0' failed. + sigh + +IRC, freenode, #hurd, 2011-10-21: + + am i missing anything, or in libpthread the __pthread_threads + list does not ever has elements removed from it? + ... thus potentially causing + "./libpthread/pthread/pt-internal.h:109: __pthread_enqueue: Assertion + `thread->prevp == 0' failed." because threads can be appended on + __pthread_dealloc() to the __pthread_free_threads list as well? + maybe reusing the same next+prevp pointers in the __pthread + struct for more than one list at the same time isn't a good idea... + +2011-10-23: + + pinotree: I don't understand the relation between thread->prevp != + 0 and the __pthread_threads list + the list never has elements removed indeed, since libpthread never + frees __pthread structures apparently + youpi: ye sorry, that relation is indeed nonsense + in which condition did you get prevp != 0 + i wa trying to find some explaination for the "thread->prevp == + 0" assertion in the _queue function + ? + *was + it's not obvious to me how libpthread makes sure the various + cond/mutex/rwlock make sure that it's not queued several times + yeah + apparently prevp/next are used for lists of held + waitcond/mutex/rwlock and free threads diff --git a/open_issues/libpthread_pthread_key_create_reuse.mdwn b/open_issues/libpthread_pthread_key_create_reuse.mdwn index a5704d0f..ca2da2f5 100644 --- a/open_issues/libpthread_pthread_key_create_reuse.mdwn +++ b/open_issues/libpthread_pthread_key_create_reuse.mdwn @@ -10,10 +10,10 @@ License|/fdl]]."]]"""]] [[!meta title="libpthread: pthread_key_create, reuse"]] -IRC, FreeNode, #hurd, 2011-07-02 - [[!tag open_issue_libpthread]] +IRC, FreeNode, #hurd, 2011-07-02: + < pinotree> hm, maybe i found a libpthread bug * pinotree tries a testcase < pinotree> yesssss, found the bug :) @@ -47,3 +47,36 @@ IRC, FreeNode, #hurd, 2011-07-02 veeery quick glance Test program: [[pthread_key_create_reuse.c]] + + +2011-11-01: + + youpi: about the bug with pthread keys (reuse): would be an + acceptable solution having a mutex for the thread_specifics of each + thread? + you mean one per thread, one global, or one per key, or ? + what is it supposed to protect? + the thread_specifics of each thread + pinotree: but against what? + the idea would be: when destroying a key, iterate over all the + exiting threads and remove the key data from the thread_specifics of each + thread + one of the issue is getting to browse through the whole list of + threads + the other is concurrency between that, and a thread dying + there's the __pthread_threads_lock rwlock + it should be enough to keep it locked during the iteration + but that wouldn't be enough when one thread is destroying a key, + and another one is doing {get,set}specific() on that key + that's not supposed to happen + mmm + “The effect of calling pthread_getspecific() or + pthread_setspecific() with a key value not obtained from + pthread_key_create() or after key has been deleted with + pthread_key_delete() is undefined.” + undefined -> you are allowed to just blow up + but it's not been deleted yet... :) + it could be, just a matter of time + you're not supposed to rely on time-luckyness :) + mmm + bah, you've convinced me ( :) ) diff --git a/open_issues/libpthread_set_stack_size.mdwn b/open_issues/libpthread_set_stack_size.mdwn new file mode 100644 index 00000000..68f81752 --- /dev/null +++ b/open_issues/libpthread_set_stack_size.mdwn @@ -0,0 +1,25 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_glibc open_issue_libpthread]] + +IRC, freenode, #hurd, 2011-10-21: + + maybe i'm missing something... what's the reason for not + allowing setting a different stack size in libpthread? + +2011-10-23: + + pinotree: the threadvars implementations + which needs to find the variables according to sp value + of course, since we now have TLS, threadvards can go away + it's simply on the so-long TODO list + +[[glibc/t/tls-threadvar]]. diff --git a/open_issues/mission_statement.mdwn b/open_issues/mission_statement.mdwn new file mode 100644 index 00000000..212d65e7 --- /dev/null +++ b/open_issues/mission_statement.mdwn @@ -0,0 +1,39 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_documentation]] + +IRC, freenode, #hurd, 2011-10-12: + + we have a mission statement: http://hurd.gnu.org + yes + but it's quite wishy washy + considering all the elegant capability Hurd potentially has to + offer + Gorodish: it's true that the mission statement is very + abstract... but then, it's hard to put anything more specific into 35 + words + not with some practice + I notice programers tend to speak and write in terms of what + something does + not what it is + the "What is Hurd" is a good example + there's a lot of interesting information there + but the way it's ordered is odd + a mission statement is not primarily a PR instrument; but rather a + guide that allows separating things that benefit the common goal from + things that don't... + I agree that some actual marketing material in addition would be + nice :-) + yes + the modesty of Developers that work on FOSS projects never + ceases to amaze me + I agree that the informational, factual, results oriented + documentation is the primary objective of documenting diff --git a/open_issues/performance/io_system/binutils_ld_64ksec.mdwn b/open_issues/performance/io_system/binutils_ld_64ksec.mdwn index 359d5fee..37fe9c53 100644 --- a/open_issues/performance/io_system/binutils_ld_64ksec.mdwn +++ b/open_issues/performance/io_system/binutils_ld_64ksec.mdwn @@ -27,6 +27,10 @@ extracted from cdf7c161ebd4a934c9e705d33f5247fd52975612 sources, 2010-10-24. On the idle grubber, this one repeatedly takes a few minutes wall time to complete successfully, contrary to a few seconds on a GNU/Linux system. +> On order of slowness may in fact be due to a Xen-specific issue, see +> [[xen_lseek]]. (But there are probably still one or two orders left, even +> without Xen.) + While processing the object files, there is heavy interaction with the relevant [[hurd/translator/ext2fs]] process. Running [[hurd/debugging/rpctrace]] on the testee shows that (primarily) an ever-repeating series of `io_seek` and diff --git a/open_issues/pfinet_vs_system_time_changes.mdwn b/open_issues/pfinet_vs_system_time_changes.mdwn index 714c8784..513cbc73 100644 --- a/open_issues/pfinet_vs_system_time_changes.mdwn +++ b/open_issues/pfinet_vs_system_time_changes.mdwn @@ -17,7 +17,7 @@ IRC, unknown channel, unknown date. This was very likely a misdiagnosis: -IRC, freenode, #hurd, 2011-03-25 +IRC, freenode, #hurd, 2011-03-25: antrik: I suspect it'S some timing stuff in pfinet that perhaps uses absolute time, and somehow wildely gets confused? @@ -40,3 +40,21 @@ IRC, freenode, #hurd, 2011-03-25 (The other way round, you might likely get some integer wrap-around, and thus the same result.) Yes. + +IRC, freenode, #hurd, 2011-10-26: + + anyways, when ntpdate adjusts to the past, the connections hang, + roughly for the amount of time being adjusted + they do not die though + (well, if it's long enough, they probably timeout on the other + side...) + +IRC, freenode, #hurd, 2011-10-27: + + oh, another interesting thing I observed is that the the subhurd + pfinet did *not* drop the connection... only the main Hurd one. I thought + the clock is system-wide?... + It is. + it's really fascinating that only the pfinet on the Hurd instance + where I set the date is affected, and not the pfinet in the other + instance diff --git a/open_issues/screen_dead_session.mdwn b/open_issues/screen_dead_session.mdwn index cdd3f322..fe51523b 100644 --- a/open_issues/screen_dead_session.mdwn +++ b/open_issues/screen_dead_session.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -14,7 +14,10 @@ Comparing to GNU/Linux, on GNU/Hurd it happens much more often and easily for *screen* sessions to become *dead*. This is annoying, as it defeats one of *screen*'s main purposes. -One reproducible scenario goes like this: +[[!toc]] + + +# One reproducible scenario goes like this * `ssh [somewhere]`, @@ -28,7 +31,8 @@ One reproducible scenario goes like this: * at some point, *P* will terminate / hang (after having received some kind of signal?), and the *screen* session will be reported as *dead*. -Another one, not as often reproduced: + +# Another one, not as often reproduced * `ssh [somewhere]`, @@ -43,3 +47,23 @@ Another one, not as often reproduced: (after having received some kind of signal?), and the *screen* session will *immediatelly* be reported as *dead*. (Perhaps the other way round: upon re-attaching, the *screen* session goes bonkers and takes *P* with it?) + + +# IRC, freenode, #hurd, 2011-10-19 + + tschwinge: hm... haven't seen screen dying in a long time + antrik: It's easy, and goes like this: have a session on one + system, log in from another, do screen -x and wait some time. + I do this regularily. haven't had a crash in ages. + (BTW, I'm not sure I ever had a crash on srceen -x... at that + time, I wasn't using -x. I often had crashes with screen -r. my + impression back then was that it works better when doing -rd -- in fact, + I always do that now, so I can't say whether crashes still happen with + only -r...) + +2011-10-26: + + so I was saying the other day that I haven't had a screen crash in + a long time... well, here it was :-( + this time it didn't crash on reconnect though, but already + before. probably when I killed the hanging ssh connection diff --git a/open_issues/user-space_device_drivers.mdwn b/open_issues/user-space_device_drivers.mdwn index e929f2bf..70c3c6dc 100644 --- a/open_issues/user-space_device_drivers.mdwn +++ b/open_issues/user-space_device_drivers.mdwn @@ -97,7 +97,7 @@ IRC, freenode, #hurd, 2011-07-27 * Hurd on L4: deva, fabrica - * [[DDE]] + * [[/DDE]] * Minix 3 @@ -195,7 +195,7 @@ IRC, freenode, #hurd, 2011-07-27 # External Projects - * [[DDE]] + * [[/DDE]] * diff --git a/open_issues/xen_lseek.mdwn b/open_issues/xen_lseek.mdwn new file mode 100644 index 00000000..accc7c8f --- /dev/null +++ b/open_issues/xen_lseek.mdwn @@ -0,0 +1,35 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_xen]] + +IRC, freenode, #hurd, 2011-11-02: + + btw, we have a performance issue with xen + an lseek() call costs a huge lot + like 1ms + while the same costs just a few dozens µs with kvm + there's of course the cost of switching between ring3, ring0, + ring1, ring0, ring3, but still + oh, nice. + lseek is supposed to perform only a back&forth + and I don't observe disk activity, so it's not waiting for the disk + to complete whatever atime change & such :) + it was mentioned that perhaps xen in hvm mode with pv drivers would + be faster + thanks to the ring3/"1" switching being done by the processor + (and assuming npt) + hm + i'll look into that, sounds fun. + :) + Here is a testcase: + http://www.gnu.org/software/hurd/open_issues/performance/io_system/binutils_ld_64ksec.html + +[[performance/io_system/binutils_ld_64ksec]]. diff --git a/source_repositories/discussion.mdwn b/source_repositories/discussion.mdwn new file mode 100644 index 00000000..ae64298e --- /dev/null +++ b/source_repositories/discussion.mdwn @@ -0,0 +1,193 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!toc]] + + +# Integrating Development Braches/External Sources + +[[!taglink open_issue_documentation]] + +IRC, freenode, #hurd, 2011-10-11: + + braunr: About integrating salloc (I'll just use this name). + braunr: Ideally, as this is an external import, there should be + an empty Git branch where the initial-import code (right out of your + tree, for example) is added. + braunr: Then this is merged into a GNU Mach salloc branch and + moved into the destination directory. + Then, any bug fix patches that do not concern salloc itself, + should stay separately. + Also, any improvements that are not specific to salloc itself. + All other salloc development commits (try this; no this; fix + other thing; etc.) can be combined into one commit on the salloc branch. + Does that make sense? + braunr: The idea of the separate external/braunr-salloc-import + branch is that any external updates are applied to this branch (where + they'll apply) cleanly, and then it is merged into GNU Mach salloc (or + later master, once salloc is merged) again. + tschwinge: if you really want to use a branch for the external + code import, better actually import it with its history, not just as a + snapshot + (disclaimer: haven't actually tried how well this works in + practice...) + antrik: Yes, generally this is even better, but: + antrik: But typically the external code we want to import is + just a subset of its external repository. And I don't think we want to + have all of the external development history. + that's the only sane way to do proper followup merges... + admittedly it makes git log a bit confusing... + antrik: Why is it the only sane way? + tschwinge: "an empty Git branch", out of the gnumach git + repository ? + tschwinge: then imported "into a GNU Mach salloc branch" ? + tschwinge: otherwise you don't have merge history. you get no idea + why anything changed in the external code; and it's hard to merge any + changes happening in the local repository vs. the imports + antrik: Of course, importing all the history may make sense at + times, but if we use the [Linux]/include/gcc-atomic.h header, we're + surely not going to import all the Linux kernel history for that. + Instead we would update our import branch with every upstream Linux + release, for example. + braunr: You want me to clarify these points? + tschwinge: yes please + braunr: Start with an empty Git repository: mkdir import && cd + import/ && git init + braunr: Add the files to-be-imported in there, adn commit that + as ``import ... version/Git commit from ...''. + tschwinge: maybe + http://book.git-scm.com/5_creating_new_empty_branches.html ? + pinotree: Oh nice. + pinotree: Thanks. + np + Yes, can also do that, or a new Git repository as I said. + braunr: Prefereably add the files in a salloc directory right + away to avoid any merge conflicts later on. + braunr: Then, in GNU Mach, branch off of master: git checkout + -b salloc origin/master + This will be the salloc integration working branch. + braunr: Actually not ``in a salloc directory'', but with the + name the thing has in your upstream repository. + Then, make the external import branch known here: git fetch + ../import master:external/import-braunr-allocator + Then you can merge the external branch into the salloc branch: + git merge external/import-braunr-allocator + got most of it, except for the "salloc directory" + Then, move everything in place, and either commit --amend it to + the merge, or a separate commit. + ah, you really suggest a separate directory not to have any + conflict + then a move + Right. + but there maybe conflict afterwards + I take it that the alloc files live in [x15]/kern/allocator, so + you'd put them into kern/allocator in the new repository, and then (after + merging that branch) move them to kern/salloc (or similar). + i'm not sure i get the point + no they won't + it will live in kern/salloc.c, and the additional required files + like list.h may be put there too + Aha, OK. + but these are clearly separate files, so it should give the same + results, right ? + Well, you could in the import branch put them all into x15/, + then merge, then move the files to kern/salloc.c, misc/list.h, etc. + hm let's call it salloc/ :) + but ok + It's just that the import branch should have the same structure + than upstream has. + To faciliate further imports later on. + well, that's why i don't see the point of the move + And after the merge it should have the file system structure as + used by GNU Mach. + and there will be no upstream; once it's in gnu mach, it has its + own life + Ah, OK. + Well, then it won't matter too much. + wouldn't it be better to make "upstream" (maskym's branch) have + the same structure as gnu mach already has before even creating the + integration branch ? + I though X15 is the upstream. + maksym* + the upstream was actually the userspace library, before x15 got + its own derived version + OK, I see. + Then we really don't care too much, and yes, you can put it all + in the right places right away. + ok + But is my approach understandable given there is an upstream + from which we may want to import later changes? + given there *were* + (fyi, the userspace code was merely a first step which purpose was + to eliminate as many bugs as possible and profile a little) + i don't intend to maintain the userspace code, no + and the x15 version already has some system specific changes that + don't apply to gnu mach + OK, I understand. + braunr: Then, what remains is essentially this: + Then, any bug fix patches that do not concern + salloc itself, should stay separately. + Also, any improvements that are not specific to + salloc itself. + All other salloc development commits (try this; no + this; fix other thing; etc.) can be combined into one commit on the + salloc branch. + Right? + ok + hm, what about the history of maksym's branch ? + braunr: We don't really need it, do we? + If there are distinct commits that make sense to be kept + separate, then that is fine, but all the development commits can usually + be merged. + ``squashed'' + i don't think we do, no + ok so, i'll use his branch as a starting point in a private git + repository, until it's good enough to be merged upstream, then we'll + create the salloc integration branch and later merge that into master + tschwinge: did i get it right ? :) + We can still either keep Maksym's development branch open, or + perhaps create a tag named salloc-before-merge. + Well, I think if we don't need to do the upstream-import thing, + then you can create a salloc integration branch right away, and prepare + everything in there. + ok + i'm not very familiar with remote branches + OK. This way you don't need any. + Just branch off of master (either your local master branch or + origin/master): git checkout -b salloc origin/master + braunr: BTW, I'm not sure it has been clear, so let me restate it: + what large projects using Git usually do when merging new features + upstream is, they entirely drop the development history of the feature, + instead creating a new branch (or reworking the old one) which is + master-centric -- it is a series of commits, which get from the original + master to a state with the new feature implemented in the most + straightforward way + antrik: in one commit ? + antrik: or a few ofc + depends. the commits should be bisectable as usual... i.e. don't + put things that can be separate into one commit, but don't separate + things that depend on each other into separate commits either :-) + the bulk of the new code often goes in single large commit; but + the various changes to existing code necessary to make it work can often + make a pretty long series. but it can differ quite a lot depending on the + specific case + just imagine you are someone reading the history in the future, + and tries to understand what changed while zalloc was replaced by + salloc. what series of commits makes this most obvious? + sure + antrik: new code first in a single commit first, then the + replacements from zalloc to salloc, then if any, special adjustements in + a third commit + (ok twice first, remove one at will :p) + yeah, something along these lines + sometimes we also need praparation commits in advance, changing + the code structure so that the new feature can be added in a + straightforward fashion + or cleanups afterwards -- cgit v1.2.3