diff options
Diffstat (limited to 'open_issues')
| -rw-r--r-- | open_issues/boehm_gc.mdwn | 13 | ||||
| -rw-r--r-- | open_issues/dde.mdwn | 77 | ||||
| -rw-r--r-- | open_issues/extern_inline.mdwn | 39 | ||||
| -rw-r--r-- | open_issues/gnumach_memory_management.mdwn | 15 | ||||
| -rw-r--r-- | open_issues/linux_as_the_kernel.mdwn | 195 | ||||
| -rw-r--r-- | open_issues/mission_statement.mdwn | 611 | ||||
| -rw-r--r-- | open_issues/performance/io_system/read-ahead.mdwn | 116 | ||||
| -rw-r--r-- | open_issues/resource_management_problems/zalloc_panics.mdwn | 47 | ||||
| -rw-r--r-- | open_issues/syslog.mdwn | 39 |
9 files changed, 1133 insertions, 19 deletions
diff --git a/open_issues/boehm_gc.mdwn b/open_issues/boehm_gc.mdwn index ca2063a5..31359da3 100644 --- a/open_issues/boehm_gc.mdwn +++ b/open_issues/boehm_gc.mdwn @@ -332,3 +332,16 @@ It has last been run and compared on 2010-11-10, based on CVS HEAD sources from make libgc use RTMIN+5/6, like done on most of other OSes <youpi> but we don't have RT signals, do we? <pinotree> right :( + + +### IRC, freenode, #hurd, 2012-03-21 + + <pinotree> civodul: given we have to realtime signals (so no range of + signals for them), libgc uses SIGUSR1/2 instead of using SIGRTMIN+5/6 for + its thread synchronization stuff + <pinotree> civodul: which means that if an application using libgc then + sets its own handlers for either of SIGUSR1/2, hell breaks + <civodul> pinotree: ok + <civodul> pinotree: is it a Debian-specific change, or included upstream? + <pinotree> libgc using SIGUSR1/2? upstream + <civodul> ok diff --git a/open_issues/dde.mdwn b/open_issues/dde.mdwn index adb070cd..84ad2f40 100644 --- a/open_issues/dde.mdwn +++ b/open_issues/dde.mdwn @@ -187,6 +187,37 @@ At the microkernel davroom at [[community/meetings/FOSDEM_2012]]: <antrik> right +# IRC, freenode, #hurd, 2012-02-19 + + <youpi> antrik: we should probably add a gsoc idea on pci bus arbitration + <youpi> DDE is still experimental for now so it's ok that you have to + configure it by hand, but it should be automatic at some ponit + + +## IRC, freenode, #hurd, 2012-02-21 + + <braunr> i'm not familiar with the new gnumach interface for userspace + drivers, but can this pci enumerator be written with it as it is ? + <braunr> (i'm not asking for a precise answer, just yes - even probably - + or no) + <braunr> (idk or utsl will do as well) + <youpi> I'd say yes + <youpi> since all drivers need is interrupts, io ports and iomem + <youpi> the latter was already available through /dev/mem + <youpi> io ports through the i386 rpcs + <youpi> the changes provide both interrupts, and physical-contiguous + allocation + <youpi> it should be way enough + <braunr> youpi: ok + <braunr> youpi: thanks for the details :) + <antrik> braunr: this was mentioned in the context of the interrupt + forwarding interface... the original one implemented by zhengda isn't + suitable for a PCI server; but the ones proposed by youpi and tschwinge + would work + <antrik> same for the physical memory interface: the current implementation + doesn't allow delegation; but I already said that it's wrong + + # IRC, freenode, #hurd, 2012-02-20 <youpi> I was a bit wary of including the ton of dde headers in hurd-dev @@ -361,3 +392,49 @@ At the microkernel davroom at [[community/meetings/FOSDEM_2012]]: <youpi> because hardware is slow anyway <ArneBab> jupp <ArneBab> but it is important to see that in real life + + +# IRC, freenode, #hurd, 2012-04-01 + + <youpi> antrik: I wonder whether you could actually not route the IRQs to a + non-zero ring, AIUI you can in the x86 IDT table + <antrik> youpi: you mean having a userspace server for each (non-timer) + interrupt? + <antrik> youpi: how would a userspace IRQ handler interact with the + scheduler? + <youpi> antrik: it doesn't necessarily have to + <youpi> provided that it's trusted + <antrik> youpi: how would you do CPU time accounting if there is no + interaction with the scheduler?... + <youpi> antrik: you don't necessarily want to care about it + <antrik> youpi: well, that would mean that all drivers handling interrupts + would have to be trusted to not use more than a very small part of CPU + time... + <youpi> yes + <youpi> which is usually needed for interrupt handlers anyway + <antrik> youpi: nah, the bottom handler only has to do very basic stuff; + afterwards, we can pass off to "normal" driver processes, scheduled just + like other processes... but that requires some interaction between the + IRQ handler and the scheduler I think + <youpi> the IRQ handler can wake up a thread, yes + <youpi> no need for anything special there + <antrik> so the userspace IRQ server would just decide what process to wake + up, and then call the scheduler to do a normal task switch? I guess + that's possible; but I'm not sure it would buy much... + <youpi> it would permit userland to quickly react to the IRQ + <youpi> such as acknowledge it to the hardware etc. + <antrik> yeah, but my point is that I don't see much benefit in having this + part of the code isolated in a userspace process... it has to be trusted + anyways, and it's pretty trivial too + <youpi> I never said it was a good idea + + +# IRC, freenode, #hurd, 2012-04-06 + + <braunr> oh i forgot about my work on pcap + <braunr> is devnode (or devopen or whatever) in the upstream repository now + ? + <antrik> can't say for sure, but I'd be surprised... don't remember seeing + any movement in that regard :-( + <braunr> wasn't it needed for dde ? + <antrik> hm... good point diff --git a/open_issues/extern_inline.mdwn b/open_issues/extern_inline.mdwn index a56d4902..a3a22b16 100644 --- a/open_issues/extern_inline.mdwn +++ b/open_issues/extern_inline.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2012 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,7 +10,10 @@ License|/fdl]]."]]"""]] [[!tag open_issue_hurd]] -IRC, unknown channel, unknown date. +[[!toc]] + + +# IRC, unknown channel, unknown date <tschwinge> youpi: Did you ever review the Savannah hurd branch master-fix_extern_inline? <youpi> why static inlines instead of extern lines ? @@ -72,3 +75,35 @@ IRC, unknown channel, unknown date. <youpi> especially since the semantic has changed over time and according to standards :) <tschwinge> And then GCC changing that according to C99. <tschwinge> Yes. + + +# IRC, freenode, #hurd, 2012-03-14 + + <youpi> + http://anonscm.debian.org/gitweb/?p=pkg-hurd/hurd.git;a=blob;f=debian/patches/extern_inline_fix.patch;h=b9eacbff97dc56e99a69ddb601a5fc948f6e44a7;hb=HEAD + <youpi> maybe review it, and then we apply it + <pinotree> + http://patch-tracker.debian.org/patch/series/view/hurd/20120222-1/extern_inline_fix.patch + ;) + <civodul> youpi: the #ifdef __USE_EXTERN_INLINES in there and the extra + "extern" decls look wrong to me + <youpi> iirc USE_EXTERN_INLINES is needed + <youpi> otherwise it's not compliant + <youpi> or maybe it's for -O0 + <youpi> anyway IIRC it's needed + <civodul> when !defined __USE_EXTERN_INLINES, you end up with extern decls + with no corresponding definition + <youpi> yes + <youpi> they are defined in the code + <civodul> where? + <youpi> there's a special .c file in each lib + <youpi> libdiskfs/extern-inline.c + <youpi> etc + <civodul> oooh, right + <youpi> extern inline means that anyway + <youpi> the compiler is allowed to not always inline + <civodul> yes + <civodul> that looks good to me, then + <civodul> youpi: can you apply it, with proper authorship & co.? + <civodul> (no rush, though) + <youpi> sure diff --git a/open_issues/gnumach_memory_management.mdwn b/open_issues/gnumach_memory_management.mdwn index d29e316c..f8e27e62 100644 --- a/open_issues/gnumach_memory_management.mdwn +++ b/open_issues/gnumach_memory_management.mdwn @@ -2101,3 +2101,18 @@ There is a [[!FF_project 266]][[!tag bounty]] on this task. branch in master ? <youpi> I was considering as soon as mcsim gets his paper <braunr> right + + +# IRC, freenode, #hurd, 2012-02-22 + + <mcsim> Do I understand correct, that real memory page should be + necessarily in one of following lists: vm_page_queue_active, + vm_page_queue_inactive, vm_page_queue_free? + <braunr> cached pages are + <braunr> some special pages used only by the kernel aren't + <braunr> pages can be both wired and cached (i.e. managed by the page + cache), so that they can be passed to external applications and then + unwired (as is the case with your host_slab_info() function if you + remember) + <braunr> use "physical" instead of "real memory" + <mcsim> braunr: thank you. diff --git a/open_issues/linux_as_the_kernel.mdwn b/open_issues/linux_as_the_kernel.mdwn index f14b777e..1d84d777 100644 --- a/open_issues/linux_as_the_kernel.mdwn +++ b/open_issues/linux_as_the_kernel.mdwn @@ -40,3 +40,198 @@ Richard's X-15 Mach re-implementation: the whole talk; but they mentioned similar possibilities to what I'm envisioning here :-) + +## IRC, freenode, #hurd, 2012-03-28 + + <mel__> is there currently any work going on with respect to + Mach-alternatives? + <antrik> mel__: no + <antrik> we have other priorities to take care of :-) + <braunr> antrik: i still intend to try giving linux a "mach personality" :) + <braunr> antrik: but i don't have much time for development currently :( + <mel__> antrik: which means that the hope is that Mach can be turned into + something really well working (i.e. secure/scalable/etc.)? + <antrik> mel__: yes, that's the venue we are pursuing + <antrik> (and IMHO the right one... though the Linux layer mentioned by + braunr is also one of my favourite projects, that we should pursue *in + parallel* to the existing Mach-based implementation) + <mel__> what is this Linux Layer exactly? + <mel__> a Linux instance running on top of Mach in parallel to Hurd + serverS? + <braunr> mel__: not exactly + <braunr> mel__: it would involve adding a mach layer on top of linux + actually + <braunr> mel__: so that linux can be used as a mach kernel + <mel__> Ah! + <mel__> Running Hurd on top of Linux + <mel__> :-D + <mel__> Funny + <braunr> ironic, yes + <braunr> but very pragmatic + <mel__> and THEN + <antrik> yeah. I most like the name: Hurd Emulation Layer on + Linux... i.e. HELL :-) + <mel__> we use a device driver framework something so that we can use Linux + device drivers in Hurd! + <mel__> on top of Linux.... + <braunr> yes + <braunr> i guess a transition phase would include using in kernel drivers + directly for a while + <mel__> and somebody is working on that? + <antrik> mel__: well, for using Linux drivers we are persuing DDE, which + allows us doing that with Mach as well + <braunr> then grabbing them out of the kernel and into dde + <braunr> not yet + <antrik> (in fact we have been using Linux drivers since forever... they + just haven't been updated for ages) + <mel__> I would _guess_ that it is not that hard. + <braunr> it's not + <mel__> Basically one would need to implement the message passing interface + thing in linux I guess. + <braunr> and many exported kernel objects like tasks, threads, etc.. + <braunr> and implement all the RPCs normally implemented by the kernel + <braunr> but it's doable + <antrik> mel__: the IPC aspect is one part, but probably the less tricky + one. the external pager interface is really the crucial element + <mel__> uh + <mel__> yeah + <mel__> hooking into linux virtual memory stuff + <mel__> sounds delicate + <braunr> it's true that some interactions between the linux VM and file + systems (the linux equivalent of our pagers) is synchronous + <braunr> but i don't think it would be that hard considering the amount of + facilities available in linux + <braunr> it's just work, takes time, needs debugging, reviewing, testing, + etc.. + <lcc> hurd on top of linux. how would that work? + <braunr> 15:30 < braunr> antrik: i still intend to try giving linux a "mach + personality" :) + <braunr> lcc: 7 system calls and a few hundreds of RPCs on top, the + internal magic of course, and voila .. + <antrik> of course porting Mach still requires work + <mel__> that would then be GNU/Hurd/Linux + <mel__> :-) + <antrik> hehe + <braunr> eh + <antrik> braunr: BTW, are you more thinking of a userspace solution on top + of standard Linux mechanisms, or additions to Linux itself?... + <antrik> (we probably talked about it already, but I don't remember all the + details...) + <braunr> antrik: adding to linux + <antrik> do you think a pure userspace solution would be realistic at all? + (at the expense of some performance of course) + <mel__> it's probably comparable to the qemu vs. qemu/kvm thing + <antrik> yeah, I guess that pretty much sums it up... + <braunr> antrik: i don't know :/ + <antrik> OK + <lcc> how challenging is it to port mach? + <antrik> lcc: it requires good low-level knowledge of the platform in + question. having that, I guess it shouldn't be too hard to add the + necessary code in Mach... + <antrik> TBH I'm not sure how much knowledge of Mach internals is required + <braunr> the pmap module is the main thing to port + <antrik> braunr: well, sartakov seemed to have most trouble with the + scheduler when he attempted the ARM port... + <braunr> that's strange + <antrik> at least there was quite a long thread where he asked about how + task switching works in Mach + <braunr> ok + <braunr> that would be interesting + <braunr> i thought intereacting with the hardclock was enough + + +## IRC, freenode, #hurd, 2012-04-05 + + <braunr> antrik: don't you think HELL is worth a try for the GSoC btw ? + <antrik> braunr: definitely. I haven't managed to rework the project ideas + list at all this year... but it's something I wanted there for a long + time + + <youngrw> just out of curiousity, what is HELL ? + <antrik> Hurd Emulation Layer on Linux + <braunr> youngrw: it can be described in several ways + <braunr> youngrw: basically, it's a mach interface on top of linux + <youngrw> implementing I suppose both the IPC mechanism and memory + management interface? + <mel__> youngrw: basically that. more generally: implement everything in + order to let Hurd run on that layer. + <antrik> well, it's slightly more complicated in my view... it's basically + anything that allows running a Hurdish environment on top of + GNU/Linux. it might be simply an implementation/emulation of Mach + mechanisms; but it also *might* work on a slightly higher level... + <youngrw> antrik: how might HELL operate at the slighty higher level like + you describe? + <antrik> let's call it low-level HELL and high-level HELL ;-) + <antrik> (a more descriptive name would be hurdenv... but HELL is so much + more catchy :-) ) + <antrik> so, high-level HELL... basically, the idea would be not to emulate + the kernel facilities and run all the real Hurd servers; but instead to + use special servers implementing the Hurd interfaces, but on top of + standard Linux facilities + <antrik> hurdish programs could run in such an environment, as long as they + aren't too low-level + <antrik> I wonder whether generic RPC interfaces could be implemented with + some side channel tunneled though the ordinary Linux FS interfaces... + <antrik> so translators would be loaded as FUSE modules for example, but + could still present generic interfaces + <youngrw> That's actually pretty different from what I was expecting + <antrik> what were you expecting? + <youngrw> maybe something where the entire kernel interface is emulated by + a running user process, like a kind of virtual machine + <youngrw> I hope that makes sense--I may be using my words incorrectly. + <antrik> youngrw: that would be in the low-level HELL category + <youngrw> antrik: right; I had the misconception that the level was defined + by how it made use of the underlying linux system + <youngrw> and that different HELL designs would always implement the mach + interface + + +## IRC, freenode, #hurd, 2012-04-06 + + <braunr> antrik: i think we have diverging ideas about how to use linux for + the hurd + <braunr> antrik: what you seem to want are really emulation componants, + like e.g. ext2fs and pfinet actually using the linux implementation + <braunr> (correct me if i'm mistaken) + <braunr> whereas my project is to make linux behave as a mach clone + <antrik> braunr: as I said, I consider both variants -- either a high-level + HELL or a low-level HELL + <braunr> ok + <antrik> (or perhaps a mix of both) + <braunr> a mix would be best imho + <antrik> yeah, probably + <braunr> so we have the real hurd, the real mach interface, and a set of + native translators (e.g. ext2fs) along some emulating their functionality + using linux code (e.g. a hypothetical ext4fs) + <antrik> I don't think we would have emulation servers for individual Linux + filesystems. rather, a generic server interfacing with the Linux VFS + tree... + <braunr> ok + + <antrik> braunr: BTW, I think I mentioned a couple of years ago that the + most realistic route towards a modern Mach in my opinion would be taking + a modern BSD (or Linux), and redo what the original Mach developers did + -- i.e. add the Mach-specific features, and drop the unnecessary UNIX + stuff + <braunr> antrik: :) + <braunr> we had discussions about it some time ago yes + <antrik> later I realised that it's better *not* to drop the UNIX + interfaces, but rather to keep them in parallel :-) + <braunr> antrik: for what purpose ? + <braunr> (i can imagine a few, but i'd like to know your idea) + <antrik> for the purpose of HELL :-) + <braunr> well hell would be the implementation, but what do you keep these + unix interfaces for ? + <antrik> i.e. people being able to play around with a Hurd environment + while staying with their existing system + <braunr> yes, i see + <braunr> i was considering doing that for development, yes + <braunr> uml first, and then i realized i wouldn't need it :) + <braunr> then i remembed netbsd and its syscall emulation layer + <antrik> also we might leverage some "foreign" userspace infrastructure + that way, such as udev + <antrik> (in the case of Linux that is... not sure whether NetBSD has + something similar at all ;-) ) + <braunr> i'll have to check, it's been a long time since i've really used + it + <braunr> they must use a pure devfs instance now diff --git a/open_issues/mission_statement.mdwn b/open_issues/mission_statement.mdwn index d136e3a8..17f148a9 100644 --- a/open_issues/mission_statement.mdwn +++ b/open_issues/mission_statement.mdwn @@ -47,3 +47,614 @@ License|/fdl]]."]]"""]] <antrik> heh, nice: http://telepathy.freedesktop.org/wiki/Rationale <antrik> most of this could be read as a rationale for the Hurd just as well ;-) + + +# IRC, freenode, #hurd, 2012-04-06 + + <braunr> LibreMan: the real feature of the hurd is its extensibility + +[[/Extensibility]], [[/advantages]]. + + <braunr> LibreMan: (though it could be improved even further) + <LibreMan> braunr: yeah, I keep reading that ... but that sounds too + abstract, I can not imagine what useful could that provide to the actual + users + <braunr> LibreMan: say fuse, but improved + <braunr> LibreMan: do you see how useful fuse is ? + <braunr> if so, you shouldn't have trouble imagining the gap between linux + without fuse and linux with fuse is about the same as linux with fuse and + the hurd + <braunr> and yes, it's abstract + <braunr> translators are not only about file systems + <LibreMan> braunr: well, its main advantage is that it's running in + user-space and therefore doesn't need root priviledges to mount whatever + fs you want? + <braunr> no + <braunr> you don't need to change the kernel, or implement weird tricks to + get what you want working + <LibreMan> braunr: okay, but there is fuse for Linux ... so the + difference/advantages need to be between Linux WITH fuse and Hurd + <braunr> that's what i'm saying + <LibreMan> the issue I have is that I do not see why anyone would have any + incentive to switch to Hurd + <braunr> there isn't much, which is why we stick with unix instead of, + e.g. plan9 or other advanced systems + <pinotree> try to use fuse on a server where there is no fuse installed + <LibreMan> if I want fuse-like functionallity I just install FUSE, no need + for Hurd ... so the reson to use it is not there + <braunr> LibreMan: read what i wrote + <braunr> using the hurd compared to using linux with fuse is about the same + as using linux with fuse compared to using linux without fuse + <LibreMan> braunr: ah, sorry ... I see + <braunr> it's a step further + <braunr> in theory, developers can add/remove the components they want, + making system development faster and more reliable + <braunr> where with unix, you need stuff like user mode linux or a virtual + machine + <LibreMan> braunr: but in practice it was the opposite so far :) + <braunr> not really + <braunr> it's a lack of manpower + <braunr> not a problem of partice versus theory + <braunr> practice* + <LibreMan> braunr: what do you think are the reasons why Hurd developement + is so slow if it should be faster in theory? + +[[faq/how_many_developers]]. + + <braunr> 17:30 < braunr> it's a lack of manpower + <braunr> pay someone to do the job + <braunr> :p + <LibreMan> braunr: then why does Linux get the manpower but Hurd doesn't? + <braunr> $$ + <LibreMan> braunr: ?? + <braunr> linux developers are paid + <LibreMan> because companies are using it :) + <braunr> yes + <LibreMan> why are they not using Hurd then? + <braunr> because it wasn't reliable enough + <LibreMan> Linux wasn't either at some point + <braunr> sure + <braunr> but when it became, the switch towards its use began + <braunr> now that they have something free and already working, there is no + point switching again + <LibreMan> paid devs join only AFTER volunteers got it to the stage that it + was useful to companies + <braunr> well linux was easier to develop at the beginning (and is still + today because of several kernel hacking features) + <braunr> it followed the traditional unix model, nothing was really new + about it + <LibreMan> braunr: exactly! that's why I think that Hurd needs to have very + compelling technical advantages to overcome that barrier + <braunr> few people/companies really care about such technical advantages + <braunr> they don't care if there are ugly tricks to overcome some problems + <LibreMan> you mean about such that Hurd can provide, right? + <braunr> it's not elegant, but most of the time they're not even aware of + it + <braunr> yes + <LibreMan> that's eaxctly my point ... most people do not care if it's + "elegant" from a programmers POV, they care whether it WORKS + <braunr> well yes + <braunr> what's your point ? + <LibreMan> all I see about Hurd is how "elegant" it is ... but that doesn't + matter if it doesn't provide any practical advantages + <braunr> you want us to expose a killer feature amazing enough to make the + world use our code ? + <LibreMan> well, I want Hurd to succeed and try to identify the resons it + doesn't + <braunr> it does, but not to the point of making people use it + <braunr> unix *is* good enough + <braunr> same reason plan9 "failed" really + <LibreMan> define your idea of Hurd succeeding then, I thought it was to + make it useful to the point that people use it :) + <braunr> there are many other attempts to make better system architectures + <braunr> it is + <braunr> people are still using windows you know, and i really don't see + why, but it does the work for them + <LibreMan> <braunr> you want us to expose a killer feature amazing enough + to make the world use our code ? --- YES ;) + <braunr> other people can think about the same between unix and the hurd + <braunr> LibreMan: well too bad, there is none, because, again, unix isn't + that bad + <braunr> it doesn't prevent us from making a better system that is usable + <LibreMan> to explain my take on this - there are two kind of people, those + who care about philosophy behind software (and its consequences, FSF + etc.) and those who don't + <LibreMan> it's the job of those who do care to make the sw so good that + those who do not care switch to it = victory :) + <LibreMan> as I said the reasons I want Hurd to succeed are more + "political" than technical ... I do not know how many Hurd devs agree + with that kind of sentiment but I'd rather want a GNU project to be in + the forefront than that of a "benevolent dictator" that doesnt' really + care about user freedom + <LibreMan> from thechnical POV I agree that Linux isn't that bad ... it's + quite good, it's the "behind the scenes" stuff I do not like about it + <LibreMan> I'm kind of confused right now ... what exactly is to point of + Hurd then? I thought it was to make it good enough or better than Linux + so users start using it (privatly or corporate) + <LibreMan> is this just a research project that isn't intended to be used + by "general population"? + <braunr> LibreMan: it's an operating system project + <braunr> some people try to make it as good as it can be, but it's not easy + <braunr> it's not a pet or research only system + <LibreMan> braunr: I see what it is ... I'm struggling to see what is the + point of it being an "OS project", what's its intended purpose + <braunr> but it doesn't suit all the needs for a general OS yet + <braunr> LibreMan: a general purpose OS like most free unices + <LibreMan> what are the motivations behind making it as good as it can be + <braunr> for us developers ? + <LibreMan> yes + <braunr> for me, the architecture + <LibreMan> whe you say that linux is goos enough then what's the point? + <braunr> we can do better + <LibreMan> for you it's just a hobby that doesn't have any real goal except + challenging yourself to do it? + <braunr> because of lack of time, you could say that + <LibreMan> so you want Hurd to challenge Linux one day, right? + <braunr> challenging isn't the point + <braunr> i'd like to be able to use it for my needs + <LibreMan> well, that wasn't the right choise of word but to be better than + Linux + <braunr> again, you miss the point + <braunr> i don't care much about hurd vs linux + <LibreMan> your own needs, so you do not want others to use it? + <braunr> i care about the hurd and what i do + <braunr> others would think the same + <braunr> they would want it to work for their needs + <LibreMan> I'm asking about you, do YOU want others to use it? is that one + of your goals? + <braunr> not really + <braunr> i let them do what they want + <LibreMan> ah I see, so it is kind of a hobby project for you - you're + doing to for yourself and your own needs + <LibreMan> and don't care if anyone else uses it or not + <braunr> yes, i don't care much about the politics around such projects tb + <braunr> tbh + <LibreMan> is this kind of sentiment prevalent is the Hurd dev community? + <braunr> i don't work on software to break any benevolent dictator or + anyone in particular + <braunr> i don't know + <braunr> i'd say so, yes + <braunr> but not sure + <braunr> i'm not saying they don't care about freedom, don't get me wrong + <braunr> i'd say we sure prefer free software over open source + <braunr> but i don't think people work on the hurd specifically for these + reasons, rather than the technical ones + <LibreMan> interesting ... from the presentation of the project by + outsiders I got the impression that it is significantly about freedom, + GNU - that those are the main drivers + <braunr> if it really was so, we would have grabbed a bsd variant, + relicenced it with GPLv3, and call it FreeGNU or NetGNU + <LibreMan> and that's how I approached the project ... maybe I was wrong, + I'm kind of disappointed if that's so :) I care about those things a + great deal, in fact that's the only reason I care about Hurd really + + <lcc> the hurd is designed to offer more freedom, in various ways, to the + user. freedom from the admin. + <lcc> right? + <braunr> lcc: that's embedded in the term "extensibility", yes + <braunr> lcc: but there are technical solutions for that on other systems + as well now + + <antrik> as for the Hurd, people who said they are interested in it only + because of freedom aspects *never* contributed anything significant + <antrik> *all* serious contributors are motivated at least equally by the + technical merits; often more + <antrik> (though the fact that it's a GNU project is what has brought many + developers here in the first place...) + <LibreMan> antrik: I would phrase it the other way - why do people who have + contributed significantly not care about freedom that much? or ... how do + you know they don't? + <antrik> most of us *do* care about freedem. but it's not our primary + motivation. the freedom aspects are just not strong enough to motivate + anyone alone + <antrik> as braunr already pointed out, if the sole purpose was creating a + GNU kernel, there would be *much* more promising venues for that + <LibreMan> I do not think so ... if you someone where to just take BSD and + rebrand it as AWSOMEnewGNUkernel it wouldn't be looked upon too favorably + <LibreMan> there is an honor aspect to it, to have something developed by + the community that stands by it + <LibreMan> so I do not think it would work + <antrik> BSD has forked countless times, and several of these forks became + very popular. I don't see why a GNU one shouldn't do well enough + <antrik> bat that's beside the point. writing a new boring monolithic + UNIX-like kernel from scratch is not that hard + <antrik> (as Linus has proven, amonst others...) + <antrik> if the sole purpose would be having a GNU kernel, I'd be strongly + advocating writing a new monolithic kernel from scratch + <LibreMan> antrik: ah, snap! not that hard you say? with all the features + Linux has? sure, it's not hard to make a kernel that barely boots but + that's not the point, is it? :) + <antrik> (yes, even now, with the Hurd being almost usable, I still think + it would be easier to get a new monolithic kernel to production quality) + <LibreMan> antrik: and here is was braunr who was pitching extensibility + and faster developement of Hurd as its advantage - and here you come + saying that it would be easier to write monolithic kernel from scratch + <LibreMan> get your story striaght guys ;) + <antrik> the Hurd makes it easier to develop new features. it's not easier + to get it production-ready in the first place + <LibreMan> antrik: what's the difference of developing a feature that makes + it "production ready" and another one that make it "production ready" for + a different use? + <antrik> features don't make a system production ready + <LibreMan> what makes a system production ready? + <LibreMan> what do you consider a "production"? + <antrik> supporting enough use cases that a non-trivial number of users + have their needs covered; and being stable enough that it's not annoying + to use + <LibreMan> either it is easier to develop or it isn't ... either it is + modular from it's core or it isn't + <antrik> well, not only stable enough, but also performant, secure etc. + <antrik> wrong + <LibreMan> are you saying that the fruits of its modularity will show only + after enough modules have been written? + <antrik> a modular system with strong isolation is inherently more + complicated to get right + <LibreMan> that sure is a weird argument to make ... + <LibreMan> right ... but when you get it right, the further development is + much easier? + <antrik> depends. making fundamental changes to how the system works will + always be tricky. but adding new stuff that doesn't require fundamental + changes, building on the existing foundations, is way easier + <antrik> we believe that once we have the fundamentals mostly right, most + things people will be adding will fall into the latter catogory + <antrik> category + <LibreMan> o what's missing to Hurd before it "got it right" and the fast + pace development kicks in? + <antrik> but so far most of the work is in the former category, meaning + progress is slow + <LibreMan> because from readin the site it seems the core is pretty much + done ... what it needs are all the translators, drivers, user-space tools + to make use of that core - is that impression wrong? + <antrik> you are missing the point. there is no unified "development pace" + measurement. it is easier to add certain things right now. but to get the + system production ready, it still requires considerable work on the hard + parts + <antrik> well, it's not as simple ;-) + <LibreMan> are you sure the work on "the hard parts" is ever going to be + done? :) + <antrik> the core is working, but it is still missing some features, and + it's missing lots of performance optimisation and bug fixing + <LibreMan> it seems more hard parts pop up every time you think it is + almost production ready + <antrik> also, we know today that the core could work much better in some + regards if we make some major changes. not a priority right now, but + something that will have to be addressed in the long run to seriously + compete with other systems + <antrik> well, no software is ever done :-) + <antrik> but I hope we will get to a point where the hard parts work well + enough for most people + <LibreMan> in fact I remember the design of Hurd was specifically chose by + RMS because he thought it would be easier to implement modular system - + that was 20 yeras ago? :) + <antrik> yes, and he admitted later that he was totally wrong on that :-) + <LibreMan> yeah, that was one unlucky choice for GNU ... + <antrik> who knows. it's hard to estimate what would have happened it GNU + chose a different route back then + <LibreMan> so ... Hurd is a hobby project for you too? + <LibreMan> or ... what do you hope to achieve by working on Hurd? + <LibreMan> I'm really interested in the motivations of people behind Hurd + as I'm kind of surprised it's not that much freedom and GNU ... + <antrik> it's a hobby project for everyone -- nobody gets paid for working + on it + <antrik> in the long run, I hope the Hurd to be a good platform for my + higher-level ideas. I have a vision of a desktop environment working + quite differently from what exists today; and I believe the extensible + architecture of the Hurd makes it easier to implement these ideas + <LibreMan> that's not what I meant as you may have guessed from my line of + reasoning so far + <LibreMan> yeah, that's my definition of a hobby project :) not whether one + gets payed to do it or not but whether one does it to satisfy their own + curiosity + <antrik> well, curiosity is clearly too narrow + <LibreMan> as far as I'm concerned I'd have a more "political" goal of + influencing the wider world to move toward more freedom + <antrik> but hackers never work on volunteer projects except to scratch + their own itch, or to work on something they are genuinely interested + in. nobody hacks free software just to save the world + <LibreMan> I find some technical aspects very interesting and fun but if + they wouldn't further the goal of more freedom they'd be without purpose + to me + <antrik> just think of the GNU high priority projects list -- it has zery + effect + <antrik> zero + <LibreMan> yeah ... and I think that is a real shame + <LibreMan> I keep thinking that it's because most hackers do not realize + the importance of freedom and the consequences of not having it + <antrik> it's a shame that some people at the FSF seem to believe they can + tell hackers what to work on :-P + <LibreMan> I do not think anybody at FSF actually believes that + <LibreMan> they believe as I do that we can persuade hackers to work on + things after they themselves recognise the significance of it + <antrik> no. there are many many hackers who genuinely believe in + supporting software freedom (both in the Hurd and in other GNU projects) + -- but there are none who would work on projects they are not personally + interested in because of that + <LibreMan> well, how does one become "personally interested" in a project? + surely it's not something you;re born with ... after recognising a + significance of some project some may become personally interested in it + - and that's the point ;) + <antrik> well, if I you mean nobody realises that software freedom is so + important they should work on it instead of doing things they actually + enjoy... they yes, I guess you are right :-P + <antrik> significance is subjective. just because something may be + important to the general public, doesn't mean I personally care about it + <LibreMan> you keep projecting your own concerns into it + <LibreMan> just because you're not interested in something doesn't mean + someone else isn't + <LibreMan> you approach it from the POV that omebody is telling YOU what + you should do ... + <LibreMan> that is not the case + <antrik> LibreMan: well, but there are obviously things no hackers care + about -- or otherwise there would be no need for the high priority + projects list... it's a list of things that would be important for + software freedom, but nobody is interested in working on. and having a + list of them won't change that fact + <LibreMan> antrik: why do you feel entitled to speak for all hackers? the + projects are high priority exactly because there isn;t enough people + working on them, if they were they wouldn't be high priority :) + <LibreMan> so maybe you have cause and effect mixed up ... + <LibreMan> there is no need to list office suite as hight priority because + there is LibreOffice, if there wasn't I'm sure it would be right there on + the priority list + <antrik> LibreMan: err... how is that different from what I said? + <antrik> these projects are there because there are not enough people + working on them -- i.e. hackers are not interested in them + <LibreMan> you said it in a way the implied that hackers are not interested + in working on projects that are required for providing freedom - but + mostly there are, it's just a few project where aren't - and those are + listed as high priority to bring attention to them + <LibreMan> well, maybe after seeing them on a high priority list some + hackes become interested in them - that is the point :) + <antrik> yes, that's what I implied. the fact that there are projects + hackers aren't working on, although they would be important for software + freedom, proves that this is not sufficient motivation for volunteers + <antrik> if software freedom alone would motivate hackers, there would be + enough people working on important projects + <LibreMan> who ever claimed that freedom alone motivated hackers? :) + <antrik> but there aren't. we have the list, and people are *still* not + working on these projects -- q.e.d. + <LibreMan> I do not get what you're trying to prove + <antrik> the track record so far clearly shows that hackers do *not* become + interested in working on these projects just because they are on the list + <antrik> err... you pretty much claimed that Hurd hackers should be + motivated by freedom alone + <antrik> and expressed great disappointment that we aren't + <braunr> LibreMan: you expected the hurd developers to share the common + goal of freedom mainly, and now you're saying you don't think hackers + would work for freedom alone ? + <LibreMan> freedom mainly == freedom alone? + <braunr> antrik: would you see an objection to using netbsd as a code base + for a mach clone ? + <braunr> LibreMan: you said share the common goal of freedom + <LibreMan> you're twisting my word to suit your own line of reasoning + <braunr> implying we all agree this is the priority + <LibreMan> being a priority doesn't mean it is there "alone", does it? + <braunr> it means it's the only one + <LibreMan> in another words, do you reject the possibility of enjoying + working on a project and doing it for freedom? because it seems you + somehow do not allow for that possibility + <braunr> if we agree on it, we can't have multiple priorities per people + <braunr> yes, that's what we're saying + <braunr> freedom isn't a goal + <braunr> it's a constraint + <braunr> the project *has* to be free + <LibreMan> so if you;re doing something to achieve freedom you can not BY + DEFINITION enjoy it? :D + <braunr> LibreMan: more or less, yes + <braunr> i enjoy the technical aspect, i advocate freedom + <LibreMan> then I've just disproven you :) I do things for freedom and + enjoy them + <braunr> no, not for freedom + <LibreMan> yes, for freedom + <braunr> i'm telling you it's not what motivates me to write code + <LibreMan> if I did not believe in freedom I wouldn't do them + <LibreMan> and I'm not talking about you + <braunr> i believe in freedom, my job consists of developing mostly + proprietary software + <braunr> how can you disprove me if you're not talking about me on this ? + <LibreMan> you said it's not possible IN PRINCIPLE, well antrik did and you + agreed - if you did not follow his line of argument then do not try to + continue where he left off ;) + <braunr> what project have you worked on ? + <LibreMan> my personal ones, nothing big + <braunr> so you're not a hacker, you're excluded from the group considered + <LibreMan> I'll tell you when it cathes on :) + <braunr> (bam) + <LibreMan> so now you decide who is and is not a hacker, well ... :) + <braunr> :) + <LibreMan> but ok, let's not talk about me I concede that I'm a lousy one + if any :) + <LibreMan> what about RMS, do you consider him a hacker? + <braunr> i think he became a hacker for other reasons than freedom + <LibreMan> would you say he is not motivated by freedom (if that can be + even concieved of)? :) + <braunr> and sees freedom as necessary too + <braunr> i can't say, i don't know him + <antrik> braunr: nope. in fact we discussed this in the past. someone even + worked on GSoC project bringing Hurd/Mach features to NetBSD -- but AFAIK + nothing came out of it + <braunr> antrik: ok + <LibreMan> well, he is pretty vocal with plenty of writings ... on the + other hand you seemed to know me well enough to proclaim me a non-hacker + <braunr> i don't know why he worked on emacs and gcc rather than the hurd + :p + <braunr> but something other than freedom must have motivated such choices + <antrik> I'm uncertain though whether NetBSD is a more useful base than + Linux. it would offer advantages on the licensing front, but it would not + offer the advantage that people could just run it on their existing + systems... + <LibreMan> gcc seems pretty significant for Linux lol + <braunr> antrik: true + <LibreMan> or GNU + <braunr> antrik: there are already system call stubs, and the VM is very, + very similar + <braunr> LibreMan: the hurd was too, at the time + <LibreMan> he can not work on everything + <braunr> so he ahd to choose, and based his choice on something else than + freedom (since all these projects are free) + <braunr> i guess he enjoyed emacs more + <antrik> LibreMan: RMS is not much of a practicing hacker anymore + nowadays... + <antrik> braunr: yeah, that's another advantage of using NetBSD as a + base... it might be easier to do + <braunr> LibreMan: what was your original question again ? + <braunr> i've been somewhat ironic since that trademark stuff, i'm serious + again now + <antrik> LibreMan: again, freedom is a factor for many of us; but not the + primary motivation + <antrik> (as braunr put, being free software is mandatory for us; but that + doesn't mean the main reason for working on the Hurd is some indirect + benefit for the free software movement...) + <LibreMan> braunr: the original goal was to understand the strong points of + Hurd to I can help communicate them to other hackers who might be + interested in Hurd + <LibreMan> because I wanted it to succeed to advance freedom more + <antrik> LibreMan: well, practice what you preach ;-) + <LibreMan> but now that I've founf that not even devs themselves are that + much interested in freedom I do not have that desire anymore + <antrik> you will hardly motivate other hackers to work on something you do + not even work on yourself... + <LibreMan> and focus my attention somewhere else + <antrik> [sigh] + <braunr> well, you can now state that the hurd has an elegant architecture + allowing many ugly hacks to disappear, and that it doesn't yet handle + sata drives or usb keys or advandced multicast routing or ... + <antrik> LibreMan: how about you listen to what we are saying? + <LibreMan> antrik: so I should work on everything in the world that + advances freedom or shut up? + <antrik> LibreMan: we *are* interested in freedom. we would work on nothing + else than a free software system. it's just not the primary motivation + for working on the Hurd + <antrik> if you primary motivation is advancing free software, the Hurd is + probably indeed not the right project to work on. other projects are more + important for that + <antrik> and that's got nothing to do with our priorities + <antrik> it's simply a matter of what areas free software is most lacking + in. the kernel is not one of them. + <braunr> antrik: my primary concern with netbsd are drivers + <LibreMan> I naively assumed that people working on a GNU project will + share GNU vlaues, instead I find that some of them poke fun at its high + priority projects + <braunr> i poke fun at you + <braunr> because you think trademark has any real value on the free + software community + <LibreMan> braunr: I see, congratulations ... I hope you enjoy it + <antrik> if there were no suitable free software kernels around, many + people might work on the Hurd mostly to advance free software. but as it + stands, having a GNU kernel is secondary + <braunr> yes, freedom is a primary goal when there are no free alternatives + <antrik> LibreMan: you are accusing us of not sharing GNU values, which is + quite outrageous I must say + <braunr> LibreMan: actually no, i'd prefer converstation with someone who + understands what i'm saying + <braunr> even if he contradicts me, like antrik often does + <braunr> (but he's usually right) + <braunr> LibreMan: you just don't want to accept some (many) of us are here + more for technical reasons than ethical ones + <LibreMan> antrik: well, some of your reasoning and tone would seem to + suggest so ... + <braunr> i didn't see antrik being particularly aggressive, but personally, + i react badly to stupidity + <LibreMan> braunr: WHAT? I've never said anything about what you should or + should not do or believe + <braunr> you clearly expected something when you first arrived + <LibreMan> I said I personally expected more enhusiastic people concerning + GNU and freedom but that was my personal expectaion and my personal + disappointment + <antrik> what makes you think we are not enthusiastic about GNU and + software freedom? + <braunr> more enthusiastic is vague, you expected us to be some sort of + freedom fighters + <antrik> just for the record, I'm part of the German core team of the FSFE + <braunr> i even stated early that we're mostly part of the free software + rather than open source movement, and you still find our point of view + disappointing + <antrik> still, it's not my major motivation for working on the Hurd + <antrik> I don't see any contradiction in that + <LibreMan> I don;t know maybe I misunderstand you, I do not mean any + disrespect + <braunr> me neither + <LibreMan> maybe "hackers" truly do think differently than I expected them + to in general and it's not specific to Hurd + <braunr> well the very word hacker describe someone interested by "hacking" + down something to get to understand it + <braunr> it's strongly technical + <LibreMan> antrik: why are you a core team member of th FSFE? what do you + do there and why? is that not motivated by the desire for more freedom? + <braunr> and we're lucky, many of them aren't deeply concerned with money + and secrecy, and prefer being open about their work + <braunr> you still don't get it ... + <antrik> LibreMan: of course it is + <antrik> and hacking free software in general also is (partly) motivated by + that + <antrik> but hacking on the Hurd specifically not so much + <braunr> 20:23 < antrik> LibreMan: we *are* interested in freedom. we would + work on nothing else than a free software system. it's just not the + primary motivation for working on the Hurd + <braunr> he already answered your question there + <antrik> (as I already said, it *is* in fact part of the motivation in my + case... just not the major part) + <LibreMan> antrik: but if it ever achieved wide success and you would be + asy on a "board" to decide future direction would you choose for exacmple + to prevent TiVO-ization over wider adpotion? + <braunr> we already answered that too + <antrik> LibreMan: that's actually not even for us to decide, as long as we + are an official GNU project + <antrik> but of course we are a GNU project because we *do* believe in + software freedom, and obviously wouldn't accept Tivoisation + <braunr> (and our discussion about using netbsd as a code base is a + relevant example of license concerns) + <LibreMan> I'm really trying to get to the core of "not motivated by + freedom" but being "interested in freedom" ... I really do not get that, + if you are interested in freedom wouldn't you want a project you work on + being used to advance it as much as possible and therefore be also + motivated to do it the best while enjoying it to achieve the goal of more + freedom since you value it that much? + <braunr> LibreMan: except for the GPLv2 vs GPLv3 debate, i don't see where + there can be a conflict between freedom and technical interest + <LibreMan> braunr: the issues around freedom are mainly not technical + ... GPLv2 and GPLv3 is also not about technical interests + <braunr> that's my problem with you, i fail to see where the problem you + think of is + <LibreMan> it tends to be about the possibility to extract money and impose + your will on the users which turns out to be highly profitable and + politicaly desirable in some instances + <LibreMan> of course it's technically the best to open-source but how are + you going to sell a product like that? that is the main question + troubling most corporations + <LibreMan> ok, I'm not going to bore you any more ;) I found out what I + needed to know ... now I'm going to try to forget about Hurd and focus on + something else where my help can be more effective at achieving what I + want ;) good luck with your endavours + <antrik> LibreMan: of course we hope for the Hurd to advance the cause of + freedom, just like any free software we would work on... still, it's not + the primary reason why we work on the Hurd, instead of the myriads of + other free software projects out there + + +# IRC, freenode, #hurd, 2012-04-09 + + <antono> what is the most impressive thing about hurd you wold like to + promote? + <antono> killing feature + <antono> i've created some simple hurd screencasts here + http://shelr.tv/records/search?utf8=%E2%9C%93&q=hurd + <antono> but probably i could share something more interesting :) + <antrik> antono: if we had such an obvious killer feature, we wouldn't have + to struggle ;-) + <antrik> the problem is that the advantages of the Hurd architecture are + too abstract for the vast majority of people to take them seriously + <antrik> IMHO the most interesting part of the Hurd is the fully + decentralised (and thus infinitely extensible) VFS mechanism + <antrik> but even that is very abstract... + <antono> antrik: cand i do somenthing relly fundamental with hurd + translator? + <antono> for example i hate old school unix FHS + <antono> I would like to have only /Users/me and /System/GNU + <antono> and i would like to only see it, but behinde the scenes it should + be Debian with FHS layout + <antono> is it possible? + <antrik> antono: of course. not sure translators offer much advantage over + FUSE in this case though... it doesn't really change the functionality of + the VFS; only rearranges the tree a bit + <antrik> (might even be doable with standard Linux features) diff --git a/open_issues/performance/io_system/read-ahead.mdwn b/open_issues/performance/io_system/read-ahead.mdwn index b6851edd..d6a98070 100644 --- a/open_issues/performance/io_system/read-ahead.mdwn +++ b/open_issues/performance/io_system/read-ahead.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,9 +10,18 @@ License|/fdl]]."]]"""]] [[!tag open_issue_gnumach open_issue_hurd]] -[[community/gsoc/project_ideas/disk_io_performance]] +[[!toc]] -IRC, #hurd, freenode, 2011-02-13: + +# [[community/gsoc/project_ideas/disk_io_performance]] + + +# 2011-02 + +[[Etenil]] has been working in this area. + + +## IRC, freenode, #hurd, 2011-02-13 <etenil> youpi: Would libdiskfs/diskfs.h be in the right place to make readahead functions? @@ -28,9 +37,8 @@ IRC, #hurd, freenode, 2011-02-13: portability <youpi> it's not in posix indeed ---- -IRC, #hurd, freenode, 2011-02-14: +## IRC, freenode, #hurd, 2011-02-14 <etenil> youpi: I've investigated prefetching (readahead) techniques. One called DiskSeen seems really efficient. I can't tell yet if it's patented @@ -51,13 +59,8 @@ IRC, #hurd, freenode, 2011-02-14: <braunr> oh, madvise() stuff <braunr> i could help him with that ---- - -[[Etenil]] is now working in this area. - ---- -IRC, freenode, #hurd, 2011-02-15 +## IRC, freenode, #hurd, 2011-02-15 <etenil> oh, I'm looking into prefetching/readahead to improve I/O performance @@ -281,9 +284,8 @@ IRC, freenode, #hurd, 2011-02-15 pretty good description of the necessary changes... unfortunately, these are not publicly visible IIRC :-( ---- -IRC, freenode, #hurd, 2011-02-16 +## IRC, freenode, #hurd, 2011-02-16 <etenil> braunr: I've looked in the kernel to see where prefetching would fit best. We talked of the VM yesterday, but I'm not sure about it. It @@ -299,3 +301,91 @@ IRC, freenode, #hurd, 2011-02-16 the right place is the VM subsystem [[clustered_page_faults]] + + +# 2012-03 + + +## IRC, freenode, #hurd, 2012-03-21 + + <mcsim> I thought that readahead should have some heuristics, like + accounting size of object and last access time, but i didn't find any in + kam's patch. Are heuristics needed or it will be overhead for + microkernel? + <youpi> size of object and last access time are not necessarily useful to + take into account + <youpi> what would usually typically be kept is the amount of contiguous + data that has been read lately + <youpi> to know whether it's random or sequential, and how much is read + <youpi> (the whole size of the object does not necessarily give any + indication of how much of it will be read) + <mcsim> if big object is accessed often, performance could be increased if + frame that will be read ahead will be increased too. + <youpi> yes, but the size of the object really does not matter + <youpi> you can just observe how much data is read and realize that it's + read a lot + <youpi> all the more so with userland fs translators + <youpi> it's not because you mount a CD image that you need to read it all + <mcsim> youpi: indeed. this will be better. But on other hand there is + principle about policy and mechanism. And kernel should implement + mechanism, but heuristics seems to be policy. Or in this case moving + readahead policy to user level would be overhead? + <antrik> mcsim: paging policy is all in kernel anyways; so it makes perfect + sense to put the readahead policy there as well + <antrik> (of course it can be argued -- probably rightly -- that all of + this should go into userspace instead...) + <mcsim> antrik: probably defpager partly could do that. AFAIR, it is + possible for defpager to return more memory than was asked. + <mcsim> antrik: I want to outline what should be done during gsoc. First, + kernel should support simple readahead for specified number of pages + (regarding direction of access) + simple heuristic for changing frame + size. Also default pager could make some analysis, for instance if it has + many data located consequentially it could return more data then was + asked. For other pagers I won't do anything. Is it suitable? + <antrik> mcsim: I think we actually had the same discussion already with + KAM ;-) + <antrik> for clustered pageout, the kernel *has* to make the decision. I'm + really not convinced it makes sense to leave the decision for clustered + pagein to the individual pagers + <antrik> especially as this will actually complicate matters because a) it + will require work in *every* pager, and b) it will probably make handling + of MADVISE & friends more complex + <antrik> implementing readahead only for the default pager would actually + be rather unrewarding. I'm pretty sure it's the one giving the *least* + benefit + <antrik> it's much, much more important for ext2 + <youpi> mcsim: maybe try to dig in the irc logs, we discussed about it with + neal. the current natural place would be the kernel, because it's the + piece that gets the traps and thus knows what happens with each + projection, while the backend just provides the pages without knowing + which projection wants it. Moving to userland would not only be overhead, + but quite difficult + <mcsim> antrik: OK, but I'm not sure that I could do it for ext2. + <mcsim> OK, I'll dig. + + +## IRC, freenode, #hurd, 2012-04-01 + + <mcsim> as part of implementing of readahead project I have to add + interface for setting appropriate behaviour for memory range. This + interface than should be compatible with madvise call, that has a lot of + possible advises, but most part of them are specific for Linux (according + to man page). Should mach also support these Linux-specific values? + <mcsim> p.s. these Linux-specific values shouldn't affect readahead + algorithm. + <youpi> the interface shouldn't prevent from adding them some day + <youpi> so that we don't have to add them yet + <mcsim> ok. And what behaviour with value MADV_NORMAL should be look like? + Seems that it should be synonym to MADV_SEQUENTIAL, isn't it? + <youpi> no, it just means "no idea what it is" + <youpi> in the linux implementation, that means some given readahead value + <youpi> while SEQUENTIAL means twice as much + <youpi> and RANDOM means zero + <mcsim> youpi: thank you. + <mcsim> youpi: Than, it seems to be better that kernel interface for + setting behaviour will accept readahead value, without hiding it behind + such constants, like VM_BEHAVIOR_DEFAULT (like it was in kam's + patch). And than implementation of madvise will call vm_behaviour_set + with appropriate frame size. Is that right? + <youpi> question of taste, better ask on the list + <mcsim> ok diff --git a/open_issues/resource_management_problems/zalloc_panics.mdwn b/open_issues/resource_management_problems/zalloc_panics.mdwn index 09710022..9c29b07c 100644 --- a/open_issues/resource_management_problems/zalloc_panics.mdwn +++ b/open_issues/resource_management_problems/zalloc_panics.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2005, 2007, 2008, 2010 Free Software Foundation, -Inc."]] +[[!meta copyright="Copyright © 2005, 2007, 2008, 2010, 2012 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -54,3 +54,46 @@ I started various other experiments with creating child processes (fork bombs), * After opening/leaking lots of ports to /dev/null (32768 it seems), the NULL translator somehow becomes disfunctional, and a new instance is started While most of these Observations clearly show an exhaustion of kernel memory which is not surprising, some of the oddities seem to indicate problems that might deserve further investigation. + + +# IRC, freenode, #hurd, 2012-04-01 + + <mel__> antrik: i just found + http://www.gnu.org/software/hurd/open_issues/resource_management_problems/zalloc_panics.html + -- that is from 2007. is this still the current status? + <youpi> mel__: probably + <mcsim> mel__: gnumach has no more zalloc allocator, so I doubt if it could + be a problem. + +[[gnumach_memory_management]]. + + <youpi> mcsim: but it still has an allocator + <youpi> which can run out of resources + <mcsim> AFAIR, now there is no such limit. + <youpi> err, there is + <youpi> the size of your RAM :) + <mcsim> In zalloc appearing of this message didn't depend of available size + of free ram. + <youpi> then update the description, but I'm still getting allocation + errors, when userland makes crazy things like creating millions of tasks + <mcsim> At least it could appear when there still was free memory + <youpi> and that's not surprising + <youpi> sure, I know that *some* limits have been removed, but there + weren't so many, and I have seen cases where it's simply mach running out + of memory + <youpi> also, we have a limited amount of virtual addressing space + <antrik> mel__: this writeup is outdated in several regards. *some* of the + observations might still be relevant, but nothing that seems + particularily important + <antrik> the zalloc panics have pretty much disappeared after the default + zalloc zone size has been considerably extended (which was not possible + before because of some bug) + <mel__> i see + <antrik> but as mcsim pointed out, with the new allocator not relying on a + fixed-sized zalloc zone at all, they are even less likely, and should + happen only if all memory is exhausted + <antrik> I guess this outdated report can just be dropped + <mcsim> I think, that now it is problem rather of absence of OOM-killer or + resource manager + <antrik> mcsim: right :-) + <antrik> (and we have separate articles about that) diff --git a/open_issues/syslog.mdwn b/open_issues/syslog.mdwn index 2e902698..19cba82e 100644 --- a/open_issues/syslog.mdwn +++ b/open_issues/syslog.mdwn @@ -1,4 +1,19 @@ -IRC, unknwon channel, unknown date. +[[!meta copyright="Copyright © 2010, 2011, 2012 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +[[!toc]] + +# IRC, unknown channel, unknown date <tschwinge> scolobb: In wiki edit 60accafa79f645ae61b578403f7fc0c11914b725 I see that you intend(ed) to use syslog for logging debug messages. I @@ -16,7 +31,7 @@ IRC, unknwon channel, unknown date. doesn't help if I restart syslogd). -IRC, freenode, #hurd, 2011-08-08 +# IRC, freenode, #hurd, 2011-08-08 < pinotree> wow, `logger` + a simple C udp server can cause havoc < pinotree> youpi: ever seen something like @@ -70,3 +85,23 @@ IRC, OFTC, #debian-hurd, 2011-11-02: <tschwinge> Yep. What I've been doing ever since, is deinstall all *syslog* packages. <tschwinge> This ``fixed'' all syslog() hangs. + + +# IRC, freenode, #hurd, 2012-03-28 + + <braunr> i can see lots of CRON processes hanging around + <braunr> pinotree: crontab -l was hanging too when trying to quickly see + what went wrong + <braunr> so it may be an unreleased lock of some kind + <antrik> braunr: do you have syslog installed by any chance?... + <antrik> IIRC that bug has never been fixed :-( + <braunr> yes syslogd is running + <antrik> that's probably the culprit then + <braunr> ok + <braunr> i'll just disable it for now then + <antrik> the error has existed for years + <antrik> was similar for me though: for a long time I have been hearing + about this issue, and only suddenly I started experiencing it myself... + <antrik> it depends on how many things are actually logged. IIRC the hang + happens when some client sends 128 messages to syslog or something like + that |