diff options
Diffstat (limited to 'hurd/translator')
| -rw-r--r-- | hurd/translator/procfs/jkoenig.mdwn | 55 | ||||
| -rw-r--r-- | hurd/translator/procfs/jkoenig/discussion.mdwn | 168 | ||||
| -rw-r--r-- | hurd/translator/tmpfs.mdwn | 12 | ||||
| -rw-r--r-- | hurd/translator/tmpfs/discussion.mdwn | 17 | ||||
| -rw-r--r-- | hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn | 121 |
5 files changed, 274 insertions, 99 deletions
diff --git a/hurd/translator/procfs/jkoenig.mdwn b/hurd/translator/procfs/jkoenig.mdwn index 1275ce52..9543b658 100644 --- a/hurd/translator/procfs/jkoenig.mdwn +++ b/hurd/translator/procfs/jkoenig.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -21,56 +21,3 @@ Testing it is as simple as this: $ make $ settrans -ca proc procfs --compatible $ ls -l proc/ - - -# Open Issues - -[[!tag open_issue_hurd]] - - * IRC, #hurd, around September 2010 - - <youpi> jkoenig: from a quick read, your procfs implementation seems quite - simple, probably much more what I was expecting from Madhusudan (who probably - now hates you :) ) - <youpi> jkoenig: is it not possible to provide a /proc/self which points at the - client's pid? - <pinotree> (also, shouldn't /proc/version say something else than "Linux"?) - <youpi> to make linux tools work, no :/ - <youpi> kfreebsd does that too - <pinotree> really? - <youpi> yes - <youpi> (kfreebsd, not freebsd) - <pinotree> does kbsd's one print just "Linux version x.y.z" too, or something - more eg in a second line? - <pinotree> (as curiosity) - <youpi> % cat /proc/version - <youpi> Linux version 2.6.16 (des@freebsd.org) (gcc version 4.3.5) #4 Sun Dec - 18 04:30:00 CET 1977 - <pinotree> k - <giselher> I had some problems with killall5 to read the pid from /proc, Is - this now more reliable? - <youpi> I haven't tested with jkoenig's implementation - [...] - <pinotree> looks like he did 'self' too, see rootdir_entries[] in rootdir.c - <youpi> but it doesn't point at self - <antrik> youpi: there is no way to provide /proc/self, because the server - doesn't know the identity of the client - <youpi> :/ - <antrik> youpi: using the existing mechanisms, we would need another magic - lookup type - <antrik> an alternative idea I discussed with cfhammer once would be for the - client to voluntarily provide it's identity to the server... but that would - be a rather fundamental change that requires careful consideration - <antrik> also, object migration could be used, so the implementation would be - provided by the server, but the execution would happen in the client... but - that's even more involved :-) - <youpi> but we've seen how much that'd help with a lot of other stuff - <antrik> I'm not sure whether we discussed this on the ML at some point, or - only on IRC - <youpi> it "just" needs to be commited :) - <antrik> in either case, it can't hurt to bring this up again :-) - - * IRC, #hurd, around October 2010 - - <pinotree> the only glitch is that files/dirs have the right user as - owner, but always with root group diff --git a/hurd/translator/procfs/jkoenig/discussion.mdwn b/hurd/translator/procfs/jkoenig/discussion.mdwn new file mode 100644 index 00000000..b66af7de --- /dev/null +++ b/hurd/translator/procfs/jkoenig/discussion.mdwn @@ -0,0 +1,168 @@ +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +[[!toc]] + + +# Miscellaneous + +IRC, #hurd, around September 2010 + + <youpi> jkoenig: from a quick read, your procfs implementation seems quite + simple, probably much more what I was expecting from Madhusudan (who + probably now hates you :) ) + <youpi> jkoenig: is it not possible to provide a /proc/self which points at + the client's pid? + <pinotree> (also, shouldn't /proc/version say something else than "Linux"?) + <youpi> to make linux tools work, no :/ + <youpi> kfreebsd does that too + <pinotree> really? + <youpi> yes + <youpi> (kfreebsd, not freebsd) + <pinotree> does kbsd's one print just "Linux version x.y.z" too, or + something more eg in a second line? + <pinotree> (as curiosity) + <youpi> % cat /proc/version + <youpi> Linux version 2.6.16 (des@freebsd.org) (gcc version 4.3.5) #4 Sun + Dec 18 04:30:00 CET 1977 + <pinotree> k + <giselher> I had some problems with killall5 to read the pid from /proc, Is + this now more reliable? + <youpi> I haven't tested with jkoenig's implementation + [...] + <pinotree> looks like he did 'self' too, see rootdir_entries[] in rootdir.c + <youpi> but it doesn't point at self + <antrik> youpi: there is no way to provide /proc/self, because the server + doesn't know the identity of the client + <youpi> :/ + <antrik> youpi: using the existing mechanisms, we would need another magic + lookup type + <antrik> an alternative idea I discussed with cfhammer once would be for + the client to voluntarily provide it's identity to the server... but that + would be a rather fundamental change that requires careful consideration + <antrik> also, object migration could be used, so the implementation would + be provided by the server, but the execution would happen in the + client... but that's even more involved :-) + <youpi> but we've seen how much that'd help with a lot of other stuff + <antrik> I'm not sure whether we discussed this on the ML at some point, or + only on IRC + <youpi> it "just" needs to be commited :) + <antrik> in either case, it can't hurt to bring this up again :-) + + +# root group + +IRC, #hurd, around October 2010 + + <pinotree> the only glitch is that files/dirs have the right user as + owner, but always with root group + + +# `/proc/$pid/stat` being 400 and not 444, and some more + +IRC, freenode, #hurd, 2011-03-27 + + <pochu> is there a reason for /proc/$pid/stat to be 400 and not 444 like on + Linux? + <pochu> there is an option to procfs to make it 444 like Linux + <pochu> jkoenig: ^ + <jkoenig> pochu, hi + <jkoenig> /proc/$pid/stat reveals information which is not usually + available on Hurd + <jkoenig> so I made it 400 by default to avoid leaking anything + <pochu> is there a security risk in providing that info? + <jkoenig> probably not so much, but it seemed like it's not really a + descision procfs should make + <jkoenig> I'm not sure which information we're speaking about, though, I + just remember the abstract reason. + <pochu> things like the pid, the memory, the priority, the state... + <pochu> sounds safe to expose + <jkoenig> also it's 0444 by default in "compatible" mode + <jkoenig> (which is necessary for the linux tools to work well) + <pochu> yeah I saw that :) + <pochu> my question is, should we change it to 0444 by default? if there + are no security risks and this improves compatibility, sounds like a good + thing to me + <pochu> we're already 'leaking' part of that info through e.g. ps + <jkoenig> I think /proc should be translated by /hurd/procfs --compatible + by default (I'm not sure whether it's already the case) + <jkoenig> also I'm not sure why hurd-ps is setuid root, rather than the + proc server being less paranoid, but maybe I'm missing something. + <pochu> jkoenig: it's not, at least not on Debian + <pochu> youpi: hi, what do you think about starting procfs with + --compatible by default? + <pochu> youpi: or changing /proc/$pid/stat to 0444 like on Linux + (--compatible does that among a few other things) + <youpi> I guess you need it for something? + <pochu> I'm porting libgtop :) + <youpi> k + <pochu> though I still think we should do this in procfs itself + <youpi> ymmv + <jkoenig> pochu, youpi, --compatible is also needed because mach's high + reported sysconf(_SC_CLK_TCK) makes some integers overflow (IIRC) + <youpi> agreed + <jkoenig> luckily, tools which use procfs usually try to detect the value + /proc uses rather than rely on CLK_TCK + <jkoenig> (so we can choose whatever reasonable value we want) + +IRC, freenode, #hurd, 2011-03-28 + + <antrik> jkoenig: does procfs expose any information that is not available + to everyone through the proc server?... + <antrik> also, why is --compatible not the default; or rather, why is there + even another mode? the whole point of procfs is compatibility... + <jkoenig> antrik, yes, through the <pid>/environ and (as mentionned above) + <pid>/stat files, but I've been careful to make these files readable only + to the process owner + <jkoenig> --compatible is not the default because it relaxes this paranoia + wrt. the stat file, and does not conform to the specification with regard + to clock tick counters + <antrik> what specification? + <jkoenig> the linux proc(5) manpage + <jkoenig> which says clock tick counters are in units of + 1/sysconf(_SC_CLK_TCK) + <antrik> so you are saying that there is some information that the Hurd + proc server doesn't expose to unprivileged processes, but linux /proc + does? + <jkoenig> yes + <antrik> that's odd. I wonder what the reasoning behind that could be + <antrik> but this information is available through Hurd ps? + <antrik> BTW, what exactly is _SC_CLK_TCK supposed to be? + <pinotree> jkoenig: hm, just tried with two random processes on linux + (2.6.32), and enrivon is 400 + <pinotree> (which makes sense, as you could have sensible informations eg + in http_proxy or other envvars) + <jkoenig> antrik, CLK_TCK is similar to HZ (maybe clock resolution instead + of time slices ?) + <jkoenig> sysconf(3) says "The number of clock ticks per second." + <jkoenig> antrik, I don't remember precisely what information this was, but + ps-hurd is setuid root. + <jkoenig> anyway, if you run procfs --compatible as a user and try to read + foo/1/stat, the result is an I/O error, which is the result of the proc + server denying access. + <antrik> but Linux /proc acutally uses HZ as the unit IIRC? or is + _SC_CLK_TCK=HZ on Linux?... + <jkoenig> I expect they're equal. + <jkoenig> in practice procps uses heuristics to guess what value /proc uses + (for compatibility purposes with older kernels) + <jkoenig> I don't think HZ is POSIX, while _SC_CLK_TCK is specifies as the + unit for (at least) the values returned by times() + <jkoenig> s/specifies/specified/ + <jkoenig> antrik, some the information is fetched directly from mach by + libps, and understandably, the proc server does not give the task port to + anyone who asks. + <antrik> well, as long as the information is exposed through ps, there is + no point in hiding it in procfs... + <antrik> and I'm aware of the crazy guessing in libproc... I was actually + mentoring the previous procfs implementation + <antrik> (though I never got around to look at his buggy code...) + <jkoenig> ok diff --git a/hurd/translator/tmpfs.mdwn b/hurd/translator/tmpfs.mdwn index 0179ad6c..d1476a92 100644 --- a/hurd/translator/tmpfs.mdwn +++ b/hurd/translator/tmpfs.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2007, 2008, 2009 Free Software Foundation, +[[!meta copyright="Copyright © 2007, 2008, 2009, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -6,8 +6,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] `tmpfs` is a file system server for temporary data storage without using a real (permanent) [[backing_store]]. @@ -21,9 +21,3 @@ with the additional block-level indirection layer that `ext2` (or any other disk-based file system) imposes. However, `tmpfs` is not working correctly at the moment: - -[[!inline -pages="hurd/translator/tmpfs/*" -show=0 -feeds=no -actions=yes]] diff --git a/hurd/translator/tmpfs/discussion.mdwn b/hurd/translator/tmpfs/discussion.mdwn new file mode 100644 index 00000000..d7a08491 --- /dev/null +++ b/hurd/translator/tmpfs/discussion.mdwn @@ -0,0 +1,17 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + + * [[notes_bing]] + + * [[notes_various]] + + * [[tmpfs_vs_defpager]] diff --git a/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn index ef041a23..f0eb473c 100644 --- a/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn +++ b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -8,9 +8,12 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] +[[!tag open_issue_hurd]] + \#hurd, freenode, 2010 - <slpz> humm... why does tmpfs try to use the default pager? that's a bad idea, and probably will never work correctly... + <slpz> humm... why does tmpfs try to use the default pager? that's a bad + idea, and probably will never work correctly... * slpz is thinking about old issues <slpz> tmpfs should create its own pagers, just like ext2fs, storeio... <slpz> slopez@slp-hurd:~$ settrans -a tmp /hurd/tmpfs 10M @@ -21,53 +24,99 @@ License|/fdl]]."]]"""]] <slpz> :-) <pochu> slpz: woo you fixed it? <slpz> pochu: well, it's WIP, but reading/writing works... - <slpz> I've replaced the use of default pager for the standard pager creation mechanism - <antrik> slpz: err... how is it supposed to use swap space if not using the default pager? - <antrik> slpz: or do you mean that it should act as a proxy, just allocating anonymous memory (backed by the default pager) itself? - <youpi> antrik: the kernel uses the default pager if the application pager isn't responsive enough - <slpz> antrik: it will just create memory objects and provide zerofilled pages when requested by the kernel (after a page fault) - <antrik> youpi: that makes sense I guess... but how is that relevant to the question at hand?... + <slpz> I've replaced the use of default pager for the standard pager + creation mechanism + <antrik> slpz: err... how is it supposed to use swap space if not using the + default pager? + <antrik> slpz: or do you mean that it should act as a proxy, just + allocating anonymous memory (backed by the default pager) itself? + <youpi> antrik: the kernel uses the default pager if the application pager + isn't responsive enough + <slpz> antrik: it will just create memory objects and provide zerofilled + pages when requested by the kernel (after a page fault) + <antrik> youpi: that makes sense I guess... but how is that relevant to the + question at hand?... <slpz> antrik: memory objects will contain the data by themselves - <slpz> antrik: as youpi said, when memory is scarce, GNU Mach will start paging out data from memory objects to the default pager + <slpz> antrik: as youpi said, when memory is scarce, GNU Mach will start + paging out data from memory objects to the default pager <slpz> antrik: that's the way in which pages will get into swap space <slpz> (if needed) - <youpi> the thing being that the tmpfs pager has a chance to select pages he doesn't care any more about - <antrik> slpz: well, the point is that instead of writing the pages to a backing store, tmpfs will just keep them in anonymous memory, and let the default pager write them out when there is pressure, right? - <antrik> youpi: no idea what you are talking about. apparently I still don't really understand this stuff :-( + <youpi> the thing being that the tmpfs pager has a chance to select pages + he doesn't care any more about + <antrik> slpz: well, the point is that instead of writing the pages to a + backing store, tmpfs will just keep them in anonymous memory, and let the + default pager write them out when there is pressure, right? + <antrik> youpi: no idea what you are talking about. apparently I still + don't really understand this stuff :-( <youpi> ah, but tmpfs doesn't have pages he doesn't care about, does it? - <slpz> antrik: yes, but the term "anonymous memory" could be a bit confusing. - <slpz> antrik: in GNU Mach, anonymous memory is backed by a memory object without a pager. In tmpfs, nodes will be allocated in memory objects, and the pager for those memory objects will be tmpfs itself - <antrik> slpz: hm... I thought anynymous memory is backed by memory objects created from the default pager? - <antrik> yes, I understand that tmpfs is supposed to be the pager for the objects it provides. they are obviously not anonymoust -- they have inodes in the tmpfs name space - <antrik> but my understanding so far was that when Mach returns pages to the pager, they end up in anonymous memory allocated to the pager process; and then this pager is responsible for writing them back to the actual backing store + <slpz> antrik: yes, but the term "anonymous memory" could be a bit + confusing. + <slpz> antrik: in GNU Mach, anonymous memory is backed by a memory object + without a pager. In tmpfs, nodes will be allocated in memory objects, and + the pager for those memory objects will be tmpfs itself + <antrik> slpz: hm... I thought anynymous memory is backed by memory objects + created from the default pager? + <antrik> yes, I understand that tmpfs is supposed to be the pager for the + objects it provides. they are obviously not anonymoust -- they have + inodes in the tmpfs name space + <antrik> but my understanding so far was that when Mach returns pages to + the pager, they end up in anonymous memory allocated to the pager + process; and then this pager is responsible for writing them back to the + actual backing store <antrik> am I totally off there?... - <antrik> (i.e. in my understanding the returned pages do not reside in the actual memory object the pager provides, but in an anonymous memory object) - <slpz> antrik: you're right. The trick here is, when does Mach return the pages? - <slpz> antrik: if we set the attribute "can_persist" in a memory object, Mach will keep it until object cache is full or memory is scarce + <antrik> (i.e. in my understanding the returned pages do not reside in the + actual memory object the pager provides, but in an anonymous memory + object) + <slpz> antrik: you're right. The trick here is, when does Mach return the + pages? + <slpz> antrik: if we set the attribute "can_persist" in a memory object, + Mach will keep it until object cache is full or memory is scarce <slpz> or we change the attributes so it can no longer persist, of course - <slpz> without a backing store, if Mach starts sending us pages to be written, we're in trouble - <slpz> so we must do something about it. One option, could be creating another pager and copying the contents between objects. + <slpz> without a backing store, if Mach starts sending us pages to be + written, we're in trouble + <slpz> so we must do something about it. One option, could be creating + another pager and copying the contents between objects. <antrik> another pager? not sure what you mean - <antrik> BTW, you didn't really say why we can't use the default pager for tmpfs objects :-) - <slpz> well, there're two problems when using the default pager as backing store for translators - <slpz> 1) Mach relies on it to do swapping tasks, so meddling with it is not a good idea - <slpz> 2) There're problems with seqnos when trying to work with the default pager from tasks other the kernel itself + <antrik> BTW, you didn't really say why we can't use the default pager for + tmpfs objects :-) + <slpz> well, there're two problems when using the default pager as backing + store for translators + <slpz> 1) Mach relies on it to do swapping tasks, so meddling with it is + not a good idea + <slpz> 2) There're problems with seqnos when trying to work with the + default pager from tasks other the kernel itself <slpz> (probably, the latter could be fixed) - <slpz> antrik: pager's terminology is a bit confusing. One can also say creating another memory object (though the function in libpager is "pager_create") + <slpz> antrik: pager's terminology is a bit confusing. One can also say + creating another memory object (though the function in libpager is + "pager_create") <antrik> not sure why "meddling" with it would be a problem... - <antrik> and yeah, I was vaguely aware that there is some seqno problem with tmpfs... though so far I didn't really understand what it was about :-) + <antrik> and yeah, I was vaguely aware that there is some seqno problem + with tmpfs... though so far I didn't really understand what it was about + :-) <antrik> makes sense now - <antrik> anyways, AIUI now you are trying to come up with a mechanism where the default pager is not used for tmpfs objects directly, but without making it inefficient? - <antrik> slpz: still don't understand what you mean by creating another memory object/pager... + <antrik> anyways, AIUI now you are trying to come up with a mechanism where + the default pager is not used for tmpfs objects directly, but without + making it inefficient? + <antrik> slpz: still don't understand what you mean by creating another + memory object/pager... <antrik> (and yeat, the terminology is pretty mixed up even in Mach itself) - <slpz> antrik: I meant creating another pager, in terms of calling again to libpager's pager_create - <antrik> slpz: well, I understand what "create another pager" means... I just don't understand what this other pager would be, when you would create it, and what for... + <slpz> antrik: I meant creating another pager, in terms of calling again to + libpager's pager_create + <antrik> slpz: well, I understand what "create another pager" means... I + just don't understand what this other pager would be, when you would + create it, and what for... <slpz> antrik: oh, ok, sorry - <slpz> antrik: creating another pager it's just a trick to avoid losing information when Mach's objects cache is full, and it decides to purge one of our objects - <slpz> anyway, IMHO object caching mechanism is obsolete and should be replaced + <slpz> antrik: creating another pager it's just a trick to avoid losing + information when Mach's objects cache is full, and it decides to purge + one of our objects + <slpz> anyway, IMHO object caching mechanism is obsolete and should be + replaced <slpz> I'm writting a comment to bug #28730 which says something about this <slpz> antrik: just one more thing :-) - <slpz> if you look at the code, for most time of their lives, anonymous memory objects don't have a pager + <slpz> if you look at the code, for most time of their lives, anonymous + memory objects don't have a pager <slpz> not even the default one - <slpz> only the pageout thread, when the system is running really low on memory, gives them a reference to the default pager by calling vm_object_pager_create + <slpz> only the pageout thread, when the system is running really low on + memory, gives them a reference to the default pager by calling + vm_object_pager_create <slpz> this is not really important, but worth noting ;-) |