From 49a086299e047b18280457b654790ef4a2e5abfa Mon Sep 17 00:00:00 2001 From: Samuel Thibault Date: Wed, 18 Feb 2015 00:58:35 +0100 Subject: Revert "rename open_issues.mdwn to service_solahart_jakarta_selatan__082122541663.mdwn" This reverts commit 95878586ec7611791f4001a4ee17abf943fae3c1. --- open_issues/mach_tasks_memory_usage.mdwn | 175 +++++++++++++++++++++++++++++++ 1 file changed, 175 insertions(+) create mode 100644 open_issues/mach_tasks_memory_usage.mdwn (limited to 'open_issues/mach_tasks_memory_usage.mdwn') diff --git a/open_issues/mach_tasks_memory_usage.mdwn b/open_issues/mach_tasks_memory_usage.mdwn new file mode 100644 index 00000000..7a7a77ce --- /dev/null +++ b/open_issues/mach_tasks_memory_usage.mdwn @@ -0,0 +1,175 @@ +[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_documentation open_issue_gnumach]] + + +# IRC, freenode, #hurd, 2011-01-06 + + hm, odd... vmstat tells me that ~500 MiB of RAM are in use; but + the sum of all RSS is <300 MiB... what's the rest? + kernel memory ? + the zone allocator maybe + or the page cache simply + braunr: which page cache? AIUI, caches are implemented by the + individual filesystem servers -- in which case any memory used by them + should show up in RSS + also, gnumach is listed among other tasks, so I'd assume the + kernel memery also to be accounted for + antrik: no, the kernel maintains a page cache, very similar to + what is done in Linux, and almost the same as in FreeBSD + the file system servers are just backing stores + the RSS for the gnumach tasks only includes kernel memory + I don't think the page cache is accounted for + because it's not really kernel memory, it's a cache of user space + memory + apparently my understanding of Mach paging is still (or again?) + rather incomplete :-( + BTW, is there any way to find out how much anonymous memory a + process is using? the "virtual" includes discardable mappings, and is + thus not very helpful... + (that applies to Linux as well though) + can you provide an example of the output of vmstat please ? + I don't have a Hurd VM near me + olaf@alien:~$ vmstat + pagesize: 4K + size: 501M + free: 6.39M + active: 155M + inactive: 310M + wired: 29.4M + zero filled: 15.3G + reactivated: 708M + pageins: 3.43G + pageouts: 1.55G + page faults: 26844574 + cow faults: 3736174 + memobj hit ratio: 92% + swap size: 733M + swap free: 432M + interesting... closing a single screen window temporarily raises + the "free" value by almost 10 MB + I guess bash is rather hungry nowadays ;-) + antrik: I guess the only way is using pmap or looking into + /proc//maps + but it won't give you the amount of physical memory used by + anonymous mappings + nah, I don't even want that... just like to know how much memory + (RAM+swap) a process is really using + antrik: then the RSS field is what you want + OTOH, anonymous doesn't include program code or other actively + used mappings... so not very useful either + nah, RSS doesn't count anything that is in swap + well + don't you have a SWAP column ? + hm + i guess not + antrik: why do you say it doesn't include other actively used + mappings ? + antrik: and the inclusion of program code also depends on the + implementation of the ELF handler + I don't know how the hurd does that, but some ELF loaders use + anonymous memory for the execution view + well, if a program maps a data file, and regularily accesses parts + of the file, they won't occupy physical RAM all the time (and show up in + RSS), but they are not anonymous mappings. similar to program code + then this anonymous memory is shared by all processes using that + code + oh, interesting + is it really a completely distinct mapping, rather than just COW? + the first is + others are COW + so if a program loads 200 MB of libraries, they are all read in on + startup, and occupy RAM or swap subsequently, even if most of the code is + never actually run?... + library code should be backed by the library file on disk, not be + swap + depends on the implementation + I guess most use the file system backend + but in the Hurd, ext2fs.static and ld.so.1 use anonymous memory + (that's the case for another reason, still, I don't think the + report in top/ps clearly indicates that fact) + braunr: yeah for bootstrapping issues, makes sense + it may also depends on the pic/pie options used when building + libraries + + +# IRC, freenode, #hurd, 2011-07-24 + + < braunr> the panic is probably due to memory shortage + < braunr> so as antrik suggested, use more swap + < antrik> gg0: you could run "vmstat 1" in another terminal to watch memory + usage + < antrik> that way we will know for sure whether it's related + < braunr> antrik: it's trickier than that + < braunr> it depends if the zones used are pageable + < antrik> braunr: well, if it's a zone map exhaustion, then the swap size + won't change anything?... + < braunr> antrik: in this case no, but if the zone is pageable and the + pager (backing anonymous memory) refuses to create memory because it + estimates it's full (all swap space is reserved), it will fail to + < braunr> too + < braunr> but i don't think there are much pageable zones in the kernel + < antrik> yes, but in that case we can see the exhaustion in vmstat :-) + < braunr> many* + < braunr> i'm not sure + < braunr> reserved swap space doesn't mean it's used + < braunr> that's one of the major changes in freebsd 4 or 5 i was + mentioning + < antrik> if it's reserved, it wouldn't show up as "free", would it?... + < braunr> (btw, it's also what makes anonymous memory merging so hard) + < braunr> yes it would + < braunr> well, it could, i'm not sure + < braunr> anonymous memory is considered as a file + < braunr> one big file filled with zeroes, which is the swap partition + < braunr> when you allocate pageable anonymous memory, a part of this + "file" is reserved + < braunr> but i don't know if the reported number if the reserved + (allocated) space, or used (actually containing data) + < braunr> is* + < braunr> i also suspect wired allocations can fail because of a full swap + (because the kernel is unable to make free pages) + < braunr> in this case vmstat will show it + < antrik> what does it matter whether there is data there or not? if it's + reserved, it's not free. if it behaves differently, I'd consider that a + serious bug + < braunr> maybe the original developers intended to monitor its actual + usage + < braunr> antrik: i've just checked how the free count gets updated, and it + looks like it is on both seqnos_memory_object_data_initialize and + seqnos_memory_object_data_write + < braunr> antrik: so i guess reserved memory is accounted for + + +# IRC, freenode, #hurd, 2013-01-12 + + darnassus linking clang: 600 MiB swap in use and 22 MiB RAM + free, of 2 GiB. But ps shows a RSS of just 100 MiB, huh? + Getting "better": near the end of the link, nearly 1 GiB swap + in use, and 200 KiB (!) RAM free. + can hurd have more than 1GB of ram ? + And then it completed; 75 MiB swap in use, and 1.2 GiB RAM + free. + tschwinge: unless i'm mistaken, mach uses the legacy "swapping" + bsd mechanism + tschwinge: i.e. when it swaps a process, it swaps all of it + tschwinge: the rest is probably one big anonymous vm object + containing the process space + cached objects aren't currently well accounted + (well, since youpi got my page cache patches in, they are, but + procfs isn't yet modified to report them) + tschwinge: right, i'm currently looking at the machine and it + doesn't add up, i suppoe there are some big files still in the cache + ah, git packed objects :p + and a few llvm .a/.so/executable files too + and since they're probably targets, they're built last, which + explains why they're retained in the cache for a while + +[[microkernel/mach/message/msgh_id]] (why on *that* page?). -- cgit v1.2.3