[[!meta copyright="Copyright © 2011, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] [[!tag open_issue_documentation open_issue_gnumach]] # IRC, freenode, #hurd, 2011-01-06 hm, odd... vmstat tells me that ~500 MiB of RAM are in use; but the sum of all RSS is <300 MiB... what's the rest? kernel memory ? the zone allocator maybe or the page cache simply braunr: which page cache? AIUI, caches are implemented by the individual filesystem servers -- in which case any memory used by them should show up in RSS also, gnumach is listed among other tasks, so I'd assume the kernel memery also to be accounted for antrik: no, the kernel maintains a page cache, very similar to what is done in Linux, and almost the same as in FreeBSD the file system servers are just backing stores the RSS for the gnumach tasks only includes kernel memory I don't think the page cache is accounted for because it's not really kernel memory, it's a cache of user space memory apparently my understanding of Mach paging is still (or again?) rather incomplete :-( BTW, is there any way to find out how much anonymous memory a process is using? the "virtual" includes discardable mappings, and is thus not very helpful... (that applies to Linux as well though) can you provide an example of the output of vmstat please ? I don't have a Hurd VM near me olaf@alien:~$ vmstat pagesize: 4K size: 501M free: 6.39M active: 155M inactive: 310M wired: 29.4M zero filled: 15.3G reactivated: 708M pageins: 3.43G pageouts: 1.55G page faults: 26844574 cow faults: 3736174 memobj hit ratio: 92% swap size: 733M swap free: 432M interesting... closing a single screen window temporarily raises the "free" value by almost 10 MB I guess bash is rather hungry nowadays ;-) antrik: I guess the only way is using pmap or looking into /proc//maps but it won't give you the amount of physical memory used by anonymous mappings nah, I don't even want that... just like to know how much memory (RAM+swap) a process is really using antrik: then the RSS field is what you want OTOH, anonymous doesn't include program code or other actively used mappings... so not very useful either nah, RSS doesn't count anything that is in swap well don't you have a SWAP column ? hm i guess not antrik: why do you say it doesn't include other actively used mappings ? antrik: and the inclusion of program code also depends on the implementation of the ELF handler I don't know how the hurd does that, but some ELF loaders use anonymous memory for the execution view well, if a program maps a data file, and regularily accesses parts of the file, they won't occupy physical RAM all the time (and show up in RSS), but they are not anonymous mappings. similar to program code then this anonymous memory is shared by all processes using that code oh, interesting is it really a completely distinct mapping, rather than just COW? the first is others are COW so if a program loads 200 MB of libraries, they are all read in on startup, and occupy RAM or swap subsequently, even if most of the code is never actually run?... library code should be backed by the library file on disk, not be swap depends on the implementation I guess most use the file system backend but in the Hurd, ext2fs.static and ld.so.1 use anonymous memory (that's the case for another reason, still, I don't think the report in top/ps clearly indicates that fact) braunr: yeah for bootstrapping issues, makes sense it may also depends on the pic/pie options used when building libraries # IRC, freenode, #hurd, 2011-07-24 < braunr> the panic is probably due to memory shortage < braunr> so as antrik suggested, use more swap < antrik> gg0: you could run "vmstat 1" in another terminal to watch memory usage < antrik> that way we will know for sure whether it's related < braunr> antrik: it's trickier than that < braunr> it depends if the zones used are pageable < antrik> braunr: well, if it's a zone map exhaustion, then the swap size won't change anything?... < braunr> antrik: in this case no, but if the zone is pageable and the pager (backing anonymous memory) refuses to create memory because it estimates it's full (all swap space is reserved), it will fail to < braunr> too < braunr> but i don't think there are much pageable zones in the kernel < antrik> yes, but in that case we can see the exhaustion in vmstat :-) < braunr> many* < braunr> i'm not sure < braunr> reserved swap space doesn't mean it's used < braunr> that's one of the major changes in freebsd 4 or 5 i was mentioning < antrik> if it's reserved, it wouldn't show up as "free", would it?... < braunr> (btw, it's also what makes anonymous memory merging so hard) < braunr> yes it would < braunr> well, it could, i'm not sure < braunr> anonymous memory is considered as a file < braunr> one big file filled with zeroes, which is the swap partition < braunr> when you allocate pageable anonymous memory, a part of this "file" is reserved < braunr> but i don't know if the reported number if the reserved (allocated) space, or used (actually containing data) < braunr> is* < braunr> i also suspect wired allocations can fail because of a full swap (because the kernel is unable to make free pages) < braunr> in this case vmstat will show it < antrik> what does it matter whether there is data there or not? if it's reserved, it's not free. if it behaves differently, I'd consider that a serious bug < braunr> maybe the original developers intended to monitor its actual usage < braunr> antrik: i've just checked how the free count gets updated, and it looks like it is on both seqnos_memory_object_data_initialize and seqnos_memory_object_data_write < braunr> antrik: so i guess reserved memory is accounted for # IRC, freenode, #hurd, 2013-01-12 darnassus linking clang: 600 MiB swap in use and 22 MiB RAM free, of 2 GiB. But ps shows a RSS of just 100 MiB, huh? Getting "better": near the end of the link, nearly 1 GiB swap in use, and 200 KiB (!) RAM free. can hurd have more than 1GB of ram ? And then it completed; 75 MiB swap in use, and 1.2 GiB RAM free. tschwinge: unless i'm mistaken, mach uses the legacy "swapping" bsd mechanism tschwinge: i.e. when it swaps a process, it swaps all of it tschwinge: the rest is probably one big anonymous vm object containing the process space cached objects aren't currently well accounted (well, since youpi got my page cache patches in, they are, but procfs isn't yet modified to report them) tschwinge: right, i'm currently looking at the machine and it doesn't add up, i suppoe there are some big files still in the cache ah, git packed objects :p and a few llvm .a/.so/executable files too and since they're probably targets, they're built last, which explains why they're retained in the cache for a while [[microkernel/mach/message/msgh_id]] (why on *that* page?).