From 8050ba0991b1542f708ada5ae7eca596f6a8099d Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Tue, 26 Apr 2011 11:50:30 +0200 Subject: IRC. --- open_issues/ext2fs_page_cache_swapping_leak.mdwn | 126 +++++++++++++++++++++++ 1 file changed, 126 insertions(+) (limited to 'open_issues/ext2fs_page_cache_swapping_leak.mdwn') diff --git a/open_issues/ext2fs_page_cache_swapping_leak.mdwn b/open_issues/ext2fs_page_cache_swapping_leak.mdwn index 0ace5cd3..575196d8 100644 --- a/open_issues/ext2fs_page_cache_swapping_leak.mdwn +++ b/open_issues/ext2fs_page_cache_swapping_leak.mdwn @@ -21,3 +21,129 @@ IRC, OFTC, #debian-hurd, 2011-03-24 so the swap tends to accumulate unuseful stuff, i see yes the disk content, basicallyt :) + +IRC, freenode, #hurd, 2011-04-18 + + damn, a cp -a simply gobbles down swap space... + really ? + that's weird + why would a copy use so much anonymous memory ? + unless the external pager is so busy that the kernel falls back to + its default pager + that's what I suggested some time ago + maybe this case should be traced in the kernel + a simple message in the kernel buffer to warn that this condition + happened may help + I'm seeing swap space being kept used on buildds for no real reason + except possibly backing ext2fs pages + that could help, yes + youpi: I think it was actually slpz who suggested that... + I think we're generally missing feedback from memory behavior + youpi: do you think andrei's kernel instrumentation work might be + helpful with analyzing such things? + antrik: I think I suggested it too, but never mind + antrik: no, because it's not a trace of events that you want + some specific events would be useful + but then we don't really need a whole framework for that + apt-get upgrade eats swap too + the upgrade itself, or the computation of the ugprade? + apt is a memory eater nowadays + installing the packages + seems to have stabilized though after a while... + so perhaps it's not a leak in this case + ideally we should have a way to know what was put in the swap + how would you represent what's in the swap ? + the apt-get process has 46M of virtual memory above the 128 M + baseline + mostly libraries i guess + are trheads stacks 8 MiB like on Linux ? + braunr: at least knowing how much of each process is in the swap + braunr: 2MiB + ok + vminfo could also report which parts of the address space are in + the swap + youpi: would be nice to have some simple utility reporting how + much of a process' address space is anonymous + (in fact, I wonder why it's not reported by standard tools such as + ps or top... this shouldn't be too difficult I would think?) + it would be much more useful information than the total virt size, + which includes rather meaningless disk and device mappings... + agreed + well + there are tools like pmap for this + unfortunately, it's difficult in mach to know what backs a + non-anonymous mapping + pagers should be able to name their mappings + that'd be helpful for debugging yes + there is almost no overhead in doing that, and it would be very + useful + and could lead to /proc/pid/maps + yes + isn't there a maps already ? + nope + ok + (probably not very useful without the names) + ithought i remembered maps without names, and guessed it might + have been on the hurd for that reason + but i'm not sure + there's the vminfo command, yes + 14:06 < youpi> braunr: at least knowing how much of each process + is in the swap + wouldn't it be clearer to do it the other way around ? + like a swapinfo tool indicating what it contains ? + sure, but it's a lot more difficult + really ? + why ? + because you have to traverse all the mappings + etc + (in all processes, I mean) + and you have to name what is waht + there are other ways + the swap is a central structure + while simply introducing the swap % in vminfo + for a given process you know what is what + right + and doing that introduction is probably very simple + that's a good point + top-down is effectively easier than bottom-up resolution in Mach + VM + hm... the memory use caused by cp doesn't seem to be reflected in + the virtual size of any particular process + ghost memory + what's cp vmsize at the time of the problem ? + it's at 134 M right now... so considering the 128 M baseline, + nothing worth speaking of + right + maybe a copy map during I/O + but I don't know Mach copy maps in detail, as they have been + eliminated from UVM + BTW, the memory eatup happens even before swap comes into + play... swapping seems to be a result of the problem, not the cause + what do you mean ? + I thought swapping was the issue + you mean RAM is full before swapping ? + well, I don't know what the actual problem is... I just don't + understand why the memory use increases without any particular process + seeing an increase in size + the "free" size in vmstat decreses + once it's eatun up, swap space use increases + well it doesn't change much of it + the anonymous memory pager will use RAM before resorting to the + external default-pager + I would suspect normal block caching... but then, shouldn't this + show up in the memory info of the ext2 process? + although, again, I'm not sure of the behaviour of the anonymous + memory pager + antrik: I don't know how block caching behaves + BTW, is it a know problem that doing ^C on a "cp -a" seems to hang + the whole system?... + (the whole hurd instance that is... the other instance is not + affected) + not that I know of + seems like a deadlock in the anonymous memory handling + (and I've never seen that) + happens both in my main system (using ancient hurd/libc) and in my + subhurd (recently upgraded to current stuff) + this make testing this stuff quite a lot harder... [sigh] + any suggestions how to debug this hang? + antrik: no :/ -- cgit v1.2.3