From 1dc28d745d45be6764072af1da0ceda52a0c17a3 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Tue, 17 Apr 2012 00:16:32 +0200 Subject: IRC. --- open_issues/performance/io_system/read-ahead.mdwn | 116 +++++++++++++++++++--- 1 file changed, 103 insertions(+), 13 deletions(-) (limited to 'open_issues/performance') diff --git a/open_issues/performance/io_system/read-ahead.mdwn b/open_issues/performance/io_system/read-ahead.mdwn index b6851edd..d6a98070 100644 --- a/open_issues/performance/io_system/read-ahead.mdwn +++ b/open_issues/performance/io_system/read-ahead.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,9 +10,18 @@ License|/fdl]]."]]"""]] [[!tag open_issue_gnumach open_issue_hurd]] -[[community/gsoc/project_ideas/disk_io_performance]] +[[!toc]] -IRC, #hurd, freenode, 2011-02-13: + +# [[community/gsoc/project_ideas/disk_io_performance]] + + +# 2011-02 + +[[Etenil]] has been working in this area. + + +## IRC, freenode, #hurd, 2011-02-13 youpi: Would libdiskfs/diskfs.h be in the right place to make readahead functions? @@ -28,9 +37,8 @@ IRC, #hurd, freenode, 2011-02-13: portability it's not in posix indeed ---- -IRC, #hurd, freenode, 2011-02-14: +## IRC, freenode, #hurd, 2011-02-14 youpi: I've investigated prefetching (readahead) techniques. One called DiskSeen seems really efficient. I can't tell yet if it's patented @@ -51,13 +59,8 @@ IRC, #hurd, freenode, 2011-02-14: oh, madvise() stuff i could help him with that ---- - -[[Etenil]] is now working in this area. - ---- -IRC, freenode, #hurd, 2011-02-15 +## IRC, freenode, #hurd, 2011-02-15 oh, I'm looking into prefetching/readahead to improve I/O performance @@ -281,9 +284,8 @@ IRC, freenode, #hurd, 2011-02-15 pretty good description of the necessary changes... unfortunately, these are not publicly visible IIRC :-( ---- -IRC, freenode, #hurd, 2011-02-16 +## IRC, freenode, #hurd, 2011-02-16 braunr: I've looked in the kernel to see where prefetching would fit best. We talked of the VM yesterday, but I'm not sure about it. It @@ -299,3 +301,91 @@ IRC, freenode, #hurd, 2011-02-16 the right place is the VM subsystem [[clustered_page_faults]] + + +# 2012-03 + + +## IRC, freenode, #hurd, 2012-03-21 + + I thought that readahead should have some heuristics, like + accounting size of object and last access time, but i didn't find any in + kam's patch. Are heuristics needed or it will be overhead for + microkernel? + size of object and last access time are not necessarily useful to + take into account + what would usually typically be kept is the amount of contiguous + data that has been read lately + to know whether it's random or sequential, and how much is read + (the whole size of the object does not necessarily give any + indication of how much of it will be read) + if big object is accessed often, performance could be increased if + frame that will be read ahead will be increased too. + yes, but the size of the object really does not matter + you can just observe how much data is read and realize that it's + read a lot + all the more so with userland fs translators + it's not because you mount a CD image that you need to read it all + youpi: indeed. this will be better. But on other hand there is + principle about policy and mechanism. And kernel should implement + mechanism, but heuristics seems to be policy. Or in this case moving + readahead policy to user level would be overhead? + mcsim: paging policy is all in kernel anyways; so it makes perfect + sense to put the readahead policy there as well + (of course it can be argued -- probably rightly -- that all of + this should go into userspace instead...) + antrik: probably defpager partly could do that. AFAIR, it is + possible for defpager to return more memory than was asked. + antrik: I want to outline what should be done during gsoc. First, + kernel should support simple readahead for specified number of pages + (regarding direction of access) + simple heuristic for changing frame + size. Also default pager could make some analysis, for instance if it has + many data located consequentially it could return more data then was + asked. For other pagers I won't do anything. Is it suitable? + mcsim: I think we actually had the same discussion already with + KAM ;-) + for clustered pageout, the kernel *has* to make the decision. I'm + really not convinced it makes sense to leave the decision for clustered + pagein to the individual pagers + especially as this will actually complicate matters because a) it + will require work in *every* pager, and b) it will probably make handling + of MADVISE & friends more complex + implementing readahead only for the default pager would actually + be rather unrewarding. I'm pretty sure it's the one giving the *least* + benefit + it's much, much more important for ext2 + mcsim: maybe try to dig in the irc logs, we discussed about it with + neal. the current natural place would be the kernel, because it's the + piece that gets the traps and thus knows what happens with each + projection, while the backend just provides the pages without knowing + which projection wants it. Moving to userland would not only be overhead, + but quite difficult + antrik: OK, but I'm not sure that I could do it for ext2. + OK, I'll dig. + + +## IRC, freenode, #hurd, 2012-04-01 + + as part of implementing of readahead project I have to add + interface for setting appropriate behaviour for memory range. This + interface than should be compatible with madvise call, that has a lot of + possible advises, but most part of them are specific for Linux (according + to man page). Should mach also support these Linux-specific values? + p.s. these Linux-specific values shouldn't affect readahead + algorithm. + the interface shouldn't prevent from adding them some day + so that we don't have to add them yet + ok. And what behaviour with value MADV_NORMAL should be look like? + Seems that it should be synonym to MADV_SEQUENTIAL, isn't it? + no, it just means "no idea what it is" + in the linux implementation, that means some given readahead value + while SEQUENTIAL means twice as much + and RANDOM means zero + youpi: thank you. + youpi: Than, it seems to be better that kernel interface for + setting behaviour will accept readahead value, without hiding it behind + such constants, like VM_BEHAVIOR_DEFAULT (like it was in kam's + patch). And than implementation of madvise will call vm_behaviour_set + with appropriate frame size. Is that right? + question of taste, better ask on the list + ok -- cgit v1.2.3