summaryrefslogtreecommitdiff
path: root/open_issues/performance
diff options
context:
space:
mode:
authorThomas Schwinge <tschwinge@gnu.org>2012-04-17 00:16:32 +0200
committerThomas Schwinge <tschwinge@gnu.org>2012-04-17 00:16:32 +0200
commit1dc28d745d45be6764072af1da0ceda52a0c17a3 (patch)
tree1271b34236f2b2577185b11b1d2ade6fb1f0dbe8 /open_issues/performance
parenta65f14df8e3d93f71acf276fb0773d6557b9fbab (diff)
IRC.
Diffstat (limited to 'open_issues/performance')
-rw-r--r--open_issues/performance/io_system/read-ahead.mdwn116
1 files changed, 103 insertions, 13 deletions
diff --git a/open_issues/performance/io_system/read-ahead.mdwn b/open_issues/performance/io_system/read-ahead.mdwn
index b6851edd..d6a98070 100644
--- a/open_issues/performance/io_system/read-ahead.mdwn
+++ b/open_issues/performance/io_system/read-ahead.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -10,9 +10,18 @@ License|/fdl]]."]]"""]]
[[!tag open_issue_gnumach open_issue_hurd]]
-[[community/gsoc/project_ideas/disk_io_performance]]
+[[!toc]]
-IRC, #hurd, freenode, 2011-02-13:
+
+# [[community/gsoc/project_ideas/disk_io_performance]]
+
+
+# 2011-02
+
+[[Etenil]] has been working in this area.
+
+
+## IRC, freenode, #hurd, 2011-02-13
<etenil> youpi: Would libdiskfs/diskfs.h be in the right place to make
readahead functions?
@@ -28,9 +37,8 @@ IRC, #hurd, freenode, 2011-02-13:
portability
<youpi> it's not in posix indeed
----
-IRC, #hurd, freenode, 2011-02-14:
+## IRC, freenode, #hurd, 2011-02-14
<etenil> youpi: I've investigated prefetching (readahead) techniques. One
called DiskSeen seems really efficient. I can't tell yet if it's patented
@@ -51,13 +59,8 @@ IRC, #hurd, freenode, 2011-02-14:
<braunr> oh, madvise() stuff
<braunr> i could help him with that
----
-
-[[Etenil]] is now working in this area.
-
----
-IRC, freenode, #hurd, 2011-02-15
+## IRC, freenode, #hurd, 2011-02-15
<etenil> oh, I'm looking into prefetching/readahead to improve I/O
performance
@@ -281,9 +284,8 @@ IRC, freenode, #hurd, 2011-02-15
pretty good description of the necessary changes... unfortunately, these
are not publicly visible IIRC :-(
----
-IRC, freenode, #hurd, 2011-02-16
+## IRC, freenode, #hurd, 2011-02-16
<etenil> braunr: I've looked in the kernel to see where prefetching would
fit best. We talked of the VM yesterday, but I'm not sure about it. It
@@ -299,3 +301,91 @@ IRC, freenode, #hurd, 2011-02-16
the right place is the VM subsystem
[[clustered_page_faults]]
+
+
+# 2012-03
+
+
+## IRC, freenode, #hurd, 2012-03-21
+
+ <mcsim> I thought that readahead should have some heuristics, like
+ accounting size of object and last access time, but i didn't find any in
+ kam's patch. Are heuristics needed or it will be overhead for
+ microkernel?
+ <youpi> size of object and last access time are not necessarily useful to
+ take into account
+ <youpi> what would usually typically be kept is the amount of contiguous
+ data that has been read lately
+ <youpi> to know whether it's random or sequential, and how much is read
+ <youpi> (the whole size of the object does not necessarily give any
+ indication of how much of it will be read)
+ <mcsim> if big object is accessed often, performance could be increased if
+ frame that will be read ahead will be increased too.
+ <youpi> yes, but the size of the object really does not matter
+ <youpi> you can just observe how much data is read and realize that it's
+ read a lot
+ <youpi> all the more so with userland fs translators
+ <youpi> it's not because you mount a CD image that you need to read it all
+ <mcsim> youpi: indeed. this will be better. But on other hand there is
+ principle about policy and mechanism. And kernel should implement
+ mechanism, but heuristics seems to be policy. Or in this case moving
+ readahead policy to user level would be overhead?
+ <antrik> mcsim: paging policy is all in kernel anyways; so it makes perfect
+ sense to put the readahead policy there as well
+ <antrik> (of course it can be argued -- probably rightly -- that all of
+ this should go into userspace instead...)
+ <mcsim> antrik: probably defpager partly could do that. AFAIR, it is
+ possible for defpager to return more memory than was asked.
+ <mcsim> antrik: I want to outline what should be done during gsoc. First,
+ kernel should support simple readahead for specified number of pages
+ (regarding direction of access) + simple heuristic for changing frame
+ size. Also default pager could make some analysis, for instance if it has
+ many data located consequentially it could return more data then was
+ asked. For other pagers I won't do anything. Is it suitable?
+ <antrik> mcsim: I think we actually had the same discussion already with
+ KAM ;-)
+ <antrik> for clustered pageout, the kernel *has* to make the decision. I'm
+ really not convinced it makes sense to leave the decision for clustered
+ pagein to the individual pagers
+ <antrik> especially as this will actually complicate matters because a) it
+ will require work in *every* pager, and b) it will probably make handling
+ of MADVISE & friends more complex
+ <antrik> implementing readahead only for the default pager would actually
+ be rather unrewarding. I'm pretty sure it's the one giving the *least*
+ benefit
+ <antrik> it's much, much more important for ext2
+ <youpi> mcsim: maybe try to dig in the irc logs, we discussed about it with
+ neal. the current natural place would be the kernel, because it's the
+ piece that gets the traps and thus knows what happens with each
+ projection, while the backend just provides the pages without knowing
+ which projection wants it. Moving to userland would not only be overhead,
+ but quite difficult
+ <mcsim> antrik: OK, but I'm not sure that I could do it for ext2.
+ <mcsim> OK, I'll dig.
+
+
+## IRC, freenode, #hurd, 2012-04-01
+
+ <mcsim> as part of implementing of readahead project I have to add
+ interface for setting appropriate behaviour for memory range. This
+ interface than should be compatible with madvise call, that has a lot of
+ possible advises, but most part of them are specific for Linux (according
+ to man page). Should mach also support these Linux-specific values?
+ <mcsim> p.s. these Linux-specific values shouldn't affect readahead
+ algorithm.
+ <youpi> the interface shouldn't prevent from adding them some day
+ <youpi> so that we don't have to add them yet
+ <mcsim> ok. And what behaviour with value MADV_NORMAL should be look like?
+ Seems that it should be synonym to MADV_SEQUENTIAL, isn't it?
+ <youpi> no, it just means "no idea what it is"
+ <youpi> in the linux implementation, that means some given readahead value
+ <youpi> while SEQUENTIAL means twice as much
+ <youpi> and RANDOM means zero
+ <mcsim> youpi: thank you.
+ <mcsim> youpi: Than, it seems to be better that kernel interface for
+ setting behaviour will accept readahead value, without hiding it behind
+ such constants, like VM_BEHAVIOR_DEFAULT (like it was in kam's
+ patch). And than implementation of madvise will call vm_behaviour_set
+ with appropriate frame size. Is that right?
+ <youpi> question of taste, better ask on the list
+ <mcsim> ok