open_issues/performance/io_system/read-ahead: Some more IRC.

1 files changed, 230 insertions, 0 deletions

diff --git a/open_issues/performance/io_system/read-ahead.mdwn b/open_issues/performance/io_system/read-ahead.mdwn
index c3a0c1bb..241cda41 100644
--- a/open_issues/performance/io_system/read-ahead.mdwn
+++ b/open_issues/performance/io_system/read-ahead.mdwn
@@ -26,6 +26,8 @@ IRC, #hurd, freenode, 2011-02-13:
       portability
     <youpi> it's not in posix indeed
 
+---
+
 IRC, #hurd, freenode, 2011-02-14:
 
     <etenil> youpi: I've investigated prefetching (readahead) techniques. One
@@ -47,4 +49,232 @@ IRC, #hurd, freenode, 2011-02-14:
     <braunr> oh, madvise() stuff
     <braunr> i could help him with that
 
+---
+
 [[Etenil]] is now working in this area.
+
+---
+
+IRC, freenode, #hurd, 2011-02-15
+
+    <etenil> oh, I'm looking into prefetching/readahead to improve I/O
+      performance
+    <braunr> etenil: ok
+    <braunr> etenil: that's actually a VM improvement, like samuel told you
+    <etenil> yes
+    <braunr> a true I/O improvement would be I/O scheduling
+    <braunr> and how to implement it in a hurdish way
+    <braunr> (or if it makes sense to have it in the kernel)
+    <etenil> that's what I've been wondering too lately
+    <braunr> concerning the VM, you should look at madvise()
+    <etenil> my understanding is that Mach considers devices without really
+      knowing what they are
+    <braunr> that's roughly the interface used both at the syscall() and the
+      kernel levels in BSD, which made it in many other unix systems
+    <etenil> whereas I/O optimisations are often hard disk drives specific
+    <braunr> that's true for almost any kernel
+    <braunr> the device knowledge is at the driver level
+    <etenil> yes
+    <braunr> (here, I separate kernels from their drivers ofc)
+    <etenil> but Mach also contains some drivers, so I'm going through the code
+      to find the apropriate place for these improvements
+    <braunr> you shouldn't tough the drivers at all
+    <braunr> touch
+    <etenil> true, but I need to understand how it works before fiddling around
+    <braunr> hm
+    <braunr> not at all
+    <braunr> the VM improvement is about pagein clustering
+    <braunr> you don't need to know how pages are fetched
+    <braunr> well, not at the device level
+    <braunr> you need to know about the protocol between the kernel and
+      external pagers
+    <etenil> ok
+    <braunr> you could also implement pageout clustering
+    <etenil> if I understand you well, you say that what I'd need to do is a
+      queuing system for the paging in the VM?
+    <braunr> no
+    <braunr> i'm saying that, when a page fault occurs, the kernel should
+      (depending on what was configured through madvise()) transfer pages in
+      multiple blocks rather than one at a time
+    <braunr> communication with external pagers is already async, made through
+      regular ports
+    <braunr> which already implement message queuing
+    <braunr> you would just need to make the mapped regions larger
+    <braunr> and maybe change the interface so that this size is passed
+    <etenil> mmh
+    <braunr> (also don't forget that page clustering can include pages *before*
+      the page which caused the fault, so you may have to pass the start of
+      that region too)
+    <etenil> I'm not sure I understand the page fault thing
+    <etenil> is it like a segmentation error?
+    <etenil> I can't find a clear definition in Mach's manual
+    <braunr> ah
+    <braunr> it's a fundamental operating system concept
+    <braunr> http://en.wikipedia.org/wiki/Page_fault
+    <etenil> ah ok
+    <etenil> I understand now
+    <etenil> so what's currently happening is that when a page fault occurs,
+      Mach is transfering pages one at a time and wastes time 
+    <braunr> sometimes, transferring just one page is what you want
+    <braunr> it depends on the application, which is why there is madvise()
+    <braunr> our rootfs, on the other hand, would benefit much from such an
+      improvement
+    <braunr> in UVM, this optimization is account for around 10% global
+      performance improvement
+    <braunr> accounted*
+    <etenil> not bad
+    <braunr> well, with an improved page cache, I'm sure I/O would matter less
+      on systems with more RAM
+    <braunr> (and another improvement would make mach support more RAM in the
+      first place !)
+    <braunr> an I/O scheduler outside the kernel would be a very good project
+      IMO
+    <braunr> in e.g. libstore/storeio
+    <etenil> yes
+    <braunr> but as i stated in my thesis, a resource scheduler should be as
+      close to its resource as it can
+    <braunr> and since mach can host several operating systems, I/O schedulers
+      should reside near device drivers
+    <braunr> and since current drivers are in the kernel, it makes sens to have
+      it in the kernel too
+    <braunr> so there must be some discussion about this
+    <etenil> doesn't this mean that we'll have to get some optimizations in
+      Mach and have the same outside of Mach for translators that access the
+      hardware directly?
+    <braunr> etenil: why ?
+    <etenil> well as you said Mach contains some drivers, but in principle, it
+      shouldn't, translators should do disk access etc, yes?
+    <braunr> etenil: ok
+    <braunr> etenil: so ?
+    <etenil> well, let's say if one were to introduce SATA support in Hurd,
+      nothing would stop him/her to do so with a translator rather than in Mach
+    <braunr> you should avoid the term translator here
+    <braunr> it's really hurd specific
+    <braunr> let's just say a user space task would be responsible for that
+      job, maybe multiple instances of it, yes
+    <etenil> ok, so in this case, let's say we have some I/O optimization
+      techniques like readahead and I/O scheduling within Mach, would these
+      also apply to the user-space task, or would they need to be
+      reimplemented?
+    <braunr> if you have user space drivers, there is no point having I/O
+      scheduling in the kernel
+    <etenil> but we also have drivers within the kernel
+    <braunr> what you call readahead, and I call pagein/out clustering, is
+      really tied to the VM, so it must be in Mach in any case
+    <braunr> well
+    <braunr> you either have one or the other
+    <braunr> currently we have them in the kernel
+    <braunr> if we switch to DDE, we should have all of them outside
+    <braunr> that's why such things must be discussed
+    <etenil> ok so if I follow you, then future I/O device drivers will need to
+      be implemented for Mach
+    <braunr> currently, yes
+    <braunr> but preferrably, someone should continue the work that has been
+      done on DDe so that drivers are outside the kernel
+    <etenil> so for the time being, I will try and improve I/O in Mach, and if
+      drivers ever get out, then some of the I/O optimizations will need to be
+      moved out of Mach
+    <braunr> let me remind you one of the things i said
+    <braunr> i said I/O scheduling should be close to their resource, because
+      we can host several operating systems
+    <braunr> now, the Hurd is the only system running on top of Mach
+    <braunr> so we could just have I/O scheduling outside too
+    <braunr> then you should consider neighbor hurds
+    <braunr> which can use different partitions, but on the same device
+    <braunr> currently, partitions are managed in the kernel, so file systems
+      (and storeio) can't make good scheduling decisions if it remains that way
+    <braunr> but that can change too
+    <braunr> a single storeio representing a whole disk could be shared by
+      several hurd instances, just as if it were a high level driver
+    <braunr> then you could implement I/O scheduling in storeio, which would be
+      an improvement for the current implementation, and reusable for future
+      work
+    <etenil> yes, that was my first instinct
+    <braunr> and you would be mostly free of the kernel internals that make it
+      a nightmare
+    <etenil> but youpi said that it would be better to modify Mach instead
+    <braunr> he mentioned the page clustering thing
+    <braunr> not I/O scheduling
+    <braunr> theseare really two different things
+    <etenil> ok
+    <braunr> you *can't* implement page clustering outside Mach because Mach
+      implements virtual memory
+    <braunr> both policies and mechanisms
+    <etenil> well, I'd rather think of one thing at a time if that's alright
+    <etenil> so what I'm busy with right now is setting up clustered page-in
+    <etenil> which need to be done within Mach
+    <braunr> keep clustered page-outs in mind too
+    <braunr> although there are more constraints on those
+    <etenil> yes
+    <etenil> I've looked up madvise(). There's a lot of documentation about it
+      in Linux but I couldn't find references to it in Mach (nor Hurd), does it
+      exist?
+    <braunr> well, if it did, you wouldn't be caring about clustered page
+      transfers, would you ?
+    <braunr> be careful about linux specific stuff
+    <etenil> I suppose not
+    <braunr> you should implement at least posix options, and if there are
+      more, consider the bsd variants
+    <braunr> (the Mach VM is the ancestor of all modern BSD VMs)
+    <etenil> madvise() seems to be posix
+    <braunr> there are system specific extensions
+    <braunr> be careful
+    <braunr> CONFORMING TO POSIX.1b.   POSIX.1-2001 describes posix_madvise(3)
+      with constants POSIX_MADV_NORMAL, etc., with a behav‐ ior close to that
+      described here.  There is a similar posix_fadvise(2) for file access.
+    <braunr> MADV_REMOVE, MADV_DONTFORK, MADV_DOFORK, MADV_HWPOISON,
+      MADV_MERGEABLE, and MADV_UNMERGEABLE  are  Linux- specific.
+    <etenil> I was about to post these
+    <etenil> ok, so basically madvise() allows tasks etc. to specify a usage
+      type for a chunk of memory, then I could apply the relevant I/O
+      optimization based on this
+    <braunr> that's it
+    <etenil> cool, then I don't need to worry about knowing what the I/O is
+      operating on, I just need to apply the optimizations as advised
+    <etenil> that's convenient
+    <etenil> ok I'll start working on this tonight
+    <etenil> making a basic readahead shouldn't be too hard
+    <braunr> readahead is a misleading name
+    <etenil> is pagein better?
+    <braunr> applies to too many things, doesn't include the case where
+      previous elements could be prefetched
+    <braunr> clustered page transfers is what i would use
+    <braunr> page prefetching maybe
+    <etenil> ok
+    <braunr> you should stick to something that's already used in the
+      literature since you're not inventing something new
+    <etenil> yes I've read a paper about prefetching
+    <etenil> ok
+    <etenil> thanks for your help braunr
+    <braunr> sure
+    <braunr> you're welcome
+    <antrik> braunr: madvise() is really the least important part of the
+      picture...
+    <antrik> very few applications actually use it. but pretty much all
+      applications will profit from clustered paging
+    <antrik> I would consider madvise() an optional goody, not an integral part
+      of the implementation
+    <antrik> etenil: you can find some stuff about KAM's work on
+      http://www.gnu.org/software/hurd/user/kam.html
+    <antrik> not much specific though
+    <etenil> thanks
+    <antrik> I don't remember exactly, but I guess there is also some
+      information on the mailing list. check the archives for last summer
+    <antrik> look for Karim Allah Ahmed
+    <etenil> antrik: I disagree, madvise gives me a good starting point, even
+      if eventually the optimisations should run even without it
+    <antrik> the code he wrote should be available from Google's summer of code
+      page somewhere...
+    <braunr> antrik: right, i was mentioning madvise() because the kernel (VM)
+      interface is pretty similar to the syscall
+    <braunr> but even a default policy would be nice
+    <antrik> etenil: I fear that many bits were discussed only on IRC... so
+      you'd better look through the IRC logs from last April onwards...
+    <etenil> ok
+
+    <etenil> at the beginning I thought I could put that into libstore
+    <etenil> which would have been fine
+
+    <antrik> BTW, I remembered now that KAM's GSoC application should have a
+      pretty good description of the necessary changes... unfortunately, these
+      are not publicly visible IIRC :-(