IRC.

1 files changed, 265 insertions, 1 deletions

diff --git a/hurd/translator/tmpfs/discussion.mdwn b/hurd/translator/tmpfs/discussion.mdwn
index 486206e3..0409f046 100644
--- a/hurd/translator/tmpfs/discussion.mdwn
+++ b/hurd/translator/tmpfs/discussion.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]]
 
 [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
 id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -19,3 +19,267 @@ License|/fdl]]."]]"""]]
   * [[!GNU_Savannah_bug 26751]]
 
   * [[!GNU_Savannah_bug 32755]]
+
+
+# [[Maksym_Planeta]]
+
+## IRC, freenode, #hurd, 2011-11-29
+
+    <mcsim> Hello. In seqno_memory_object_data_request I call
+      memory_object_data_supply and supply one zero filled page, but seems that
+      kernel ignores this call because this page stays filled in specified
+      memory object. In what cases kernel may ignore this call? It is written
+      in documentation that "kernel prohibits the overwriting of live data
+      pages". But when I called memory_object_lock_request on this page with
+      should flush and MEMORY_OBJECT_RETURN_ALL nothing change
+    <braunr> what are you trying to do ?
+    <mcsim> I think that memory object holds wrong data, so I'm trying to
+      replace them. This happens when file is truncated, so I should notify
+      memory object that there is no some data. But since gnumach works only
+      with sizes that are multiple of vm_page_size, I should manually correct
+      last page for case when file size isn't multiple of vm_page_size. This is
+      needed for case when file grows again and that tail of last page, which
+      wasn't part of file should be filled wit
+    <mcsim> I've put some printf's in kernel and it seems that page that holds
+      data which I want replace both absent and busy:
+    <mcsim> m = vm_page_lookup(object,offset);
+    <mcsim> ...
+    <mcsim> if (m->absent && m->busy) { <-- Condition is true
+    <mcsim> in vm/memory_object.c:169
+    <slpz> mcsim: Receiving m_o_data_request means there's no page in the
+      memory object at that offset, so m_o_data_supply should work
+    <slpz> are you sure that page is not being installed into the memory
+      object?
+    <braunr> it seems normal it's both absent and busy
+    <braunr> absent because, as sergio said, the page is missing, and busy
+      because the kernel starts a transfer for its content
+    <braunr> i don't understand how you determine the kernel ignores your
+      data_supply
+    <braunr> "because this page stays filled in specified memory object"
+    <braunr> please explain this with more detail
+    <slpz> mcsim: anyway, when truncating a file to a non page-aligned length,
+      you can just zero fill the rest of the page by mapping the object and
+      writing to it with memset/bzero 
+    <braunr> (avoid bzero, it's obsolete)
+    <mcsim> slpz: I'll try try it now.
+    <braunr> slpz: i think that's what he's trying to do
+    <mcsim> I don't vm_map it
+    <braunr> how do you zero it then ?
+    <braunr> "I call memory_object_data_supply and supply one zero filled page"
+    <mcsim> First I call mo_lock_request and ask to return this page, than I
+      memset tail and try to mo_data_supply
+    <mcsim> I use this function when I try to replace kr =
+      memory_object_data_supply(reply_to, offset, addr, vm_page_size, FALSE,
+      VM_PROT_NONE, FALSE, MACH_PORT_NULL);
+    <mcsim> where addr points to new data, offset points to old data in
+      object. and reply_to is memory_control which I get as parameter in
+      mo_data_request
+    <braunr> why would you want to vm_map it then ?
+    <mcsim> because mo_data_supply doesn't work.
+    <braunr> mcsim: i still don't see why you want to vm_map
+    <mcsim> I just want to try it.
+    <braunr> but what do you think will happen ?
+    <mcsim> But seems that it doesn't work too, because I can't vm_map
+      memory_object from memory_manager of this object.
+
+
+## IRC, freenode, #hurd, 2012-01-05
+
+    <mcsim> Seems tmpfs works now. The code really needs cleaning, but the main
+      is that it works. So in nearest future it will be ready for merging to
+      master branch. BTW, anyone knows good tutorial about refactoring using
+      git (I have a lot of pointless commits and I want to gather and scatter
+      them to sensible ones).
+    <antrik> I wonder whether he actually got the "proper" tmpfs with the
+      defaul pager working? or only the hack with a private pager?
+    <mcsim> antrik: with default pager
+    <antrik> mcsim: wow, that's great :-)
+    <antrik> how did you fix it?
+    <mcsim> antrik: The main code I wrote before December, so I forgot some of
+      what exactly I were doing. So I have to look through my code :)
+    <mcsim> antrik: old defpager was using old functions like m_o_data_write
+      instead of m_o_data_return etc. I changed it, mostly because of my
+      misunderstanding. But I hope that this is not a problem.
+
+
+## IRC, freenode, #hurd, 2012-01-18
+
+    <antrik> mcsim: did you publish your in-progress work?
+    <mcsim> there is a branch with working tmpfs in git repository:
+      http://git.savannah.gnu.org/cgit/hurd/hurd.git/log/?h=mplaneta/tmpfs/defpager
+    <jd823592> sorry for interrupting the meeting but i wonder what is a
+      lazyfs?
+    <mcsim> jd823592: lazyfs is tmpfs which uses own pager
+    <antrik> mcsim: ah, nice :-)
+    <antrik> BTW, what exactly did you need to fix to make it work?
+    <mcsim> most fixes wore in defpager in default_pager_object_set_size. Also,
+      as i said earlier, I switched to new functions (m_o_data_return instead
+      of m_o_data_write and so on). I said that this was mostly because of my
+      misunderstanding, but it turned out that new function provide work with
+      precious attribute of page.
+    <mcsim> Also there were some small errors like this:
+    <mcsim>  	  pager->map = (dp_map_t) kalloc (PAGEMAP_SIZE (new_size));
+    <mcsim> 	  memcpy (pager->map, old_mapptr, PAGEMAP_SIZE (old_size));
+    <mcsim> where in second line should be new_size too
+    <mcsim> I removed all warnings in compiling defpager (and this helped to
+      find an error).
+    <antrik> great work :-)
+    <jd823592> tmpfs is nice thing to have :), are there other recent
+      improvements that were not yet published in previous moth?
+    <mcsim> BTW, i measured tmpfs speed in it is up to 6 times faster than
+      ramdisk+ext2fs
+    <antrik> mcsim: whow, that's quite a difference... didn't expect that
+
+
+## IRC, freenode, #hurd, 2012-01-24
+
+    <mcsim> braunr: I'm just wondering is there any messages before hurd
+      breaks. I have quite strange message: memory_object_data_request(0x0,
+      0x0, 0xf000, 0x1000, 0x1) failed, 10000003
+    <braunr> hm i don't think so
+    <braunr> usually it either freezes completely, or it panics because of an
+      exhausted resource
+    <mcsim> where first and second 0x0 are pager and pager_request for memory
+      object in vm_fault_page from gnumach/vm_fault.c
+    <braunr> if you're using the code you're currently working on (which i
+      assume), then look for a bug there first
+    <tschwinge> mcsim: Maybe you're running out of swap?
+    <mcsim> tschwinge: no
+    <braunr> also, translate the error code
+    <mcsim> AFAIR that's MACH_INVALID_DEST
+    <braunr> and what does it mean in this situation ?
+    <mcsim> I've run fsx as long as possible several times. It runs quite long
+      but it breaks in different ways.
+    <mcsim> MACH_SEND_INVALID_DEST
+    <mcsim> this means that kernel tries to call rpc with pager 0x0
+    <mcsim> this is invalid destiantion
+    <braunr> null port
+    <braunr> ok
+    <braunr> did the pager die ?
+    <mcsim> When I get this message pager dies, but also computer can suddenly
+      reboot
+    <braunr> i guess the pager crashing makes mach print this error
+    <braunr> but then you may have a dead port instead of a null port, i don't
+      remember the details
+    <mcsim> braunr: thank you.
+    <mcsim> btw, for big file sizes fsx breaks on ext2fs
+    <braunr> could you identify the threshold ?
+    <braunr> and what's fsx exactly ?
+    <mcsim> fsx is a testing utility for filesystems
+    <mcsim> see http://codemonkey.org.uk/projects/fsx/
+    <braunr> ah, written by tevanian
+    <mcsim> threshold seems to be 8Mb
+    <braunr> fyi, avadis tevanian is the main author of the mach 3 core
+      services and VM parts
+    <braunr> well, ext2fs is bugged, we already know that
+    <braunr> old code maintained as well as possible, but still
+    <mcsim> hmm, with 6mb it breaks too
+    <braunr> i guess that it may break on anything larger than a page actually
+      :p
+    <mcsim> When I tested with size of 256kb, fsx worked quite long and didn't
+      break
+    <braunr> mcsim: without knowing exactly what the test actually does, it's
+      hard to tell
+    <mcsim> I see, I just wanted to tell that there are bugs in ext2fs too. But
+      I didn't debugged it.
+    <mcsim> fsx performs different operations, like read, write, truncate file,
+      grow file in random order.
+    <braunr> in parellel too ?
+    <braunr> parellel
+    <braunr> parallel*
+    <mcsim> no
+    <mcsim> I run several fsx's parallel on tmpfs, but they break on file with
+      size 8mb.
+    <braunr> that must match something in mach
+    <braunr> s/must/could/ :)
+    <mcsim> braunr: I've pushed my commits to mplaneta/tmpfs/master branch in
+      hurd repository, so you could review it.
+    <braunr> you shouldn't do that just for me :p
+    <braunr> you should do that regularly, and ask for reviews after
+      (e.g. during the meetings) 
+    <mcsim> everyone could do that :)
+    <braunr> i'm quite busy currently unfortunately
+    <braunr> i'll try when i have time, but the best would be to ask very
+      specific questions
+    <braunr> these are usually the faster to answer for people ho have the
+      necessary expertise to help you
+    <braunr> fastest*
+    <mcsim> ok.
+    <mcsim> braunr: probably, I was doing something wrong, because now parallel
+      works only for small sizes. Sorry, for disinformation.
+
+
+### IRC, freenode, #hurd, 2012-01-25
+
+    <antrik> braunr: actually, the paging errors are *precisely* the way my
+      system tends to die...
+    <antrik> (it's after about a month of uptime usually though, not a week...)
+    <antrik> tschwinge: in my case at least, I have still plenty of swap when
+      this happens. swap usage is generally at about the amount of physical
+      memory -- I have no idea though whether there is an actual connection, or
+      it's just coincidence
+    <braunr> antrik: ok, your hurd dies because of memory issues, my virtual
+      machines die because of something else (though idk what)
+    <antrik> before I aquired the habit of running my box 24/7 and thus hitting
+      this issue, most of the hangs I experienced were also of a different
+      nature... but very rare in general, except when doing specific
+      problematic actions
+    <mcsim> antrik: yes. Do you get messages like that I posted?
+    <mcsim> here is it: memory_object_data_request(0x0, 0x0, 0xf000, 0x1000,
+      0x1) failed, 10000003
+    <antrik> mcsim: I can't tell for sure (never noted them down, silly me...)
+    <antrik> but I definitely get paging errors right before it hangs
+    <antrik> I guess that was unclear... what I'm trying to say is: I do get
+      memory_object_data_request() failed; but I'm not sure about the
+      parameters
+    <mcsim> antrik: ok. Thank you.
+    <mcsim> I'll try to find something in defpager, but there should be errors
+      in mach too. At least because sometimes computer suddenly reboots during
+      test.
+    <antrik> mcsim: I don't get sudden reboots
+    <antrik> might be a different error
+    <antrik> do you have debugging mode activated in Mach? otherwise it reboots
+      on kernel panics...
+    <mcsim> antrik: no. But usually on kernel panics mach waits for some time
+      showing the error message and only than reboots.
+    <antrik> OK
+    <mcsim> how can I know that tmpfs is stable enough? Correcting errors in
+      kernel to make fsx test work seems to be very complex.
+    <mcsim> *If errors are in kernel.
+    <antrik> well, it seems that you tested it already much more thoroughly
+      than any other code in the Hurd was ever tested :-)
+    <antrik> of course it would be great if you could pinpoint some of the
+      problems you see nevertheless :-)
+    <antrik> but that's not really necessary before declaring tmpfs good enough
+      I'd say
+    <mcsim> ok. I'll describe every error I meet on my userpage
+    <mcsim> but it will take some time, not before weekend.
+    <antrik> don't worry, it's not urgent
+    <antrik> the reason I'd really love to see those errors investigated is
+      that most likely they are the same ones that cause stability problems in
+      actual use...
+    <antrik> having an easy method for reproducing them is already a good start
+    <mcsim> no. they are not the same
+    <mcsim> every time i get different one
+    <mcsim> especially when i just start one process fsx and wait error
+    <antrik> mcsim: have you watched memory stats while running it? if it's
+      related to the problems I'm experiencing, you will probably see rising
+      memory use while the test is running
+    <mcsim> it could be reboot, message, I posted and also fsx could stop
+      telling that something wrong with data
+    <antrik> you get all of these also on ext2?
+    <mcsim> i've done it only once. Here is the log:
+      http://paste.debian.net/153511/
+    <mcsim> I saved "free" output every 30 seconds
+    <mcsim> no. I'll do it now
+    <antrik> would be better to log with "vmstat 1"
+    <mcsim> ok.
+    <mcsim> as you can see, there is now any leek during work. But near end
+      free memory suddenly decreases
+    <antrik> yeah... it's a bit odd, as there is a single large drop, but seems
+      stable again afterwards...
+    <antrik> a more detailed log might shed some light
+    <mcsim> drop at the beginning was when I started translator.
+    <mcsim> what kind of log do you mean?
+    <antrik> vmstat 1 I mean
+    <mcsim> ah...