IRC.

1 files changed, 397 insertions, 0 deletions

diff --git a/open_issues/gnumach_memory_management.mdwn b/open_issues/gnumach_memory_management.mdwn
index 448aafcc..a728fc9d 100644
--- a/open_issues/gnumach_memory_management.mdwn
+++ b/open_issues/gnumach_memory_management.mdwn
@@ -923,3 +923,400 @@ There is a [[!FF_project 266]][[!tag bounty]] on this task.
     <braunr> 20 years ago
     <braunr> but it's a source of deadlock
     <mcsim> Indeed. I'll won't use kmem_alloc_pageable.
+
+
+# IRC, freenode, #hurd, 2011-08-09
+
+    < braunr> mcsim: what's the "bug related to MEM_CF_VERIFY" you refer to in
+      one of your commits ?
+    < braunr> mcsim: don't use spin_lock_t as a member of another structure
+    < mcsim> braunr: I confused with types in *_verify functions, so they
+      didn't work. Than I fixed it in the commit you mentioned.
+    < braunr> in gnumach, most types are actually structure pointers
+    < braunr> use simple_lock_data_t
+    < braunr> mcsim: ok
+    < mcsim> > use simple_lock_data_t
+    < mcsim> braunr: ok
+    < braunr> mcsim: don't make too many changes to the code base, and if
+      you're unsure, don't hesitate to ask
+    < braunr> also, i really insist you rename the allocator, as done in x15
+      for example
+      (http://git.sceen.net/rbraun/x15mach.git/?a=blob;f=vm/kmem.c), instead of
+      a name based on mine :/
+    < mcsim> braunr: Ok. It was just work name. When I finish I'll rename the
+      allocator.
+    < braunr> other than that, it's nice to see progress
+    < braunr> although again, it would be better with some reports along
+    < braunr> i won't be present at the meeting tomorrow unfortunately, but you
+      should use those to report the status of your work
+    < mcsim> braunr: You've said that I have to tweak gc process. Did you mean
+      to call mem_gc() when physical memory ends instead of calling it every x
+      seconds? Or something else?
+    < braunr> there are multiple topics, alhtough only one that really matters
+    < braunr> study how zone_gc was called
+    < braunr> reclaiming memory should happen when there is pressure on the VM
+      subsystem
+    < braunr> but it shouldn't happen too ofte, otherwise there is trashing
+    < braunr> and your caches become mostly useless
+    < braunr> the original slab allocator uses a 15-second period after a
+      reclaim during which reclaiming has no effect
+    < braunr> this allows having a somehow stable working set for this duration
+    < braunr> the linux slab allocator uses 5 seconds, but has a more
+      complicated reclaiming mechanism
+    < braunr> it releases memory gradually, and from reclaimable caches only
+      (dentry for example)
+    < braunr> for x15 i intend to implement the original 15 second interval and
+      then perform full reclaims
+    < mcsim> In zalloc mem_gc is called by vm_pageout_scan, but not often than
+      once a second.
+    < mcsim> In balloc I've changed interval to once in 15 seconds.
+    < braunr> don't use the code as it is
+    < braunr> the version you've based your work on was meant for userspace
+    < braunr> where there isn't memory pressure
+    < braunr> so a timer is used to trigger reclaims at regular intervals
+    < braunr> it's different in a kernel
+    < braunr> mcsim: where did you see vm_pageout_scan call the zone gc once a
+      second ?
+    < mcsim> vm_pageout_scan calls consider_zone_gc and consider_zone_gc checks
+      if second is passed.
+    < braunr> where ?
+    < mcsim> Than zone_gc can be called.
+    < braunr> ah ok, it's in zaclloc.c then
+    < braunr> zalloc.c
+    < braunr> yes this function is fine
+    < mcsim> so old gc didn't consider vm pressure. Or I missed something.
+    < braunr> it did
+    < mcsim> how?
+    < braunr> well, it's called by the pageout daemon
+    < braunr> under memory pressure
+    < braunr> so it's fine
+    < mcsim> so if mem_gc is called by pageout daemon is it fine?
+    < braunr> it must be changed to do something similar to what
+      consider_zone_gc does
+    < mcsim> It does. mem_gc does the same work as consider_zone_gc and
+      zone_gc.
+    < braunr> good
+    < mcsim> so gc process is fine?
+    < braunr> should be
+    < braunr> i see mem.c only includes mem.h, which then includes other
+      headers
+    < braunr> don't do that
+    < braunr> always include all the headers you need where you need them
+    < braunr> if you need avltree.h in both mem.c and mem.h, include it in both
+      files
+    < braunr> and by the way, i recommend you use the red black tree instead of
+      the avl type
+    < braunr> (it's the same interface so it shouldn't take long)
+    < mcsim> As to report. If you won't be present at the meeting, I can tell
+      you what I have to do now.
+    < braunr> sure
+    < braunr> in addition, use GPLv2 as the license, teh BSD one is meant for
+      the userspace version only
+    < braunr> GPLv2+ actually
+    < braunr> hm you don't need list.c
+    < braunr> it would only add dead code
+    < braunr> "Zone for dynamical allocator", don't mix terms
+    < braunr> this comment refers to a vm_map, so call it a map
+    < mcsim> 1. Change constructor for kentry_alloc_cache.
+    < mcsim> 2. Make measurements.
+    < mcsim> +
+    < mcsim> 3. Use simple_lock_data_t
+    < mcsim> 4. Replace license
+    < braunr> kentry_alloc_cache <= what is that ?
+    < braunr> cache for kernel map entries in vm_map ?
+    < braunr> the comment for mem_cpu_pool_get doesn't apply in gnumach, as
+      there is no kernel preemption
+    < braunr> "Don't attempt mem GC more frequently than hz/MEM_GC_INTERVAL
+      times a second.
+    < braunr> "
+    < mcsim> sorry. I meant vm_map_kentry_cache
+    < braunr> hm nothing actually about this comment
+    < braunr> mcsim: ok
+    < braunr> yes kernel map entries need special handling
+    < braunr> i don't know how it's done in gnumach though
+    < braunr> static preallocation ?
+    < mcsim> yes
+    < braunr> that's ugly :p
+    < mcsim> but it uses dynamic allocation further even for vm_map kernel
+      entries
+    < braunr> although such bootstrapping issues are generally difficult to
+      solve elegantly
+    < braunr> ah
+    < mcsim> now I use only static allocation, but I'll add dynamic allocation
+      too
+    < braunr> when you have time, mind the coding style (convert everything to
+      gnumach style, which mostly implies using tabs instead of 4-spaces
+      indentation)
+    < braunr> when you'll work on dynamic allocation for the kernel map
+      entries, you may want to review how it's done in x15
+    < braunr> the mem_source type was originally intended for that purpose, but
+      has slightly changed once the allocator was adapted to work in my kernel
+    < mcsim> ok
+    < braunr> vm_map_kentry_zone is the only zone created with ZONE_FIXED
+    < braunr> and it is zcram()'ed immediately after
+    < braunr> so you can consider it a statically allocated zone
+    < braunr> in x15 i use another strategy: there is a special kernel submap
+      named kentry_map which contains only one map entry (statically allocated)
+    < braunr> this map is the backend (mem_source) for the kentry_cache
+    < braunr> the kentry_cache is created with a special flag that tells it
+      memory can't be reclaimed
+    < braunr> when the cache needs to grow, the single map entry is extended to
+      cover the allocated memory
+    < braunr> it's similar to the way pmap_growkernel() works for kernel page
+      table pages
+    < braunr> (and is actually based on that idea)
+    < braunr> it's a compromise between full static and dynamic allocation
+      types
+    < braunr> the advantage is that the allocator code can be used (so there is
+      no need for a special allocator like in netbsd)
+    < braunr> the drawback is that some resources can never be returned to
+      their source (and under peaks, the amount of unfreeable resources could
+      become large, but this is unexpected)
+    < braunr> mcsim: for now you shouldn't waste your time with this
+    < braunr> i see the number of kernel map entries is fixed at 256
+    < braunr> and i've never seen the kernel use more than around 30 entries
+    < mcsim> Do you think that I have to left this problem to the end?
+    < braunr> yes
+
+
+# IRC, freenode, #hurd, 2011-08-11
+
+    < mcsim> braunr: Hello. Can you give me an advice how can I make
+      measurements better?
+    < braunr> mcsim: what kind of measurements
+    < mcsim> braunr: How much is your allocator better than zalloc.
+    < braunr> slightly :p
+    < braunr> that's why i never took the time to put it in gnumach
+    < mcsim> braunr: Just I thought that there are some rules or
+      recommendations of such measurements. Or I can do them any way I want?
+    < braunr> mcsim: i don't know
+    < braunr> mcsim: benchmarking is an art of its own, and i don't even know
+      how to use the bits of profiling code available in gnumach (if it still
+      works)
+    < antrik> mcsim: hm... are you saying you already have a running system
+      with slab allocator?... :-)
+    < braunr> mcsim: the main advantage i can see is the removal of many
+      arbitrary hard limits
+    < mcsim> antrik: yes
+    < antrik> \o/
+    < antrik> nice work!
+    < braunr> :)
+    < braunr> the cpu layer should also help a bit, but it's hard to measure
+    < braunr> i guess it could be seen on the ipc path for very small buffers
+    < mcsim> antrik: Thanks. But I still have to 1. Change constructor for
+      kentry_alloc_cache. and 2. Make measurements.
+    < braunr> and polish the whole thing :p
+    < antrik> mcsim: I'm not sure this can be measured... the performance
+      differente in any real live usage is probably just a few percent at most
+      -- it's hard to construct a benchmark giving enough precision so it's not
+      drowned in noise...
+    < antrik> perhaps it conserves some memory -- but that too would be hard to
+      measure I fear
+    < braunr> yes
+    < braunr> there *should* be better allocation times, less fragmentation,
+      better accounting ... :)
+    < braunr> and no arbitrary limits !
+    < antrik> :-)
+    < braunr> oh, and the self debugging features can be nice too
+    < mcsim> But I need to prove that my work wasn't useless
+    < braunr> well it wasn't, but that's hard to measure
+    < braunr> it's easy to prove though, since there are additional features
+      that weren't present in the zone allocator
+    < mcsim> Ok. If there are some profiling features in gnumach can you give
+      me a link with their description?
+    < braunr> mcsim: sorry, no
+    < braunr> mcsim: you could still write the basic loop test, which counts
+      the number of allocations performed in a fixed time interval
+    < braunr> but as it doesn't match many real life patterns, it won't be very
+      useful
+    < braunr> and i'm afraid that if you consider real life patterns, you'll
+      see how negligeable the improvement can be compared to other operations
+      such as memory copies or I/O (ouch)
+    < mcsim> Do network drivers use this allocator?
+    < mcsim> ok. I'll scrape up some test and than I'll report results.
+
+
+# IRC, freenode, #hurd, 2011-08-26
+
+    < mcsim> hello. Are there any analogs of copy_to_user and copy_from_user in
+      linux for gnumach?
+    < mcsim> Or how can I determine memory map if I know address? I need this
+      for vm_map_copyin
+    < guillem> mcsim: vm_map_lookup_entry?
+    < mcsim> guillem: but I need to transmit map to this function and it will
+      return an entry which contains specified address.
+    < mcsim> And I don't know what map have I transmit.
+    < mcsim> I need to transfer static array from kernel to user. What map
+      contains static data?
+    < antrik> mcsim: Mach doesn't have copy_{from,to}_user -- instead, large
+      chunks of data are transferred as out-of-line data in IPC messages
+      (i.e. using VM magic)
+    < mcsim> antrik: can you give me an example? I just found using
+      vm_map_copyin in host_zone_info.
+    < antrik> no idea what vm_map_copyin is to be honest...
+
+
+# IRC, freenode, #hurd, 2011-08-27
+
+    < braunr> mcsim: the primitives are named copyin/copyout, and they are used
+      for messages with inline data
+    < braunr> or copyinmsg/copyoutmsg
+    < braunr> vm_map_copyin/out should be used for chunks larger than a page
+      (or roughly a page)
+    < braunr> also, when writing to a task space, see which is better suited:
+      vm_map_copyout or vm_map_copy_overwrite
+    < mcsim> braunr: and what will be src_map for vm_map_copyin/out?
+    < braunr> the caller map
+    < braunr> which you can get with current_map() iirc
+    < mcsim> braunr: thank you
+    < braunr> be careful not to leak anything in the transferred buffers
+    < braunr> memset() to 0 if in doubt
+    < mcsim> braunr:ok
+    < braunr> antrik: vm_map_copyin() is roughly vm_read()
+    < antrik> braunr: what is it used for?
+    < braunr> antrik: 01:11 < antrik> mcsim: Mach doesn't have
+      copy_{from,to}_user -- instead, large chunks of data are transferred as
+      out-of-line data in IPC messages (i.e. using VM magic)
+    < braunr> antrik: that "VM magic" is partly implemented using vm_map_copy*
+      functions
+    < antrik> braunr: oh, you mean it doesn't actually copy data, but only page
+      table entries? if so, that's *not* really comparable to
+      copy_{from,to}_user()...
+
+
+# IRC, freenode, #hurd, 2011-08-28
+
+    < braunr> antrik: the equivalent of copy_{from,to}_user are
+      copy{in,out}{,msg}
+    < braunr> antrik: but when the data size is about a page or more, it's
+      better not to copy, of course
+    < antrik> braunr: it's actually not clear at all that it's really better to
+      do VM magic than to copy...
+
+
+# IRC, freenode, #hurd, 2011-08-29
+
+    < braunr> antrik: at least, that used to be the general idea, and with a
+      simpler VM i suspect it's still true
+    < braunr> mcsim: did you progress on your host_zone_info replacement ?
+    < braunr> mcsim: i think you should stick to what the original
+      implementation did
+    < braunr> which is making an inline copy if caller provided enough space,
+      using kmem_alloc_pageable otherwise
+    < braunr> specify ipc_kernel_map if using kmem_alloc_pageable
+    < mcsim> braunr: yes. And it works. But I use kmem_alloc, not pageable. Is
+      it worse?
+    < mcsim> braunr: host_zone_info replacement is pushed to savannah
+      repository. 
+    < braunr> mcsim: i'll have a look
+    < mcsim> braunr: I've pushed one more commit just now, which has attitude
+      to host_zone_info.
+    < braunr> mem_alloc_early_init should be renamed mem_bootstrap
+    < mcsim> ok
+    < braunr> mcsim: i don't understand your call to kmem_free
+    < mcsim> braunr: It shouldn't be there?
+    < braunr> why should it be there ?
+    < braunr> you're freeing what the copy object references
+    < braunr> it's strange that it even works
+    < braunr> also, you shouldn't pass infop directly as the copy object
+    < braunr> i guess you get a warning for that
+    < braunr> do what the original code does: use an intermediate copy object
+      and a cast
+    < mcsim> ok
+    < braunr> another error (without consequence but still, you should mind it)
+    < braunr> simple_lock(&mem_cache_list_lock);
+    < braunr> [...]
+    < braunr> kr = kmem_alloc(ipc_kernel_map, &info, info_size);
+    < braunr> you can't hold simple locks while allocating memory
+    < braunr> read how the original implementation works around this
+    < mcsim> ok
+    < braunr> i guess host_zone_info assumes the zone list doesn't change much
+      while unlocked
+    < braunr> or that's it's rather unimportant since it's for debugging
+    < braunr> a strict snapshot isn't required
+    < braunr> list_for_each_entry(&mem_cache_list, cache, node) max_caches++;
+    < braunr> you should really use two separate lines for readability
+    < braunr> also, instead of counting each time, you could just maintain a
+      global counter
+    < braunr> mcsim: use strncpy instead of strcpy for the cache names
+    < braunr> not to avoid overflow but rather to clear the unused bytes at the
+      end of the buffer
+    < braunr> mcsim: about kmem_alloc vs kmem_alloc_pageable, it's a minor
+      issue
+    < braunr> you're handing off debugging data to a userspace application
+    < braunr> a rather dull reporting tool in most cases, which doesn't require
+      wired down memory
+    < braunr> so in order to better use available memory, pageable memory
+      should be used
+    < braunr> in the future i guess it could become a not-so-minor issue though
+    < mcsim> ok. I'll fix it
+    < braunr> mcsim: have you tried to run the kernel with MC_VERIFY always on
+      ?
+    < braunr> MEM_CF_VERIFY actually
+    < mcsim1> yes.
+    < braunr> oh
+    < braunr> nothing wrong 
+    < braunr> ?
+    < mcsim1> it is always set
+    < braunr> ok
+    < braunr> ah, you set it in macros.h ..
+    < braunr> don't
+    < braunr> put it in mem.c if you want, or better, make it a compile-time
+      option
+    < braunr> macros.h is a tiny macro library, it shouldn't define such
+      unrelated options
+    < mcsim1> ok.
+    < braunr> mcsim1: did you try fault injection to make sure the checking
+      code actually works and how it behaves when an error occurs ?
+    < mcsim1> I think that when I finish I'll merge files cpu.h and macros.h
+      with mem.c
+    < braunr> yes that would simplify things
+    < mcsim1> Yes. When I confused with types mem_buf_fill worked wrong and
+      panic occurred.
+    < braunr> very good
+    < braunr> have you progressed concerning the measurements you wanted to do
+      ?
+    < mcsim1> not much.
+    < braunr> ok
+    < mcsim1> I think they will be ready in a few days.
+    < antrik> what measurements are these?
+    < mcsim1> braunr: What maximal size for static data and stack in kernel?
+    < braunr> what do you mean ?
+    < braunr> kernel stacks are one page if i'm right
+    < braunr> static data (rodata+data+bss) are limited by grub bugs only :)
+    < mcsim1> braunr: probably they are present, because when I created too big
+      array I couldn't boot kernel
+    < braunr> local variable or static ?
+    < mcsim1> static
+    < braunr> how large ?
+    < mcsim1> 4Mb
+    < braunr> hm
+    < braunr> it's not a grub bug then
+    < braunr> i was able to embed as much as 32 MiB in x15 while doing this
+      kind of tests
+    < braunr> I guess it's the gnu mach boot code which only preallocates one
+      page for the initial kernel mapping
+    < braunr> one PTP (page table page) maps 4 MiB
+    < braunr> (x15 does this completely dynamically, unlike mach or even
+      current BSDs)
+    < mcsim1> antrik: First I want to measure time of each cache
+      creation/allocation/deallocation and then compile kernel.
+    < braunr> cache creation is irrelevant
+    < braunr> because of the cpu pools in the new allocator, you should test at
+      least two different allocation patterns
+    < braunr> one with quick allocs/frees
+    < braunr> the other with large numbers of allocs then their matching frees
+    < braunr> (larger being at least 100)
+    < braunr> i'd say the cpu pool layer is the real advantage over the
+      previous zone allocator
+    < braunr> (from a performance perspective)
+    < mcsim1> But there is only one cpu
+    < braunr> it doesn't matter
+    < braunr> it's stil a very effective cache
+    < braunr> in addition to reducing contention
+    < braunr> compare mem_cpu_pool_pop() against mem_cache_alloc_from_slab()
+    < braunr> mcsim1: work is needed to polish the whole thing, but getting it
+      actually working is a nice achievement for someone new on the project
+    < braunr> i hope it helped you learn about memory allocation, virtual
+      memory, gnu mach and the hurd in general :)
+    < antrik> indeed :-)