1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
|
[[!meta copyright="Copyright © 2011, 2014 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
## About Myself
- Name: Sergio Lopez
- Email: <slp@sinrega.org>
- Blog: <http://blogs.nologin.es/slopez>
- Languages: Spanish, English
## Stuff I'm working on
### Advisory Pageout
[[open_issues/libpager_deadlock]], [[open_issues/gnumach_memory_management_2]].
#### Rationale
This work has two objectives:
- Put a limit on the number of dirty (or better said, potentially dirty) pages that can be at one given time on Hurd. Having lots of dirty pages stresses the translators when syncing its memory objects, since they could receive a huge amount of m_o_data_return messages in a short amount of time. This is the primary cause for thread explosions.
- When memory is scarce due to work load, try to avoid flushing anonymous memory. This is done by trying to satisfy the target of free pages releasing clean external pages (which can be considered as cache).
#### Trying it
I've commited this work to this branch:
<https://git.savannah.gnu.org/cgit/hurd/gnumach.git/log/?h=k0ro/advisory_pageout/master>
You'll also need the counterpart for user space:
<http://blogs.nologin.es/slopez/files/hurd-advisory_pageout.patch>
If you try it, please let me know your experience with it.
### Reduce task map fragmentation
After some hours of workload over an ext2fs translator, the number of entries in its memory map (which uses to be something around 60 when clean) is increased up to a few thousand. This increases the cost of each vm_map_lookup_entry, and this is perceived from a user perspective as a significant slowdown on the services provided by that translator.
The causes for the fragmentation of the task map are:
#### The problem
- When a big chunk of memory is reserved, and the protection level is changed in small fractions of it, each of those generates a new entry in the map. This happens in glibc's malloc and sysdeps/mach/hurd/brk.c.
- Allocating memory for permanent data structures in small chunks, without putting them at an specific address. GNU Mach has the ability of coalescing similar memory entries, but they must be contiguous. As every Hurd server needs to allocate small, temporal buffers for sending and receiving data, these ones will get in the middle of the permanent chunks, preventing them for being coalesced. This is the case of glibc's hurd/hurdmalloc.c, libpager's pagemap and libthread's per thread stack (and possibly some others).
#### The solution
- GNU Mach should be able of coalescing entries with similar properties after a call to vm_protect. I'm working in this direction.
- The use of vm_allocate/mmap in translators should be avoided, except for temporal buffers for IPC operations.
#### Workaround
While trying to find the causes of the task map fragmentation, I developed some changes which don't provide a complete, clean solution for this problem, but they minimize it (at least, for ext2fs) without significant drawbacks. You can find them here:
- <http://blogs.nologin.es/slopez/files/glibc-memory_fragmentation.patch>
- <http://blogs.nologin.es/slopez/files/libpager-memory_fragmentation.patch>
- <http://blogs.nologin.es/slopez/files/libthreads-memory_fragmentation.patch>
### memfs (Memory FS translator)
### Other miscellaneous changes
- ext2fs: write multiple pages at once to the backend storage. <http://blogs.nologin.es/slopez/files/ext2fs-multipage_data_return.patch>
- GNU Mach: avoid calling vm_object_deactivate_pages every time an object enters the cache. <http://blogs.nologin.es/slopez/files/gnumach-optimize_page_deactivation.patch>
|
