1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
|
[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
[[!toc]]
#Notes on tmpfs
## mach-defpager
[[defpager|http://www.bddebian.com:8888/~hurd-web/user/Maksym_Planeta/#defpager81111]]
[[http://www.mail-archive.com/bug-hurd@gnu.org/msg18859.html]]
[[My patch that fixes tmpfs and defpager|http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00086.html]]
My vision of the problem in short: when user tries to access to the memory backed by tmpfs first time, kernel asks defpager to initialize memory object and sets it's request port to some value. But sometimes user directly accesses to defpager and supplies it own request port. So defpager should handle different request port, what causes errors.
## Steps
1. Find out what causes crashes in tmpfs with defpager
[[http://www.gnu.org/s/hurd/hurd/translator/tmpfs/notes_various.html]]
TODO: Consider deleting of parameter "port" in function mach-defpager/default_pager.c:pager_port_list_insert
since this parameter is unused
2. Write own pager
6.11.11 Reading/writing for files that fit in vm_page_size works
7.11.11 Works for any size.
3. Make links work
Symlinks behavior: [[links|http://www.bddebian.com:8888/~hurd-web/user/Maksym_Planeta/#links81111]]
8.11.11 Symlinks work.
4. Control of used space by tmpfs.
TODO: Make tmpfs use not more space than it was allowed.
5. Thread safety.
TODO: During execution tmpfs hangs in random places. The most possible is variant is deadlocks,
because nothing was undertaken for thread safety.
6. After sometime of inactivity tmpfs exits.
TODO: Find out why and correct this.
#Chalanges
Translators vs FUSE:
[[What can a translator do that FUSE can't?|http://lists.gnu.org/archive/html/bug-hurd/2010-07/msg00061.html]]
[[Re: Hurd translators on FUSE|http://lists.gnu.org/archive/html/l4-hurd/2009-09/msg00146.html]]
[[Example of sane utilization of filesystem stored in RAM|http://habrahabr.ru/blogs/gdev/131043/]] (Russian). Author of this article copied some resources of game "World of Tanks" to RAM-drive and game started load much faster. Although he used Windows in this article, this could be good example of benefits, which filesystem, stored in RAM, could give.
#Debugging
To debug tmpfs, using libraries from "$PWD"/lib and trace rpc:
settrans -ca foo /usr/bin/env LD_LIBRARY_PATH="$PWD"/lib utils/rpctrace -I /usr/share/msgids/ tmpfs/tmpfs 1M
LD_LIBRARY_PATH="$PWD"/lib gdb tmpfs/tmpfs `pidof tmpfs`
For debugging ext2fs:
settrans --create --active ramdisk0 /hurd/storeio -T copy zero:32M && \
/sbin/mkfs.ext2 -F -b 4096 ramdisk0 && \
settrans --active --orphan ramdisk0 /usr/bin/env LD_LIBRARY_PATH="$PWD"/lib utils/rpctrace -I /usr/share/msgids/ \
ext2fs/ext2fs.static ramdisk0
#Questions
1. What are sequence numbers? What are they used for?
2. Is there any way to debug mach-defpager? When I set breakpoint to any function in it, pager never breaks.
#Links
1. [[Cthreads manuals|http://www.cc.gatech.edu/classes/cs6432_99_winter/threads_man/]]
#Conversations
## 8.11.11
### links<a id="links81111"/>
(10:29:11) braunr: mcsim: ln -s foo/bar foo/baz means the link name is baz in the foo directory,
and its target (relative to its directory) is foo/bar (which would mean /tmp/foo/foo/bar in canonical form)
(10:29:42) braunr: youpi: tschwinge: what did ludovic achieve ?
(10:30:06) tschwinge: mcsim: As Richard says, symlink targets are always relative to the directory they're contained in.
(10:30:51) tschwinge: braunr: This Hydra/Nix (I wtill mix it all up) thing is kind of a package managing system.
(10:31:17) tschwinge: He has written scripts for bootstrapping a Hurd toolchain.
(10:31:26) braunr: oh ok
(10:31:27) mcsim: so, if I want to create link in cd, first I need to cd there?
(10:31:28) tschwinge: And then uses that to build a whole bootable image.
(10:31:36) mcsim: in foo*
(10:31:36) braunr: mcsim: just provide the right paths
(10:32:11) braunr: $ touch foo/bar
(10:32:14) braunr: $ ln -s bar foo/baz
(10:32:32) braunr: bar
(10:32:35) braunr: baz -> bar
### defpager<a id="defpager81111"/>
earlier:
<tschwinge>: 1. On every system there is a ``default pager'' (mach-defpager). That one is responsible
for all ``anonymous memory''. For example, when you do malloc(10 MiB), and then there is memory pressure,
this 10 MiB memory region is backed by the default pager, whose job then is it to provide the backing store for this.
<tschwinge>: This is what commonly would be known as a swap partition.
<tschwinge>: And this is also the way tmpfs works (as I understand it).
<tschwinge>: malloc(10 MiB) can also be mmap(MAP_ANONYMOUS, 10 MIB); that's the same, essentially.
<tschwinge>: Now, for ext2fs or any other disk-based file system, this is different:
<tschwinge>: The ext2fs translator implements its own backing store, namely it accesses the disk for storing
changed file content, or to read in data from disk if a new file is opened.
...
(10:36:14) mcsim: who else uses defpager besides tmpfs and kernel?
(10:36:27) braunr: normally, nothing directly
(10:37:04) mcsim: than why tmpfs should use defpager?
(10:37:22) braunr: it's its backend
(10:37:28) braunr: backign store rather
(10:37:38) braunr: the backing store of most file systems are partitions
(10:37:44) braunr: tmpfs has none, it uses the swap space
(10:39:31) mcsim: if we allocate memory for tmpfs using vm_allocate, will it be able to use swap partition?
(10:39:56) braunr: it should
(10:40:27) braunr: vm_allocate just maps anonymous memory
(10:41:27) braunr: anonymous memory uses swap space as its backing store too
(10:43:47) braunr: but be aware that this part of the vm system is known to have deficiencies
(10:44:14) braunr: which is why all mach based implementations have rewritten their default pager
(10:45:11) mcsim: what kind of deficiencies?
(10:45:16) braunr: bugs
(10:45:39) braunr: and design issues, making anonymous memory fragmentation horrible
...
(15:23:33) antrik: mcsim: vm_allocate doesn't return a memory object; so it can't be passed to clients for mmap()
(15:50:37) mcsim: antrik: I use vm_allocate in pager_read_page
(15:54:43) antrik: mcsim: well, that means that you have to actually implement a pager yourself
(15:56:10) antrik: also, when the kernel asks the pager to write back some pages, it expects the memory to become free.
if you are "paging" to ordinary anonymous memory, this doesn't happen; so I expect it to have a very bad effect
on system performance
(15:56:54) antrik: both can be avoided by just passing a real anonymous memory object, i.e. one provided by the defpager
(15:57:07) antrik: only problem is that the current defpager implementation can't really handle that...
|
