1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
|
[[!meta copyright="Copyright © 2008, 2009, 2010, 2013, 2024 Free
Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
[[!tag open_issue_gnumach open_issue_hurd open_issue_viengoos]]
[[microkernel/Mach]] interfaces do not allow for proper resource accounting,
when a server allocates resources on behalf of a client.
Mach can't do a good job at resource management, as it doesn't have enough
information how resources are used: which data is important and which is
discardable, for example.
<!-- the following 3 paragraphs comes from this email:
lists.gnu.org/archive/html/bug-hurd/2009-07/msg00084.html -->
This is the fundamental failing of the Mach/Hurd architecture. The
Hurd operates via many server to client relationships, in which
servers request resources on behalf of their clients. For example at
any given time, `extfs` could have many different clients (emacs, vim,
git etc.) requesting data, creating files, deleting files, re-naming
files, etc. Suppose one rogue client out of 50 is continually
requesting increasingly more memory, which is exhasting the machine's
resources. As far as Mach knows, `ext2fs` is wasting RAM. It doesn't
know that one `ext2fs`' client program is at fault. There is no way
for Mach to fix this, since it should not kill `ext2fs`, and it cannot
know which `ext2fs` client to kill.
This server/client architecture is a problem that exists elsewhere. A
good example is `X`. Firefox might allocate a lot of pixmaps, which
causes `X` to use more memory. Linux actually used to kill X, because
of this several years ago.
This problem is much worse on a multiserver system, because we have
many server/client relationships. A simple fix that would limit these
issues is to introduce fixed limits on various kinds of resource
usage. A proper fix requires a way to attribute all resource usage to
the clients -- either by avoiding server-side allocation or by keeping
track of who is requesting resources. Both of these changes requires
lots of changes to low-level code.
These issues are what Neal Walfield explored with his kernel:
[[microkernel/viengoos]].
# Kernel
Inside the [[kernel]], there is commonly a need to allocate resources according
to externally induced demand, dynamically. For example, for memory-management
data structures (page tables), process table entries, thread control blocks,
[[capability]] tables, incoming network packages, blocks that are read in from
disk, the keyboard type-ahead buffer for a in-kernel keyboard driver. Some of
these are due to actions driven by user-space requests, others are due to
actions internal to the the kernel itself. Some of these buffers can be sized
statically (keyboard type-ahead buffer), and are thus unproblematic. Others
are not, and should thus be attributed to their user space entities. In the
latter (ideal) case, all resources -- that is, including those needed inside
the kernel -- that a user space task needs for execution are provided by itself
(and, in turn, provided by its parent / principal), and the kernel itself does
not need to allocate any resources dynamically out of an its own memory pool.
This avoids issues like [[microkernel/Mach]]'s [[zalloc_panics]] upon user
space processes allocating too many [[microkernel/mach/port]]s, for example.
[[!toggleable id=fof_plos09 text="""[[!template id=note
text="*[[fof\_plos09|microkernel/barrelfish]]*:
{{$microkernel/barrelfish#fof_plos09}}"]]"""]]
[[!toggleable id=sel4 text="""[[!template id=note
text="[[*sel4*|microkernel/l4]]: {{$microkernel/l4#sel4}}"]]"""]]
In [[!toggle id=fof_plos09 text="[fof\_plos09]"]], the authors describe in
section 3 how they model their [[capability]] system according to [[!toggle
id=sel4 text="[sel4]"]] using a *retype* operation that *takes an existing
capability and produces one or more derived capabilities [...] used to create
new kernel-level memory objects (such as page tables or execution contexts)
from capabilities to raw regions of RAM*.
This is, of course, non-trivial to implement, and also requires changing the
[[RPC]] interfaces, for example, but it is a valid approach, a research topic.
([[!taglink open_issue_documentation]]: compare this to Linux [`vmsplice`'s
SPLICE_F_GIFT
flag](http://www.kernel.org/doc/man-pages/online/pages/man2/vmsplice.2.html#DESCRIPTION).)
## IRC, freenode, #hurd, 2011-07-31
< braunr> one of the biggest problems on the hurd is that, when a client
makes a call, kernel (and other) resources are allocated on behalf of the
server performaing the requested action
< braunr> performing*
< braunr> this makes implementing scheduling and limits difficult
< CTKArcher> And could changing the kernel change anything to that ?
< braunr> yes but you'd probably need to change its interface as well
< braunr> iirc, the critique describes resource containers
< braunr> but no work has been done on the current hurd (hence the hurdng
attempts)
## IRC, freenode, #hurd, 2013-08-13
In context of <https://teythoon.cryptobitch.de/posts/my-worst-week-yet/>.
<braunr> teythoon: actually, thread migration isn't required for resource
accounting
[[Mach_migrating_threads]].
<teythoon> braunr: but it solves it for free, doesn't it?
<braunr> teythoon: no
<braunr> it's really more complicated than that
# Further Examples
* [[hurd/critique]]
* [[IO_accounting]]
* [[translators_set_up_by_untrusted_users]], and [[pagers]]
* [[configure_max_command_line_length]]
## [[hurd/translator/exec]] server
### IRC, freenode, #hurd, 2013-08-05
<teythoon> unzipping stuff in the exec server enables a dos on filesystem
translators
<teythoon> https://teythoon.cryptobitch.de/gsoc/heap/hello-1g.bz2 is
/hurd/hello padded with a gig of zeros, compressed with bzip2
<teythoon> if set as an passive translator, it stalls other requests to the
filesystem, at least it does if ext2fs is used
<braunr> teythoon: ?
<braunr> teythoon: what's the dos here ?
<teythoon> I can prevent you from doing anything with the root filesystem
<teythoon> I'm kind of surprised myself, maybe a lock is held during the
exec of the translator?
<teythoon> the filesystem the hello-1g.bz2 translator is bound to is
affected
<braunr> teythoon: i don't understand
<braunr> have you tried starting something from another file system ?
<braunr> the lock may simply be in the exec server itself
<teythoon> no, starting other things works fine
<teythoon> but on the other hand, a find / is stalled
<braunr> :/
<braunr> *sigh*
<teythoon> don't worry
<teythoon> there is a solution :p
<braunr> :)
<teythoon> and it only requires deleting code
|
