[[!meta copyright="Copyright © 2008, 2009, 2010, 2013, 2024 Free
Software Foundation, Inc."]]

[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]

[[!tag open_issue_gnumach open_issue_hurd open_issue_viengoos]]

[[microkernel/Mach]] interfaces do not allow for proper resource accounting,
when a server allocates resources on behalf of a client.

Mach can't do a good job at resource management, as it doesn't have enough
information how resources are used: which data is important and which is
discardable, for example.

<!-- the following 3 paragraphs comes from this email:
	lists.gnu.org/archive/html/bug-hurd/2009-07/msg00084.html -->

This is the fundamental failing of the Mach/Hurd architecture.  The
Hurd operates via many server to client relationships, in which
servers request resources on behalf of their clients.  For example at
any given time, `extfs` could have many different clients (emacs, vim,
git etc.) requesting data, creating files, deleting files, re-naming
files, etc.  Suppose one rogue client out of 50 is continually
requesting increasingly more memory, which is exhasting the machine's
resources.  As far as Mach knows, `ext2fs` is wasting RAM.  It doesn't
know that one `ext2fs`' client program is at fault.  There is no way
for Mach to fix this, since it should not kill `ext2fs`, and it cannot
know which `ext2fs` client to kill.

This server/client architecture is a problem that exists elsewhere.  A
good example is `X`.  Firefox might allocate a lot of pixmaps, which
causes `X` to use more memory.  Linux actually used to kill X, because
of this several years ago.

This problem is much worse on a multiserver system, because we have
many server/client relationships.  A simple fix that would limit these
issues is to introduce fixed limits on various kinds of resource
usage.  A proper fix requires a way to attribute all resource usage to
the clients -- either by avoiding server-side allocation or by keeping
track of who is requesting resources.  Both of these changes requires
lots of changes to low-level code.

These issues are what Neal Walfield explored with his kernel:
[[microkernel/viengoos]].


# Kernel

Inside the [[kernel]], there is commonly a need to allocate resources according
to externally induced demand, dynamically.  For example, for memory-management
data structures (page tables), process table entries, thread control blocks,
[[capability]] tables, incoming network packages, blocks that are read in from
disk, the keyboard type-ahead buffer for a in-kernel keyboard driver.  Some of
these are due to actions driven by user-space requests, others are due to
actions internal to the the kernel itself.  Some of these buffers can be sized
statically (keyboard type-ahead buffer), and are thus unproblematic.  Others
are not, and should thus be attributed to their user space entities.  In the
latter (ideal) case, all resources -- that is, including those needed inside
the kernel -- that a user space task needs for execution are provided by itself
(and, in turn, provided by its parent / principal), and the kernel itself does
not need to allocate any resources dynamically out of an its own memory pool.
This avoids issues like [[microkernel/Mach]]'s [[zalloc_panics]] upon user
space processes allocating too many [[microkernel/mach/port]]s, for example.

[[!toggleable id=fof_plos09 text="""[[!template id=note
text="*[[fof\_plos09|microkernel/barrelfish]]*:
{{$microkernel/barrelfish#fof_plos09}}"]]"""]]

[[!toggleable id=sel4 text="""[[!template id=note
text="[[*sel4*|microkernel/l4]]: {{$microkernel/l4#sel4}}"]]"""]]

In [[!toggle id=fof_plos09 text="[fof\_plos09]"]], the authors describe in
section 3 how they model their [[capability]] system according to [[!toggle
id=sel4 text="[sel4]"]] using a *retype* operation that *takes an existing
capability and produces one or more derived capabilities [...] used to create
new kernel-level memory objects (such as page tables or execution contexts)
from capabilities to raw regions of RAM*.

This is, of course, non-trivial to implement, and also requires changing the
[[RPC]] interfaces, for example, but it is a valid approach, a research topic.

([[!taglink open_issue_documentation]]: compare this to Linux [`vmsplice`'s
SPLICE_F_GIFT
flag](http://www.kernel.org/doc/man-pages/online/pages/man2/vmsplice.2.html#DESCRIPTION).)


## IRC, freenode, #hurd, 2011-07-31

    < braunr> one of the biggest problems on the hurd is that, when a client
      makes a call, kernel (and other) resources are allocated on behalf of the
      server performaing the requested action
    < braunr> performing*
    < braunr> this makes implementing scheduling and limits difficult
    < CTKArcher> And could changing the kernel change anything to that ?
    < braunr> yes but you'd probably need to change its interface as well
    < braunr> iirc, the critique describes resource containers
    < braunr> but no work has been done on the current hurd (hence the hurdng
      attempts)


## IRC, freenode, #hurd, 2013-08-13

In context of <https://teythoon.cryptobitch.de/posts/my-worst-week-yet/>.

    <braunr> teythoon: actually, thread migration isn't required for resource
      accounting

[[Mach_migrating_threads]].

    <teythoon> braunr: but it solves it for free, doesn't it?
    <braunr> teythoon: no
    <braunr> it's really more complicated than that


# Further Examples

  * [[hurd/critique]]

  * [[IO_accounting]]

  * [[translators_set_up_by_untrusted_users]], and [[pagers]]

  * [[configure_max_command_line_length]]


## [[hurd/translator/exec]] server

### IRC, freenode, #hurd, 2013-08-05

    <teythoon> unzipping stuff in the exec server enables a dos on filesystem
      translators
    <teythoon> https://teythoon.cryptobitch.de/gsoc/heap/hello-1g.bz2 is
      /hurd/hello padded with a gig of zeros, compressed with bzip2
    <teythoon> if set as an passive translator, it stalls other requests to the
      filesystem, at least it does if ext2fs is used
    <braunr> teythoon: ?
    <braunr> teythoon: what's the dos here ?
    <teythoon> I can prevent you from doing anything with the root filesystem
    <teythoon> I'm kind of surprised myself, maybe a lock is held during the
      exec of the translator?
    <teythoon> the filesystem the hello-1g.bz2 translator is bound to is
      affected
    <braunr> teythoon: i don't understand
    <braunr> have you tried starting something from another file system ?
    <braunr> the lock may simply be in the exec server itself
    <teythoon> no, starting other things works fine
    <teythoon> but on the other hand, a find / is stalled
    <braunr> :/
    <braunr> *sigh*
    <teythoon> don't worry
    <teythoon> there is a solution :p
    <braunr> :)
    <teythoon> and it only requires deleting code