1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
|
[[!meta copyright="Copyright © 2015, 2016, 2018 Free Software Foundation,
Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled
[[GNU Free Documentation License|/fdl]]."]]"""]]
[[!meta title="Physical memory management"]]
[[!template id=highlight text="""/!\ Obsolete /!\
---
This is no longer valid as a Google Summer of Code project; it was basically
done by Richard."""]]
GNU Mach is currently suffering from severe limitations caused by the way
it manages physical memory. For example, since it requires pages to be mapped
in kernel space in order to be used, the maximum amount of usable physical
memory is currently around 800MB (or 1.8GB if a 2/2 split is set). And
because the page allocator is unable to easily return blocks of contiguous
pages, the kernel has to use virtual memory to provide contiguity.
But the kernel virtual space is separate from the direct mapping of
physical memory, so the larger it is, the less physical pages available.
The size of the kernel space is currently around 200MB, with around 100MB
for kernel objects. This small size prevents the system from achieving
scalability, since a panic occurs when the kernel is unable to allocate
a kernel object such as a port. In addition, the kernel uses mainly tables
to store IPC rights. When a table is full, it is enlarged through a kernel
specific version of realloc(). When a file system starts managing many
files (e.g. because some of their content is cached in physical memory),
these tables can get big enough to make realloc() fail because of
fragmentation.
The goal of this project is to make as much physical memory available as
possible for both the kernel and applications, by rewriting the page
allocator into a buddy allocator to support contiguous block allocations,
using it directly instead of virtual memory as the backend of the slab
allocator for kernel objects, and, if time allows it, transform IPC right
tables (e.g. into radix trees) and get rid of realloc().
This project requires a good understanding of virtual memory (both physical
mappings at the MMU level and virtual mappings at the VM level), and strong
skills in C programming. Note that some work has already been done in the
X15 project about this, and can be reused as a reference.
Useful links :
* <https://www.sceen.net/mapping-physical-memory-directly/>
* <http://git.sceen.net/rbraun/x15.git/>
* <https://git.sceen.net/rbraun/librbraun.git/plain/rdxtree.h>
# Current Status 2018-10-25
Recent GNU Mach releases have replaced the physical allocator for a more standard buddy allocator, and the kernel can service much more memory for its objects, making kernel memory exhaustion very unlikely. Justus Winter contributed to this project by using radix-tree based IPC spaces and various bug fixes (page queue management) to the kernel. Richard Braun helped see the project through to completion.
For those that are curious about the details, the buddy allocator provides efficient allocation of continuous chunks of memory at the cost of high internal fragmentation. To avoid the fragmentation overhead, we do not use it directly, but through the slab allocator that is already used for most of the allocations inside the kernel.
The "biosmem" provides three different segments, DMA, DIRECTMAP, and HIGHMEM. The DMA segment provides memory at low addresses for drivers that need buffers that devices can read from and write into using DMA.
The DIRECTMAP segment is directly mapped into the kernels address space. We can use this segment for kernel objects and can directly access it in inter-task memory copies. You can see its size capped at 880M because the kernel is using a 1G kernel / 3G userspace split (the stock Debian kernel uses a 2/2 split to maximize the amount of mappable physical memory).
We cannot use the HIGHMEM segment for kernel objects, but we can give it out to userspace tasks.
You can then see the drivers probing for devices, modules being loaded and the Hurd server bootstrap running. The system then deadlocks, with a custom message being emitted right before that.
Useful links:
* <http://lists.gnu.org/archive/html/bug-hurd/2016-02/msg00118.html>
* <http://lists.gnu.org/archive/html/bug-hurd/2015-09/msg00081.html>
|
