The mmap
call is generally supported on GNU Hurd, as indicated by
_POSIX_MAPPED_FILES
(sysconf (_SC_MAPPED_FILES)
).
Flags
Flags contain mapping type, sharing type and options.
Mapping type (must choose one and only one of these).
MAP_FILE
(Mapped from a file or device.)MAP_ANON
/MAP_ANONYMOUS
(Allocated from anonymous virtual memory.)
Even though it is not defined to zero (it is for the Linux kernel; why not for us?),
MAP_FILE
is the default and can be omitted.Sharing types (must choose one and only one of these).
MAP_SHARED
(Share changes.)MAP_PRIVATE
(Changes private; copy pages on write.)MAP_COPY
(Virtual copy of region at mapping time.)
For us,
MAP_PRIVATE
is the default (is defined to zero), for the Linux kernel, one ofMAP_SHARED
orMAP_PRIVATE
has to be specified explicitly.The Linux kernel does not support
MAP_COPY
, and as per the comment inelf/dl-load.c
,MAP_PRIVATE | MAP_DENYWRITE
is Linux' replacement forMAP_COPY
. However,MAP_DENYWRITE
is defunct (mmap
manpage).In contrast to
MAP_COPY
, forMAP_PRIVATE
it is unspecified whether changes made to the file after themmap
call are visible in the mapped region (mmap
manpage).MAP_COPY
:What exactly is that? `elf/dl-load.c` has some explanation. <http://lkml.indiana.edu/hypermail/linux/kernel/0110.1/1506.html> It is only handled in `dl-sysdep.c`, when `flags & (MAP_COPY|MAP_PRIVATE)` is used for <a href="../../microkernel/mach/interface/vm_map/">`vm map`</a>'s `copy` parameter, and `mmap.c` uses `! (flags & MAP_SHARED)` instead, which seems inconsistent? Usage in glibc: * `catgets/open_catalog.c:__open_catalog`, `locale/loadlocale.c:_nl_load_locale`: *Linux seems to lack read-only copy-on-write.*
MAP_TYPE
(Mask for type field./Mask for type of mapping.)In
bits/mman.h
this is described and defined to be a mask for the mapping type, in thebits/mman.h
files corresponding to Linux kernel it is described an defined to be a mask for the sharing type.Other flags.
fa81d5e7226ff1297277af60e5703330Implementation
Essentially, `mmap` is implemented by means of `io map` (not for `MAP_ANON`) followed by `vm map`. There are two implementations: `sysdeps/mach/hurd/mmap.c` (main implementation) and `sysdeps/mach/hurd/dl-sysdep.c` (*Minimal mmap implementation sufficient for initial loading of shared libraries.*).
Do we implement that (equivalently to `MAP_ANON`)?mmap ("/dev/zero")
Mapping Size
From the `mmap` manpage:A file is mapped in multiples of the page size. For a file that is not a multiple of the page size, the remaining memory is zeroed when mapped, and writes to that region are not written out to the file. The effect of changing the size of the underlying file of a mapping on the pages that correspond to added or removed regions of the file is unspecified.
Do we implement that?
Use of a Mapped Region
From the
mmap
manpage:Use of a mapped region can result in these signals: SIGSEGV Attempted write into a region mapped as read-only. SIGBUS Attempted access to a portion of the buffer that does not correspond to the file (for example, beyond the end of the file, including the case where another process has truncated the file).
Do we implement that?
Usage in glibc itself
Review of
mmap
usage in generic bits of glibc (omitted:nptl/
,sysdeps/unix/sparc/
,sysdepts/unix/sysv/linux/
), based on a1bcbd4035ac2483dc10da150d4db46f3e1744f8 (2012-03-11).MAP_FILE
is the interesting case;MAP_ANON
is generally fine. Some of themmap
usages in glibc have fallback code for theMAP_FAILED
case, some do not.catgets/open_catalog.c: (struct catalog_obj *) __mmap (NULL, st.st_size, PROT_READ, catgets/open_catalog.c- MAP_FILE|MAP_COPY, fd, 0);
Has fallback for
MAP_FAILED
.elf/cache.c: = mmap (NULL, st.st_size, PROT_READ, MAP_PRIVATE, fd, 0); elf/cache.c: = mmap (NULL, aux_cache_size, PROT_READ, MAP_PRIVATE, fd, 0);
No fallback for
MAP_FAILED
.elf/dl-load.c: l->l_map_start = (ElfW(Addr)) __mmap ((void *) mappref, maplength, elf/dl-load.c- c->prot, elf/dl-load.c- MAP_COPY|MAP_FILE, elf/dl-load.c- fd, c->mapoff); elf/dl-load.c: && (__mmap ((void *) (l->l_addr + c->mapstart), elf/dl-load.c- c->mapend - c->mapstart, c->prot, elf/dl-load.c- MAP_FIXED|MAP_COPY|MAP_FILE, elf/dl-load.c- fd, c->mapoff)
No fallback for
MAP_FAILED
.elf/dl-misc.c: result = __mmap (NULL, *sizep, prot, elf/dl-misc.c-#ifdef MAP_COPY elf/dl-misc.c- MAP_COPY elf/dl-misc.c-#else elf/dl-misc.c- MAP_PRIVATE elf/dl-misc.c-#endif elf/dl-misc.c-#ifdef MAP_FILE elf/dl-misc.c- | MAP_FILE elf/dl-misc.c-#endif elf/dl-misc.c- , fd, 0);
No fallback for
MAP_FAILED
.elf/dl-profile.c: addr = (struct gmon_hdr *) __mmap (NULL, expected_size, PROT_READ|PROT_WRITE, elf/dl-profile.c- MAP_SHARED|MAP_FILE, fd, 0);
No fallback for
MAP_FAILED
.elf/readlib.c: file_contents = mmap (0, statbuf.st_size, PROT_READ, MAP_SHARED, elf/readlib.c- fileno (file), 0);
No fallback for
MAP_FAILED
.elf/sprof.c: result->symbol_map = mmap (NULL, max_offset - min_offset, elf/sprof.c- PROT_READ, MAP_SHARED|MAP_FILE, symfd, elf/sprof.c- min_offset); elf/sprof.c: addr = mmap (NULL, st.st_size, PROT_READ, MAP_SHARED|MAP_FILE, fd, 0);
No fallback for
MAP_FAILED
.iconv/gconv_cache.c: gconv_cache = __mmap (NULL, cache_size, PROT_READ, MAP_SHARED, fd, 0); iconv/iconv_charmap.c: && ((addr = mmap (NULL, st.st_size, PROT_READ, MAP_PRIVATE, iconv/iconv_charmap.c- fd, 0)) != MAP_FAILED)) iconv/iconv_prog.c: && ((addr = mmap (NULL, st.st_size, PROT_READ, MAP_PRIVATE, iconv/iconv_prog.c- fd, 0)) != MAP_FAILED))
Have fallback for
MAP_FAILED
.intl/loadmsgcat.c: data = (struct mo_file_header *) mmap (NULL, size, PROT_READ, intl/loadmsgcat.c- MAP_PRIVATE, fd, 0);
Has fallback for
MAP_FAILED
.libio/fileops.c: p = __mmap (NULL, st.st_size, PROT_READ, MAP_SHARED, libio/fileops.c- fp->_fileno, 0); libio/fileops.c: p = __mmap (NULL, st.st_size, PROT_READ, MAP_SHARED, fp->_fileno, 0);
Has fallback for
MAP_FAILED
.locale/loadarchive.c: result = __mmap64 (NULL, mapsize, PROT_READ, MAP_FILE|MAP_COPY, fd, 0); locale/loadarchive.c: result = __mmap64 (NULL, mapsize, PROT_READ, MAP_FILE|MAP_COPY, locale/loadarchive.c- fd, 0); locale/loadarchive.c: addr = __mmap64 (NULL, to - from, PROT_READ, MAP_FILE|MAP_COPY, locale/loadarchive.c- fd, from);
Some have fallback for
MAP_FAILED
.locale/programs/locale.c: void *mapped = mmap64 (NULL, st.st_size, PROT_READ, locale/programs/locale.c- MAP_SHARED, fd, 0); locale/programs/locale.c: && ((mapped = mmap64 (NULL, st.st_size, PROT_READ, locale/programs/locale.c- MAP_SHARED, fd, 0)) locale/programs/locale.c: addr = mmap64 (NULL, len, PROT_READ, MAP_SHARED, fd, 0); locale/programs/locarchive.c: void *p = mmap64 (NULL, RESERVE_MMAP_SIZE, PROT_NONE, MAP_SHARED, fd, 0); locale/programs/locarchive.c: p = mmap64 (p, total, PROT_READ | PROT_WRITE, MAP_SHARED | xflags, fd, 0); locale/programs/locarchive.c: void *p = mmap64 (ah->addr + start, st.st_size - start, locale/programs/locarchive.c- PROT_READ | PROT_WRITE, MAP_SHARED | MAP_FIXED, locale/programs/locarchive.c- ah->fd, start); locale/programs/locarchive.c: ah->addr = mmap64 (ah->addr, st.st_size, PROT_READ | PROT_WRITE, locale/programs/locarchive.c- MAP_SHARED | MAP_FIXED, ah->fd, 0); locale/programs/locarchive.c: ah->addr = mmap64 (NULL, st.st_size, PROT_READ | PROT_WRITE, locale/programs/locarchive.c- MAP_SHARED, ah->fd, 0); locale/programs/locarchive.c: p = mmap64 (p, total, PROT_READ | PROT_WRITE, MAP_SHARED | xflags, fd, 0); locale/programs/locarchive.c: ah->addr = mmap64 (p, st.st_size, PROT_READ | (readonly ? 0 : PROT_WRITE), locale/programs/locarchive.c- MAP_SHARED | xflags, fd, 0); locale/programs/locarchive.c: data[cnt].addr = mmap64 (NULL, st.st_size, PROT_READ, MAP_SHARED, locale/programs/locarchive.c- fd, 0);
No fallback for
MAP_FAILED
.nscd/connections.c: else if ((mem = mmap (NULL, dbs[cnt].max_db_size, nscd/connections.c- PROT_READ | PROT_WRITE, nscd/connections.c- MAP_SHARED, fd, 0)) nscd/connections.c: || (mem = mmap (NULL, dbs[cnt].max_db_size, nscd/connections.c- PROT_READ | PROT_WRITE, nscd/connections.c- MAP_SHARED, fd, 0)) == MAP_FAILED) nscd/nscd_helper.c: void *mapping = __mmap (NULL, mapsize, PROT_READ, MAP_SHARED, mapfd, 0);
No fallback for
MAP_FAILED
.nss/makedb.c: const struct nss_db_header *header = mmap (NULL, st.st_size, PROT_READ, nss/makedb.c- MAP_PRIVATE|MAP_POPULATE, fd, 0); nss/nss_db/db-open.c: mapping->header = mmap (NULL, header.allocate, PROT_READ, nss/nss_db/db-open.c- MAP_PRIVATE, fd, 0);
No fallback for
MAP_FAILED
.posix/tst-mmap.c: ptr = mmap (NULL, 1000, PROT_READ, MAP_SHARED, fd, ps); posix/tst-mmap.c: ptr = mmap64 (NULL, 1000, PROT_READ, MAP_SHARED, fd, ps); rt/tst-mqueue3.c: void *mem = mmap (NULL, ps, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); rt/tst-mqueue5.c: void *mem = mmap (NULL, ps, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); rt/tst-shm.c: mem = mmap (NULL, 4000, PROT_READ | PROT_WRITE, MAP_SHARED, fd, 0); stdio-common/tst-fmemopen.c: if ((mmap_data = (char *) mmap (NULL, fs.st_size, PROT_READ, stdio-common/tst-fmemopen.c- MAP_SHARED, fd, 0)) == MAP_FAILED)
No fallback for
MAP_FAILED
.io_map
FailureThis is the libnetfs:
io map
issue.tschwinge's current plan is to make the following cases do the same (if that is possible); probably by introducing a generic
mmap_or_read
function, that first triesmmap
(and that will succeed on Linux-based systems and also on Hurd-based, if it's backed by libdiskfs), and if that fails triesmmap
on anonymous memory and then fills it byread
ing the required data. This is also what the ?exec server is doing (and is the reason that the./true
invocation on libnetfs:io map
works, to my understanding): seeexec.c:prepare
, ifio_map
fails,e->filemap == MACH_PORT_NULL
; thenexec.c:map
(as invoked fromexec.c:load_section
,exec.c:check_elf
,exec.c:do_exec
, orhashexec.c:check_hashbang
) will useio_read
instead.Doing so potentially means reading in a lot of unused data -- but we probably can't do any better?
In parallel (or even alternatively?), it should be researched how Linux (or any other kernel) implements
mmap
on NFS and similar file systems, and then implement the same in libnetfs and/or nfs, etc.Here, also probably the whole mapping region has to be read (bug-hurd list archive) at
mmap
time. Then, onlyMAP_PRIVATE
(or rather:MAP_COPY
) is possible, but notMAP_SHARED
.