summaryrefslogtreecommitdiff
path: root/open_issues/implementing_hurd_on_top_of_another_system.mdwn
diff options
context:
space:
mode:
Diffstat (limited to 'open_issues/implementing_hurd_on_top_of_another_system.mdwn')
-rw-r--r--open_issues/implementing_hurd_on_top_of_another_system.mdwn420
1 files changed, 0 insertions, 420 deletions
diff --git a/open_issues/implementing_hurd_on_top_of_another_system.mdwn b/open_issues/implementing_hurd_on_top_of_another_system.mdwn
deleted file mode 100644
index cf3db0ab..00000000
--- a/open_issues/implementing_hurd_on_top_of_another_system.mdwn
+++ /dev/null
@@ -1,420 +0,0 @@
-[[!meta copyright="Copyright © 2010, 2011, 2012, 2013 Free Software Foundation,
-Inc."]]
-
-[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
-id="license" text="Permission is granted to copy, distribute and/or modify this
-document under the terms of the GNU Free Documentation License, Version 1.2 or
-any later version published by the Free Software Foundation; with no Invariant
-Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
-is included in the section entitled [[GNU Free Documentation
-License|/fdl]]."]]"""]]
-
-[[!tag open_issue_documentation]]
-
-It is possible to run Hurd stuff on top of another system instead of on Mach.
-One obvious variant is [[emulation]] (using [[hurd/running/QEMU]], for
-example), but
-doing that does not really integratable the Hurd guest into the host system.
-There is also a more direct way, more powerful, but it also has certain
-requirements to do it effectively.
-
-See also [[Mach_on_top_of_POSIX]].
-
-
-# IRC, freenode, #hurd, August / September 2010
-
- <marcusb> silver_hook: the Hurd can also refer to the interfaces of the
- filesystems etc, and a lot of that is really just server/client APIs that
- could be implemented on any system that has transferable rights to
- message capabilities.
- <marcusb> silver_hook: it's surprising how few systems *have* transferable
- rights, though!
- <marcusb> silver_hook: usually it is added as an afterthought
- <marcusb> and comes with restriction
- <youpi> marcusb: there's SCM_RIGHTS to transfer fds, which is quite often
- available
- <marcusb> youpi: yes, I know this as "fdpassing"
- <marcusb> youpi: it's described in the Stevens series even
- [...]
- <marcusb> ArneBab: well, let me put it this way. the Linux kernel has no
- interface to manipulate another tasks's virtual address space, ie you
- can't map/unmap stuff in another process
- <marcusb> ArneBab: you would have to use ptrace and load some stub code in
- that process to make that happen.
- <marcusb> ArneBab: so for complete transparent manipulation, you need a
- kernel module
- <marcusb> that is what the User Mode Linux kernel module does
- <marcusb> ArneBab: so say you use the User Mode Linux kernel module for
- that one feature. Then you can do everything that User Mode Linux can
- do, which, I assure you, includes running subhurds :)
- <marcusb> it can be a bit tricky to implement those features, but it is not
- harder than writing a kernel in the first place
- <ArneBab> So, if I got an admin to install User Mode Linux and Mach
- emulation, I’d get the flexibility (and independence from admin
- decisions) I have in the Hurd?
- <marcusb> ArneBab: one problem is that you still use Linux. For those who
- want to get rid of Linux for political reasons, that would mean complete
- failure
- <marcusb> ArneBab: if you have UML kernel module, you can implement Mach in
- user space
- <marcusb> ArneBab: in fact, John Tobey did this a couple of years ago, or
- started it
-
-[[Mach_on_top_of_POSIX]].
-
- <marcusb> ArneBab: or you can just implement parts of it and relay to Linux
- for the rest
- <marcusb> the point is, that if you don't care for kernel improvements, and
- are sufficiently happy with the translator stuff, it's not hard to bring
- the Hurd to Linux or BSD
-
-Continue reading about the [[benefits_of_a_native_Hurd_implementation]].
-
-
-# IRC, freenode, #hurd, 2010-12-28
-
- <antrik> kilobug: there is no real requirement for the Hurd to run on a
- microkernel... as long as the important mechanisms are provided (most
- notably external pagers and Mach IPC), the Hurd could run an top of
- pretty much any kernel...
- <antrik> whether it makes sense is another question of course :-)
- <antrik> though I must say that I'm more and more convinced running the
- Hurd on top of a monolithic kernel would actually be a useful approach
- for the time being...
-
-
-# IRC, freenode, #hurd, 2011-02-11
-
- <neal> marcus and I were discussing how to add Mach to Linux
- <neal> one could write a module to implement Mach IPC
- <neal> and another to implement Mach VM
- <neal> the big thing missing with Mach VM is the ability for a tracing
- process to easily map or unmap an inferior process's memory
- <antrik> neal: why would a tracing process need to map the inferior's
- memory?
- <neal> the simple answer is that is how it is done on Mach
- <antrik> neal: is it? not sure we are talking about the same thing
- here. GDB uses vm_read()/vm_write() to access the inferior's memory AFAIK
- <neal> on linux?
- <neal> I think it use /proc/pid/mem
- <antrik> on Hurd
- <neal> I'm talking about adding Mach to Linux
- <neal> by adding some functionality to Linux
- <neal> and then implementing a bunch in user space
- <antrik> yeah, but I don't understand the point about mapping inferior's
- memory :-(
- <antrik> what would be in user space?
- <neal> there are a number of different cut points
- <neal> one could imagine just using Linux's device drivers, CPU scheduler,
- memory management, etc.
- <neal> another possibility would be something higher where Hurd processes
- just use some Hurdish servers
- <antrik> neal: yeah, these are all options I have been considering... too
- bad I wasn't able to come to FOSDEM -- I'd love to have participated in
- this discussion :-(
- <antrik> neal: BTW, were you just discussing this as a hypothetical idea,
- or something you are seriously considering?
- <neal> I'm unlikely to work on it, sorry
- <antrik> didn't really expect that :-)
- <antrik> would be nice though if you could write up your conclusions...
-
-
-# IRC, freenode, #hurd, 2012-10-12
-
- <peo-xaci> do hurd system libraries make raw system calls ever
- (i.e. inlined syscall() / raw assembly)?
- <braunr> sure
- <peo-xaci> hmm, so a hurd emulation layer would need to use ptrace if it
- should be fool proof? :/
- <braunr> there is no real need for raw assembly, and the very syscalls are
- all available through macros
- <braunr> hum what are you trying to say ?
- <peo-xaci> well, if they are done through syscall, as a function, not a
- macro, then they can be intercepted with LD_PRELOAD
- <peo-xaci> so applications that do Hurd (Mach?) syscalls could work on
- f.e. Linux, if a special libc is injected into the program with
- LD_PRELOAD
- <peo-xaci> same thing with making standard Linux-applications go through
- the Hurd emulation layer
- <peo-xaci> without recompilation
- <mel-_> peo-xaci: the second direction is implemented in glibc.
- <mel-_> for the other direction, I personally see little use for it
- <braunr> peo-xaci: ok i misunderstood
- <braunr> peo-xaci: i don't think there is any truely direct syscall usage
- in the hurd
- <peo-xaci> hmm, I'm not sure I understand what directions you are referring
- to mel-_
- <braunr> peo-xaci: what are you trying to achieve ?
- <peo-xaci> I want to make the Hurd design more accessible by letting Hurd
- application run on the Linux kernel, preferably without
- recompilation. This would be done with a daemon that implements Mach and
- which all syscalls would go to.
- <peo-xaci> then, I also want so that standard Linux applications can go
- through that Mach daemon as well, if a special libc is preloaded
- <braunr> you might want to discuss this with antrik
- <peo-xaci> what I'm trying to figure out specifically is if there is some
- library/interface that glue Hurd with Mach and would be better suited to
- emulate than Mach? Mach seems to be more of an implementation detail to
- the hurd and not something an application would directly use.
- <braunr> yes, the various hurd libraries (libports and libpager mostly)
- <peo-xaci> From [http://www.gnu.org/software/hurd/hurd/libports.html]:
- "libports is not (at least, not for now) a generalization / abstraction
- of Mach ports to the functionality the Hurd needs, that is, it is not
- meant to provide an interface independently of the underlying
- microkernel."
- <peo-xaci> Is this still true?
- <peo-xaci> Does libpager abstract the rest?
- <peo-xaci> (and the other hurd libraries)
- <braunr> there is nothing that really abstracts the hurd from mach
- <braunr> for example, reference counting often happens here and there
- <braunr> and core libraries like glibc and libpthread heavily rely on it
- (through sysdeps specific code though)
- <braunr> libports and libpager are meant to simplify object manipulation
- for the former, and pager operations for the latter
- <peo-xaci> and applications, such as translators, often use Mach interfaces
- directly?
- <peo-xaci> correct?
- <braunr> depends on what often means
- <braunr> let's say they do
- <peo-xaci> :/ then it probably is better to emulate Mach after all
- <braunr> there was a mach on posix port a long time ago
- <peo-xaci> I thought applications were completely separated from the
- microkernel in use by the Hurd
- <braunr> that level of abstraction is pretty new
- <braunr> genode is the only system i know which does that
-
-[[microkernel/Genode]].
-
- <braunr> and it's still for "l4 variants"
- <pinotree> ah, thanks (i forgot that name)
- <antrik> braunr: Genode also runs on Linux and a few other non-L4
- environments IIRC
- <antrik> peo-xaci: I'm not sure binary emulation is really useful. rather,
- I'd recompile stuff as "regular" Linux executables, only using a special
- glibc
- <antrik> where the special glibc could be basically a port of the Hurd
- glibc communicating with the Mach emulation instead of real Mach; or it
- could do emulation at a higher level
- <antrik> a higher level emulation would be more complicated to implement,
- but more efficient, and allow better integration with the ordinary
- GNU/Linux environment
- <antrik> also note that any regular program could be recompiled against the
- HELL glibc to run in the Hurdish environment...
- <antrik> (well, glibc + hurd server libraries)
- <peo-xaci> I'm willing to accept that Hurd-application would need to be
- recompiled to work on the HELL
- <peo-xaci> but not Linux-applications :)
- <antrik> peo-xaci: if you happen to understand German, there is a fairly
- good overview in my thesis report ;-)
- <antrik> peo-xaci: there are no "Hurd applications" or "Linux applications"
- <peo-xaci> well, let me define what I mean by the terms: Hurd applications
- use Hurd-specific interfaces/syscalls, and Linux applications use
- Linux-specific interfaces/syscalls
- <antrik> a few programs use Linux-specific interfaces (and we probably
- can't run them in HELL just as we can't run them on actual Hurd); but all
- other programs work in any glibc environment
- <antrik> (usually in any POSIX environment in fact...)
- <antrik> peo-xaci: no sane application uses syscalls
- <peo-xaci> they do under the hood
- <peo-xaci> I have read about inlined syscalls
- <antrik> again, there are *some* applications using Linux-specific
- interfaces (sometimes because they are inherently bound to Linux
- features, sometimes unnecessarily)
- <antrik> so far there are no applications using Hurd-specific interfaces
- <peo-xaci> translators do?
- <peo-xaci> they are standard executables are they not?
- <peo-xaci> I would like so that translators also can be run in the HELL
- <antrik> I wouldn't consider them applications. all existing translators
- are pretty much components of the Hurd itself
- <peo-xaci> okay, it's a question about semantics, perhaps I should use
- another word than "applications" :)
- <peo-xaci> for me, applications are what have a main-function, or similar
- single entry point
- <braunr> hum
- <braunr> that's not a good enough definition
- <antrik> anyways, as I said, I think recompiling translators against a
- Hurdish glibc and ported translator libraries seems the most reasonable
- approach to me
- <braunr> let's say applications are userspace processes that make use of
- services provided by the operating system
- <braunr> translators being part of the operating system here
- <antrik> braunr: do you know whether the Mach-on-POSIX was actually
- functional, or just an abandoned experiment?...
- <antrik> (I don't remember hearing of it before...)
- <braunr> incomplete iirc
- <peo-xaci> braunr: still, when I've explained what I meant, even if I used
- the wrong term, then my previous statements should come in another light
- <peo-xaci> antrik / braunr: are you still interested in hearing my
- thoughts/ideas about HELL?
- <antrik> oh, there is more to come? ;-)
- <peo-xaci> yes! I don't think I have made myself completely understood :/
- <peo-xaci> what I envision is a HELL system that works on as low level as
- feasible, to make it possible to do almost anything that can be done on
- the real Hurd (except possibly testing hardware drivers and such very low
- level stuff).
- <braunr> sure
- <peo-xaci> I want it to be more than just allowing programs to access a
- virtual filesystem à la FUSE. My idea is that all user space system
- libraries/programs of the Hurd should be inside the HELL as well, and
- they should not be emulated.
- <peo-xaci> The system should at the very least be API compatible, so at the
- very most a recompilation is necessary.
- <peo-xaci> I also want so that GNU/Linux-programs can access the features
- of the HELL with little effort on the user. At most perhaps a script that
- wraps LD_PRELOADing has to be run on the binary. Best would be if it
- could work also with insane assembly programs using raw system calls, or
- if glibc happens to have some well hidden syscall being inlined to raw
- assembly code.
- <peo-xaci> And I think I have an idea on how an implementation could
- satisfy these things!
- <peo-xaci> By modifying the kernel and replace those syscalls that make
- sense for the Hurd/Mach
- <peo-xaci> with "the kernel", I meant Linux
- <braunr> it's possible but tedious and not very useful so better do that
- later
- <braunr> mach did something similar at its time
- <braunr> there was a syscall emulation library
- <peo-xaci> but isn't it about as much work as emulating the interface on
- user-level?
- <braunr> and the kernel cooperated so that unmodified unix binaries
- performing syscalls would actually jump to functions provided by that
- library, which generally made an RPC
- <peo-xaci> instead of a bunch of extern-declerations, one would put the
- symbols in the syscall table
- <braunr> define what "those syscalls that make sense for the Hurd/Mach"
- actually means
- <peo-xaci> open/close, for example
- <braunr> otherwise i don't see another better way than what the old mach
- folks did
- <braunr> well, with that old, but existing support, your open would perform
- a syscall
- <braunr> the kernel would catch it and redirect the caller to its syscall
- emulation library
- <braunr> which would call the open RPC instead
- <peo-xaci> wait, so this "existing support" you're talking about; is this a
- module for the Linux kernel (or a fork, or something else)?
- <peo-xaci> where can I find it?
- <braunr> no
- <braunr> it was for mach
- <braunr> in order to run unmodified unix binaries
- <braunr> the opposite of what you're trying to do
- <peo-xaci> ah okay
- <braunr> well
- <braunr> not really either :)
- <peo-xaci> does posix/unix define a standard for how a syscall table should
- look like, to allow binary syscall compatibility?
- <braunr> absolutely not
- <peo-xaci> so how could this mach module run any unmodified unix binary? if
- they expected different sys calls at different offsets?
- <braunr> posix specifically (and very early) states that it almost forbids
- itself to deal with anything regarding to ABIs
- <braunr> depends
- <braunr> since it was old, there weren't that many unix systems
- <braunr> and even today, there are techniques like those used by netbsd
- (and many other actually)
- <braunr> that are able to inspect the binary and load a syscall emulation
- environment depending on its exposed ABI
- <braunr> e.g. file on an executable states which system it's for
- <peo-xaci> hmm, I'm not sure how a kernel would implement that in
- practice.. I thought these things were so hard coded and dependent on raw
- memory reads that it would not be possible
- <braunr> but i really think it's not worth the time for your project
- <peo-xaci> to be honest I have virtually no experience of practical kernel
- programming
- <braunr> with an LDT on x86 for example
- <braunr> no, there is really not that much hardcoded
- <braunr> quite the contrary
- <braunr> there is a lot of runtime detection today
- <peo-xaci> well I mean how the syscall table is read
- <braunr> it's not read
- <peo-xaci> it's read to find the function pointer to the syscall handler in
- the kernel?
- <braunr> no
- <braunr> that's the really basic approach
- <braunr> (and in practice it can happen of course)
- <braunr> what really happens is that, for example, on linux, the user space
- system call code is loaded as a virtual shared library
- <braunr> use ldd on an executable to see it
- <braunr> this virtual object provides code that, depending on what the
- kernel has detected, will use the appropriate method to perform a system
- call
- <peo-xaci> but this user space system calls need to make some kind of cpu
- interupt to communicate with the kernel, right?
- <braunr> the glibc itself has no idea how a system call will look like in
- the end
- <braunr> yes
- <peo-xaci> an assembler programmer would be able to get around this glue
- code?
- <braunr> that's precisely what is embedded in this virtual library
- <braunr> it could yes
- <braunr> i think even when sysenter/sysexit is supported, legacy traps are
- still implemented to support old binaries
- <braunr> but then all these different entry points will lead to the same
- code inside the kernel
- <peo-xaci> but when the glue code is used, then its API compatible, and
- then I can understand that the kernel can allow different syscall
- implementations for different executables
- <braunr> what glue code ?
- <peo-xaci> what you talked about above "the user space system call code is
- loaded as a virtual shared library"
- <braunr> let's call it vdso
-
-[[vDSO]]
-
- <peo-xaci> thanks, I looked it up on Wikipedia and understand immediately
- :P
- <peo-xaci> so VDSOs are provided by the kernel, not a regular library file,
- right?
- <vdox2> What does HELL stand for :) ?
- <dardevelin> vdox2, Hurd Emulation Layer for Linux
- <vdox2> dardevelin: thanks
- <braunr> peo-xaci: yes
- <antrik> peo-xaci: I believe your goals are conflicting. a low-level
- implementation makes it basically impossible to interact between the HELL
- environment and the GNU/Linux environment in any meaningful way. to allow
- such interaction, you *have* to have some glue at a higher semantic level
- <braunr> agreed
- <antrik> peo-xaci: BTW, if you want regular Linux binaries to get somehow
- redirected to access HELL facilities, there is already a framework (don't
- remember the name right now) that allows this kind of system call
- redirection on Linux
- <antrik> (it can run both through LD_PRELOAD or as a kernel module -- where
- obviously only the latter would allow raw system call redirection... but
- TBH, I don't think that's worthwhile anyways. the rare cases where
- programs use raw system calls are usually for extremely system-specific
- stuff anyways...)
- <antrik> ViewOS is the name
- <antrik> err... View-OS I mean
- <antrik> or maybe View OS ? ;-)
- <antrik> whatever, you'll find it :-)
-
-[[Virtual_Square_View-OS]].
-
- <antrik> I'm not sure it's really worthwhile to use this either
- though... the most meaningful interaction is probably at the FS level,
- and that can be done with FUSE
- <antrik> OHOH, View-OS probably allows doing more interesting stuff that
- FUSE, such as modyfing the way the VFS works...
- <antrik> OTOH
- <antrik> so it could expose more of the Hurd features, at least in theory
-
-
-## IRC, freenode, #hurd, 2012-10-13
-
- <peo-xaci> antrik / braunr: thanks for your input! I'm not entirely
- convinced though. :) I will probably return to this project once I have
- acquired a lot more knowledge about low level stuff. I want to see for
- myself whether a low level HELL is not feasible. :P
- <braunr> peo-xaci: what's the point of a low level hell ?
- <peo-xaci> more Hurd code can be tested in the hell, if the hell is at a
- low level
- <peo-xaci> at a higher level, some Hurd code cannot run, because the
- interfaces they use would not be accessible from the higher level
- emulation
- <antrik> peo-xaci: I never said it's not possible. I actually said it would
- be easier to do. I just said you can't do it low level *and* have
- meaningful interaction with the host system
- <peo-xaci> I don't understand why
- <braunr> peo-xaci: i really don't see what you want to achieve with low
- level support
- <braunr> what would be unavailable with a higher level approach ?