diff options
Diffstat (limited to 'open_issues/implementing_hurd_on_top_of_another_system.mdwn')
-rw-r--r-- | open_issues/implementing_hurd_on_top_of_another_system.mdwn | 420 |
1 files changed, 0 insertions, 420 deletions
diff --git a/open_issues/implementing_hurd_on_top_of_another_system.mdwn b/open_issues/implementing_hurd_on_top_of_another_system.mdwn deleted file mode 100644 index cf3db0ab..00000000 --- a/open_issues/implementing_hurd_on_top_of_another_system.mdwn +++ /dev/null @@ -1,420 +0,0 @@ -[[!meta copyright="Copyright © 2010, 2011, 2012, 2013 Free Software Foundation, -Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -[[!tag open_issue_documentation]] - -It is possible to run Hurd stuff on top of another system instead of on Mach. -One obvious variant is [[emulation]] (using [[hurd/running/QEMU]], for -example), but -doing that does not really integratable the Hurd guest into the host system. -There is also a more direct way, more powerful, but it also has certain -requirements to do it effectively. - -See also [[Mach_on_top_of_POSIX]]. - - -# IRC, freenode, #hurd, August / September 2010 - - <marcusb> silver_hook: the Hurd can also refer to the interfaces of the - filesystems etc, and a lot of that is really just server/client APIs that - could be implemented on any system that has transferable rights to - message capabilities. - <marcusb> silver_hook: it's surprising how few systems *have* transferable - rights, though! - <marcusb> silver_hook: usually it is added as an afterthought - <marcusb> and comes with restriction - <youpi> marcusb: there's SCM_RIGHTS to transfer fds, which is quite often - available - <marcusb> youpi: yes, I know this as "fdpassing" - <marcusb> youpi: it's described in the Stevens series even - [...] - <marcusb> ArneBab: well, let me put it this way. the Linux kernel has no - interface to manipulate another tasks's virtual address space, ie you - can't map/unmap stuff in another process - <marcusb> ArneBab: you would have to use ptrace and load some stub code in - that process to make that happen. - <marcusb> ArneBab: so for complete transparent manipulation, you need a - kernel module - <marcusb> that is what the User Mode Linux kernel module does - <marcusb> ArneBab: so say you use the User Mode Linux kernel module for - that one feature. Then you can do everything that User Mode Linux can - do, which, I assure you, includes running subhurds :) - <marcusb> it can be a bit tricky to implement those features, but it is not - harder than writing a kernel in the first place - <ArneBab> So, if I got an admin to install User Mode Linux and Mach - emulation, I’d get the flexibility (and independence from admin - decisions) I have in the Hurd? - <marcusb> ArneBab: one problem is that you still use Linux. For those who - want to get rid of Linux for political reasons, that would mean complete - failure - <marcusb> ArneBab: if you have UML kernel module, you can implement Mach in - user space - <marcusb> ArneBab: in fact, John Tobey did this a couple of years ago, or - started it - -[[Mach_on_top_of_POSIX]]. - - <marcusb> ArneBab: or you can just implement parts of it and relay to Linux - for the rest - <marcusb> the point is, that if you don't care for kernel improvements, and - are sufficiently happy with the translator stuff, it's not hard to bring - the Hurd to Linux or BSD - -Continue reading about the [[benefits_of_a_native_Hurd_implementation]]. - - -# IRC, freenode, #hurd, 2010-12-28 - - <antrik> kilobug: there is no real requirement for the Hurd to run on a - microkernel... as long as the important mechanisms are provided (most - notably external pagers and Mach IPC), the Hurd could run an top of - pretty much any kernel... - <antrik> whether it makes sense is another question of course :-) - <antrik> though I must say that I'm more and more convinced running the - Hurd on top of a monolithic kernel would actually be a useful approach - for the time being... - - -# IRC, freenode, #hurd, 2011-02-11 - - <neal> marcus and I were discussing how to add Mach to Linux - <neal> one could write a module to implement Mach IPC - <neal> and another to implement Mach VM - <neal> the big thing missing with Mach VM is the ability for a tracing - process to easily map or unmap an inferior process's memory - <antrik> neal: why would a tracing process need to map the inferior's - memory? - <neal> the simple answer is that is how it is done on Mach - <antrik> neal: is it? not sure we are talking about the same thing - here. GDB uses vm_read()/vm_write() to access the inferior's memory AFAIK - <neal> on linux? - <neal> I think it use /proc/pid/mem - <antrik> on Hurd - <neal> I'm talking about adding Mach to Linux - <neal> by adding some functionality to Linux - <neal> and then implementing a bunch in user space - <antrik> yeah, but I don't understand the point about mapping inferior's - memory :-( - <antrik> what would be in user space? - <neal> there are a number of different cut points - <neal> one could imagine just using Linux's device drivers, CPU scheduler, - memory management, etc. - <neal> another possibility would be something higher where Hurd processes - just use some Hurdish servers - <antrik> neal: yeah, these are all options I have been considering... too - bad I wasn't able to come to FOSDEM -- I'd love to have participated in - this discussion :-( - <antrik> neal: BTW, were you just discussing this as a hypothetical idea, - or something you are seriously considering? - <neal> I'm unlikely to work on it, sorry - <antrik> didn't really expect that :-) - <antrik> would be nice though if you could write up your conclusions... - - -# IRC, freenode, #hurd, 2012-10-12 - - <peo-xaci> do hurd system libraries make raw system calls ever - (i.e. inlined syscall() / raw assembly)? - <braunr> sure - <peo-xaci> hmm, so a hurd emulation layer would need to use ptrace if it - should be fool proof? :/ - <braunr> there is no real need for raw assembly, and the very syscalls are - all available through macros - <braunr> hum what are you trying to say ? - <peo-xaci> well, if they are done through syscall, as a function, not a - macro, then they can be intercepted with LD_PRELOAD - <peo-xaci> so applications that do Hurd (Mach?) syscalls could work on - f.e. Linux, if a special libc is injected into the program with - LD_PRELOAD - <peo-xaci> same thing with making standard Linux-applications go through - the Hurd emulation layer - <peo-xaci> without recompilation - <mel-_> peo-xaci: the second direction is implemented in glibc. - <mel-_> for the other direction, I personally see little use for it - <braunr> peo-xaci: ok i misunderstood - <braunr> peo-xaci: i don't think there is any truely direct syscall usage - in the hurd - <peo-xaci> hmm, I'm not sure I understand what directions you are referring - to mel-_ - <braunr> peo-xaci: what are you trying to achieve ? - <peo-xaci> I want to make the Hurd design more accessible by letting Hurd - application run on the Linux kernel, preferably without - recompilation. This would be done with a daemon that implements Mach and - which all syscalls would go to. - <peo-xaci> then, I also want so that standard Linux applications can go - through that Mach daemon as well, if a special libc is preloaded - <braunr> you might want to discuss this with antrik - <peo-xaci> what I'm trying to figure out specifically is if there is some - library/interface that glue Hurd with Mach and would be better suited to - emulate than Mach? Mach seems to be more of an implementation detail to - the hurd and not something an application would directly use. - <braunr> yes, the various hurd libraries (libports and libpager mostly) - <peo-xaci> From [http://www.gnu.org/software/hurd/hurd/libports.html]: - "libports is not (at least, not for now) a generalization / abstraction - of Mach ports to the functionality the Hurd needs, that is, it is not - meant to provide an interface independently of the underlying - microkernel." - <peo-xaci> Is this still true? - <peo-xaci> Does libpager abstract the rest? - <peo-xaci> (and the other hurd libraries) - <braunr> there is nothing that really abstracts the hurd from mach - <braunr> for example, reference counting often happens here and there - <braunr> and core libraries like glibc and libpthread heavily rely on it - (through sysdeps specific code though) - <braunr> libports and libpager are meant to simplify object manipulation - for the former, and pager operations for the latter - <peo-xaci> and applications, such as translators, often use Mach interfaces - directly? - <peo-xaci> correct? - <braunr> depends on what often means - <braunr> let's say they do - <peo-xaci> :/ then it probably is better to emulate Mach after all - <braunr> there was a mach on posix port a long time ago - <peo-xaci> I thought applications were completely separated from the - microkernel in use by the Hurd - <braunr> that level of abstraction is pretty new - <braunr> genode is the only system i know which does that - -[[microkernel/Genode]]. - - <braunr> and it's still for "l4 variants" - <pinotree> ah, thanks (i forgot that name) - <antrik> braunr: Genode also runs on Linux and a few other non-L4 - environments IIRC - <antrik> peo-xaci: I'm not sure binary emulation is really useful. rather, - I'd recompile stuff as "regular" Linux executables, only using a special - glibc - <antrik> where the special glibc could be basically a port of the Hurd - glibc communicating with the Mach emulation instead of real Mach; or it - could do emulation at a higher level - <antrik> a higher level emulation would be more complicated to implement, - but more efficient, and allow better integration with the ordinary - GNU/Linux environment - <antrik> also note that any regular program could be recompiled against the - HELL glibc to run in the Hurdish environment... - <antrik> (well, glibc + hurd server libraries) - <peo-xaci> I'm willing to accept that Hurd-application would need to be - recompiled to work on the HELL - <peo-xaci> but not Linux-applications :) - <antrik> peo-xaci: if you happen to understand German, there is a fairly - good overview in my thesis report ;-) - <antrik> peo-xaci: there are no "Hurd applications" or "Linux applications" - <peo-xaci> well, let me define what I mean by the terms: Hurd applications - use Hurd-specific interfaces/syscalls, and Linux applications use - Linux-specific interfaces/syscalls - <antrik> a few programs use Linux-specific interfaces (and we probably - can't run them in HELL just as we can't run them on actual Hurd); but all - other programs work in any glibc environment - <antrik> (usually in any POSIX environment in fact...) - <antrik> peo-xaci: no sane application uses syscalls - <peo-xaci> they do under the hood - <peo-xaci> I have read about inlined syscalls - <antrik> again, there are *some* applications using Linux-specific - interfaces (sometimes because they are inherently bound to Linux - features, sometimes unnecessarily) - <antrik> so far there are no applications using Hurd-specific interfaces - <peo-xaci> translators do? - <peo-xaci> they are standard executables are they not? - <peo-xaci> I would like so that translators also can be run in the HELL - <antrik> I wouldn't consider them applications. all existing translators - are pretty much components of the Hurd itself - <peo-xaci> okay, it's a question about semantics, perhaps I should use - another word than "applications" :) - <peo-xaci> for me, applications are what have a main-function, or similar - single entry point - <braunr> hum - <braunr> that's not a good enough definition - <antrik> anyways, as I said, I think recompiling translators against a - Hurdish glibc and ported translator libraries seems the most reasonable - approach to me - <braunr> let's say applications are userspace processes that make use of - services provided by the operating system - <braunr> translators being part of the operating system here - <antrik> braunr: do you know whether the Mach-on-POSIX was actually - functional, or just an abandoned experiment?... - <antrik> (I don't remember hearing of it before...) - <braunr> incomplete iirc - <peo-xaci> braunr: still, when I've explained what I meant, even if I used - the wrong term, then my previous statements should come in another light - <peo-xaci> antrik / braunr: are you still interested in hearing my - thoughts/ideas about HELL? - <antrik> oh, there is more to come? ;-) - <peo-xaci> yes! I don't think I have made myself completely understood :/ - <peo-xaci> what I envision is a HELL system that works on as low level as - feasible, to make it possible to do almost anything that can be done on - the real Hurd (except possibly testing hardware drivers and such very low - level stuff). - <braunr> sure - <peo-xaci> I want it to be more than just allowing programs to access a - virtual filesystem à la FUSE. My idea is that all user space system - libraries/programs of the Hurd should be inside the HELL as well, and - they should not be emulated. - <peo-xaci> The system should at the very least be API compatible, so at the - very most a recompilation is necessary. - <peo-xaci> I also want so that GNU/Linux-programs can access the features - of the HELL with little effort on the user. At most perhaps a script that - wraps LD_PRELOADing has to be run on the binary. Best would be if it - could work also with insane assembly programs using raw system calls, or - if glibc happens to have some well hidden syscall being inlined to raw - assembly code. - <peo-xaci> And I think I have an idea on how an implementation could - satisfy these things! - <peo-xaci> By modifying the kernel and replace those syscalls that make - sense for the Hurd/Mach - <peo-xaci> with "the kernel", I meant Linux - <braunr> it's possible but tedious and not very useful so better do that - later - <braunr> mach did something similar at its time - <braunr> there was a syscall emulation library - <peo-xaci> but isn't it about as much work as emulating the interface on - user-level? - <braunr> and the kernel cooperated so that unmodified unix binaries - performing syscalls would actually jump to functions provided by that - library, which generally made an RPC - <peo-xaci> instead of a bunch of extern-declerations, one would put the - symbols in the syscall table - <braunr> define what "those syscalls that make sense for the Hurd/Mach" - actually means - <peo-xaci> open/close, for example - <braunr> otherwise i don't see another better way than what the old mach - folks did - <braunr> well, with that old, but existing support, your open would perform - a syscall - <braunr> the kernel would catch it and redirect the caller to its syscall - emulation library - <braunr> which would call the open RPC instead - <peo-xaci> wait, so this "existing support" you're talking about; is this a - module for the Linux kernel (or a fork, or something else)? - <peo-xaci> where can I find it? - <braunr> no - <braunr> it was for mach - <braunr> in order to run unmodified unix binaries - <braunr> the opposite of what you're trying to do - <peo-xaci> ah okay - <braunr> well - <braunr> not really either :) - <peo-xaci> does posix/unix define a standard for how a syscall table should - look like, to allow binary syscall compatibility? - <braunr> absolutely not - <peo-xaci> so how could this mach module run any unmodified unix binary? if - they expected different sys calls at different offsets? - <braunr> posix specifically (and very early) states that it almost forbids - itself to deal with anything regarding to ABIs - <braunr> depends - <braunr> since it was old, there weren't that many unix systems - <braunr> and even today, there are techniques like those used by netbsd - (and many other actually) - <braunr> that are able to inspect the binary and load a syscall emulation - environment depending on its exposed ABI - <braunr> e.g. file on an executable states which system it's for - <peo-xaci> hmm, I'm not sure how a kernel would implement that in - practice.. I thought these things were so hard coded and dependent on raw - memory reads that it would not be possible - <braunr> but i really think it's not worth the time for your project - <peo-xaci> to be honest I have virtually no experience of practical kernel - programming - <braunr> with an LDT on x86 for example - <braunr> no, there is really not that much hardcoded - <braunr> quite the contrary - <braunr> there is a lot of runtime detection today - <peo-xaci> well I mean how the syscall table is read - <braunr> it's not read - <peo-xaci> it's read to find the function pointer to the syscall handler in - the kernel? - <braunr> no - <braunr> that's the really basic approach - <braunr> (and in practice it can happen of course) - <braunr> what really happens is that, for example, on linux, the user space - system call code is loaded as a virtual shared library - <braunr> use ldd on an executable to see it - <braunr> this virtual object provides code that, depending on what the - kernel has detected, will use the appropriate method to perform a system - call - <peo-xaci> but this user space system calls need to make some kind of cpu - interupt to communicate with the kernel, right? - <braunr> the glibc itself has no idea how a system call will look like in - the end - <braunr> yes - <peo-xaci> an assembler programmer would be able to get around this glue - code? - <braunr> that's precisely what is embedded in this virtual library - <braunr> it could yes - <braunr> i think even when sysenter/sysexit is supported, legacy traps are - still implemented to support old binaries - <braunr> but then all these different entry points will lead to the same - code inside the kernel - <peo-xaci> but when the glue code is used, then its API compatible, and - then I can understand that the kernel can allow different syscall - implementations for different executables - <braunr> what glue code ? - <peo-xaci> what you talked about above "the user space system call code is - loaded as a virtual shared library" - <braunr> let's call it vdso - -[[vDSO]] - - <peo-xaci> thanks, I looked it up on Wikipedia and understand immediately - :P - <peo-xaci> so VDSOs are provided by the kernel, not a regular library file, - right? - <vdox2> What does HELL stand for :) ? - <dardevelin> vdox2, Hurd Emulation Layer for Linux - <vdox2> dardevelin: thanks - <braunr> peo-xaci: yes - <antrik> peo-xaci: I believe your goals are conflicting. a low-level - implementation makes it basically impossible to interact between the HELL - environment and the GNU/Linux environment in any meaningful way. to allow - such interaction, you *have* to have some glue at a higher semantic level - <braunr> agreed - <antrik> peo-xaci: BTW, if you want regular Linux binaries to get somehow - redirected to access HELL facilities, there is already a framework (don't - remember the name right now) that allows this kind of system call - redirection on Linux - <antrik> (it can run both through LD_PRELOAD or as a kernel module -- where - obviously only the latter would allow raw system call redirection... but - TBH, I don't think that's worthwhile anyways. the rare cases where - programs use raw system calls are usually for extremely system-specific - stuff anyways...) - <antrik> ViewOS is the name - <antrik> err... View-OS I mean - <antrik> or maybe View OS ? ;-) - <antrik> whatever, you'll find it :-) - -[[Virtual_Square_View-OS]]. - - <antrik> I'm not sure it's really worthwhile to use this either - though... the most meaningful interaction is probably at the FS level, - and that can be done with FUSE - <antrik> OHOH, View-OS probably allows doing more interesting stuff that - FUSE, such as modyfing the way the VFS works... - <antrik> OTOH - <antrik> so it could expose more of the Hurd features, at least in theory - - -## IRC, freenode, #hurd, 2012-10-13 - - <peo-xaci> antrik / braunr: thanks for your input! I'm not entirely - convinced though. :) I will probably return to this project once I have - acquired a lot more knowledge about low level stuff. I want to see for - myself whether a low level HELL is not feasible. :P - <braunr> peo-xaci: what's the point of a low level hell ? - <peo-xaci> more Hurd code can be tested in the hell, if the hell is at a - low level - <peo-xaci> at a higher level, some Hurd code cannot run, because the - interfaces they use would not be accessible from the higher level - emulation - <antrik> peo-xaci: I never said it's not possible. I actually said it would - be easier to do. I just said you can't do it low level *and* have - meaningful interaction with the host system - <peo-xaci> I don't understand why - <braunr> peo-xaci: i really don't see what you want to achieve with low - level support - <braunr> what would be unavailable with a higher level approach ? |