From 95878586ec7611791f4001a4ee17abf943fae3c1 Mon Sep 17 00:00:00 2001 From: "https://me.yahoo.com/a/g3Ccalpj0NhN566pHbUl6i9QF0QEkrhlfPM-#b1c14" Date: Mon, 16 Feb 2015 20:08:03 +0100 Subject: rename open_issues.mdwn to service_solahart_jakarta_selatan__082122541663.mdwn --- ...implementing_hurd_on_top_of_another_system.mdwn | 420 --------------------- 1 file changed, 420 deletions(-) delete mode 100644 open_issues/implementing_hurd_on_top_of_another_system.mdwn (limited to 'open_issues/implementing_hurd_on_top_of_another_system.mdwn') diff --git a/open_issues/implementing_hurd_on_top_of_another_system.mdwn b/open_issues/implementing_hurd_on_top_of_another_system.mdwn deleted file mode 100644 index cf3db0ab..00000000 --- a/open_issues/implementing_hurd_on_top_of_another_system.mdwn +++ /dev/null @@ -1,420 +0,0 @@ -[[!meta copyright="Copyright © 2010, 2011, 2012, 2013 Free Software Foundation, -Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -[[!tag open_issue_documentation]] - -It is possible to run Hurd stuff on top of another system instead of on Mach. -One obvious variant is [[emulation]] (using [[hurd/running/QEMU]], for -example), but -doing that does not really integratable the Hurd guest into the host system. -There is also a more direct way, more powerful, but it also has certain -requirements to do it effectively. - -See also [[Mach_on_top_of_POSIX]]. - - -# IRC, freenode, #hurd, August / September 2010 - - silver_hook: the Hurd can also refer to the interfaces of the - filesystems etc, and a lot of that is really just server/client APIs that - could be implemented on any system that has transferable rights to - message capabilities. - silver_hook: it's surprising how few systems *have* transferable - rights, though! - silver_hook: usually it is added as an afterthought - and comes with restriction - marcusb: there's SCM_RIGHTS to transfer fds, which is quite often - available - youpi: yes, I know this as "fdpassing" - youpi: it's described in the Stevens series even - [...] - ArneBab: well, let me put it this way. the Linux kernel has no - interface to manipulate another tasks's virtual address space, ie you - can't map/unmap stuff in another process - ArneBab: you would have to use ptrace and load some stub code in - that process to make that happen. - ArneBab: so for complete transparent manipulation, you need a - kernel module - that is what the User Mode Linux kernel module does - ArneBab: so say you use the User Mode Linux kernel module for - that one feature. Then you can do everything that User Mode Linux can - do, which, I assure you, includes running subhurds :) - it can be a bit tricky to implement those features, but it is not - harder than writing a kernel in the first place - So, if I got an admin to install User Mode Linux and Mach - emulation, I’d get the flexibility (and independence from admin - decisions) I have in the Hurd? - ArneBab: one problem is that you still use Linux. For those who - want to get rid of Linux for political reasons, that would mean complete - failure - ArneBab: if you have UML kernel module, you can implement Mach in - user space - ArneBab: in fact, John Tobey did this a couple of years ago, or - started it - -[[Mach_on_top_of_POSIX]]. - - ArneBab: or you can just implement parts of it and relay to Linux - for the rest - the point is, that if you don't care for kernel improvements, and - are sufficiently happy with the translator stuff, it's not hard to bring - the Hurd to Linux or BSD - -Continue reading about the [[benefits_of_a_native_Hurd_implementation]]. - - -# IRC, freenode, #hurd, 2010-12-28 - - kilobug: there is no real requirement for the Hurd to run on a - microkernel... as long as the important mechanisms are provided (most - notably external pagers and Mach IPC), the Hurd could run an top of - pretty much any kernel... - whether it makes sense is another question of course :-) - though I must say that I'm more and more convinced running the - Hurd on top of a monolithic kernel would actually be a useful approach - for the time being... - - -# IRC, freenode, #hurd, 2011-02-11 - - marcus and I were discussing how to add Mach to Linux - one could write a module to implement Mach IPC - and another to implement Mach VM - the big thing missing with Mach VM is the ability for a tracing - process to easily map or unmap an inferior process's memory - neal: why would a tracing process need to map the inferior's - memory? - the simple answer is that is how it is done on Mach - neal: is it? not sure we are talking about the same thing - here. GDB uses vm_read()/vm_write() to access the inferior's memory AFAIK - on linux? - I think it use /proc/pid/mem - on Hurd - I'm talking about adding Mach to Linux - by adding some functionality to Linux - and then implementing a bunch in user space - yeah, but I don't understand the point about mapping inferior's - memory :-( - what would be in user space? - there are a number of different cut points - one could imagine just using Linux's device drivers, CPU scheduler, - memory management, etc. - another possibility would be something higher where Hurd processes - just use some Hurdish servers - neal: yeah, these are all options I have been considering... too - bad I wasn't able to come to FOSDEM -- I'd love to have participated in - this discussion :-( - neal: BTW, were you just discussing this as a hypothetical idea, - or something you are seriously considering? - I'm unlikely to work on it, sorry - didn't really expect that :-) - would be nice though if you could write up your conclusions... - - -# IRC, freenode, #hurd, 2012-10-12 - - do hurd system libraries make raw system calls ever - (i.e. inlined syscall() / raw assembly)? - sure - hmm, so a hurd emulation layer would need to use ptrace if it - should be fool proof? :/ - there is no real need for raw assembly, and the very syscalls are - all available through macros - hum what are you trying to say ? - well, if they are done through syscall, as a function, not a - macro, then they can be intercepted with LD_PRELOAD - so applications that do Hurd (Mach?) syscalls could work on - f.e. Linux, if a special libc is injected into the program with - LD_PRELOAD - same thing with making standard Linux-applications go through - the Hurd emulation layer - without recompilation - peo-xaci: the second direction is implemented in glibc. - for the other direction, I personally see little use for it - peo-xaci: ok i misunderstood - peo-xaci: i don't think there is any truely direct syscall usage - in the hurd - hmm, I'm not sure I understand what directions you are referring - to mel-_ - peo-xaci: what are you trying to achieve ? - I want to make the Hurd design more accessible by letting Hurd - application run on the Linux kernel, preferably without - recompilation. This would be done with a daemon that implements Mach and - which all syscalls would go to. - then, I also want so that standard Linux applications can go - through that Mach daemon as well, if a special libc is preloaded - you might want to discuss this with antrik - what I'm trying to figure out specifically is if there is some - library/interface that glue Hurd with Mach and would be better suited to - emulate than Mach? Mach seems to be more of an implementation detail to - the hurd and not something an application would directly use. - yes, the various hurd libraries (libports and libpager mostly) - From [http://www.gnu.org/software/hurd/hurd/libports.html]: - "libports is not (at least, not for now) a generalization / abstraction - of Mach ports to the functionality the Hurd needs, that is, it is not - meant to provide an interface independently of the underlying - microkernel." - Is this still true? - Does libpager abstract the rest? - (and the other hurd libraries) - there is nothing that really abstracts the hurd from mach - for example, reference counting often happens here and there - and core libraries like glibc and libpthread heavily rely on it - (through sysdeps specific code though) - libports and libpager are meant to simplify object manipulation - for the former, and pager operations for the latter - and applications, such as translators, often use Mach interfaces - directly? - correct? - depends on what often means - let's say they do - :/ then it probably is better to emulate Mach after all - there was a mach on posix port a long time ago - I thought applications were completely separated from the - microkernel in use by the Hurd - that level of abstraction is pretty new - genode is the only system i know which does that - -[[microkernel/Genode]]. - - and it's still for "l4 variants" - ah, thanks (i forgot that name) - braunr: Genode also runs on Linux and a few other non-L4 - environments IIRC - peo-xaci: I'm not sure binary emulation is really useful. rather, - I'd recompile stuff as "regular" Linux executables, only using a special - glibc - where the special glibc could be basically a port of the Hurd - glibc communicating with the Mach emulation instead of real Mach; or it - could do emulation at a higher level - a higher level emulation would be more complicated to implement, - but more efficient, and allow better integration with the ordinary - GNU/Linux environment - also note that any regular program could be recompiled against the - HELL glibc to run in the Hurdish environment... - (well, glibc + hurd server libraries) - I'm willing to accept that Hurd-application would need to be - recompiled to work on the HELL - but not Linux-applications :) - peo-xaci: if you happen to understand German, there is a fairly - good overview in my thesis report ;-) - peo-xaci: there are no "Hurd applications" or "Linux applications" - well, let me define what I mean by the terms: Hurd applications - use Hurd-specific interfaces/syscalls, and Linux applications use - Linux-specific interfaces/syscalls - a few programs use Linux-specific interfaces (and we probably - can't run them in HELL just as we can't run them on actual Hurd); but all - other programs work in any glibc environment - (usually in any POSIX environment in fact...) - peo-xaci: no sane application uses syscalls - they do under the hood - I have read about inlined syscalls - again, there are *some* applications using Linux-specific - interfaces (sometimes because they are inherently bound to Linux - features, sometimes unnecessarily) - so far there are no applications using Hurd-specific interfaces - translators do? - they are standard executables are they not? - I would like so that translators also can be run in the HELL - I wouldn't consider them applications. all existing translators - are pretty much components of the Hurd itself - okay, it's a question about semantics, perhaps I should use - another word than "applications" :) - for me, applications are what have a main-function, or similar - single entry point - hum - that's not a good enough definition - anyways, as I said, I think recompiling translators against a - Hurdish glibc and ported translator libraries seems the most reasonable - approach to me - let's say applications are userspace processes that make use of - services provided by the operating system - translators being part of the operating system here - braunr: do you know whether the Mach-on-POSIX was actually - functional, or just an abandoned experiment?... - (I don't remember hearing of it before...) - incomplete iirc - braunr: still, when I've explained what I meant, even if I used - the wrong term, then my previous statements should come in another light - antrik / braunr: are you still interested in hearing my - thoughts/ideas about HELL? - oh, there is more to come? ;-) - yes! I don't think I have made myself completely understood :/ - what I envision is a HELL system that works on as low level as - feasible, to make it possible to do almost anything that can be done on - the real Hurd (except possibly testing hardware drivers and such very low - level stuff). - sure - I want it to be more than just allowing programs to access a - virtual filesystem à la FUSE. My idea is that all user space system - libraries/programs of the Hurd should be inside the HELL as well, and - they should not be emulated. - The system should at the very least be API compatible, so at the - very most a recompilation is necessary. - I also want so that GNU/Linux-programs can access the features - of the HELL with little effort on the user. At most perhaps a script that - wraps LD_PRELOADing has to be run on the binary. Best would be if it - could work also with insane assembly programs using raw system calls, or - if glibc happens to have some well hidden syscall being inlined to raw - assembly code. - And I think I have an idea on how an implementation could - satisfy these things! - By modifying the kernel and replace those syscalls that make - sense for the Hurd/Mach - with "the kernel", I meant Linux - it's possible but tedious and not very useful so better do that - later - mach did something similar at its time - there was a syscall emulation library - but isn't it about as much work as emulating the interface on - user-level? - and the kernel cooperated so that unmodified unix binaries - performing syscalls would actually jump to functions provided by that - library, which generally made an RPC - instead of a bunch of extern-declerations, one would put the - symbols in the syscall table - define what "those syscalls that make sense for the Hurd/Mach" - actually means - open/close, for example - otherwise i don't see another better way than what the old mach - folks did - well, with that old, but existing support, your open would perform - a syscall - the kernel would catch it and redirect the caller to its syscall - emulation library - which would call the open RPC instead - wait, so this "existing support" you're talking about; is this a - module for the Linux kernel (or a fork, or something else)? - where can I find it? - no - it was for mach - in order to run unmodified unix binaries - the opposite of what you're trying to do - ah okay - well - not really either :) - does posix/unix define a standard for how a syscall table should - look like, to allow binary syscall compatibility? - absolutely not - so how could this mach module run any unmodified unix binary? if - they expected different sys calls at different offsets? - posix specifically (and very early) states that it almost forbids - itself to deal with anything regarding to ABIs - depends - since it was old, there weren't that many unix systems - and even today, there are techniques like those used by netbsd - (and many other actually) - that are able to inspect the binary and load a syscall emulation - environment depending on its exposed ABI - e.g. file on an executable states which system it's for - hmm, I'm not sure how a kernel would implement that in - practice.. I thought these things were so hard coded and dependent on raw - memory reads that it would not be possible - but i really think it's not worth the time for your project - to be honest I have virtually no experience of practical kernel - programming - with an LDT on x86 for example - no, there is really not that much hardcoded - quite the contrary - there is a lot of runtime detection today - well I mean how the syscall table is read - it's not read - it's read to find the function pointer to the syscall handler in - the kernel? - no - that's the really basic approach - (and in practice it can happen of course) - what really happens is that, for example, on linux, the user space - system call code is loaded as a virtual shared library - use ldd on an executable to see it - this virtual object provides code that, depending on what the - kernel has detected, will use the appropriate method to perform a system - call - but this user space system calls need to make some kind of cpu - interupt to communicate with the kernel, right? - the glibc itself has no idea how a system call will look like in - the end - yes - an assembler programmer would be able to get around this glue - code? - that's precisely what is embedded in this virtual library - it could yes - i think even when sysenter/sysexit is supported, legacy traps are - still implemented to support old binaries - but then all these different entry points will lead to the same - code inside the kernel - but when the glue code is used, then its API compatible, and - then I can understand that the kernel can allow different syscall - implementations for different executables - what glue code ? - what you talked about above "the user space system call code is - loaded as a virtual shared library" - let's call it vdso - -[[vDSO]] - - thanks, I looked it up on Wikipedia and understand immediately - :P - so VDSOs are provided by the kernel, not a regular library file, - right? - What does HELL stand for :) ? - vdox2, Hurd Emulation Layer for Linux - dardevelin: thanks - peo-xaci: yes - peo-xaci: I believe your goals are conflicting. a low-level - implementation makes it basically impossible to interact between the HELL - environment and the GNU/Linux environment in any meaningful way. to allow - such interaction, you *have* to have some glue at a higher semantic level - agreed - peo-xaci: BTW, if you want regular Linux binaries to get somehow - redirected to access HELL facilities, there is already a framework (don't - remember the name right now) that allows this kind of system call - redirection on Linux - (it can run both through LD_PRELOAD or as a kernel module -- where - obviously only the latter would allow raw system call redirection... but - TBH, I don't think that's worthwhile anyways. the rare cases where - programs use raw system calls are usually for extremely system-specific - stuff anyways...) - ViewOS is the name - err... View-OS I mean - or maybe View OS ? ;-) - whatever, you'll find it :-) - -[[Virtual_Square_View-OS]]. - - I'm not sure it's really worthwhile to use this either - though... the most meaningful interaction is probably at the FS level, - and that can be done with FUSE - OHOH, View-OS probably allows doing more interesting stuff that - FUSE, such as modyfing the way the VFS works... - OTOH - so it could expose more of the Hurd features, at least in theory - - -## IRC, freenode, #hurd, 2012-10-13 - - antrik / braunr: thanks for your input! I'm not entirely - convinced though. :) I will probably return to this project once I have - acquired a lot more knowledge about low level stuff. I want to see for - myself whether a low level HELL is not feasible. :P - peo-xaci: what's the point of a low level hell ? - more Hurd code can be tested in the hell, if the hell is at a - low level - at a higher level, some Hurd code cannot run, because the - interfaces they use would not be accessible from the higher level - emulation - peo-xaci: I never said it's not possible. I actually said it would - be easier to do. I just said you can't do it low level *and* have - meaningful interaction with the host system - I don't understand why - peo-xaci: i really don't see what you want to achieve with low - level support - what would be unavailable with a higher level approach ? -- cgit v1.2.3