path: root/open_issues/linux_as_the_kernel.mdwn

[[!meta copyright="Copyright © 2012, 2014 Free Software Foundation, Inc."]]

[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]

Instead of attempting a [[history/port_to_another_microkernel]], or writing an
own one, an implementation of a Hurd system could use another existing
operating system/kernel, like [[UNIX]], for example, the Linux kernel.  This is
not a [[microkernel]], but that is not an inherent hindrance; depending on what
the goals are.

There has been an attempt for building a [[Mach_on_top_of_POSIX]].


# IRC, freenode, #hurd, 2012-02-08

Richard's X-15 Mach re-implementation:

    <braunr> and in case you didn't notice, it's stalled
    <braunr> actually i don't intend to work on it for the time being
    <braunr> i'd rather do as neal suggested: take linux, strip it, and give it
      a mach interface
    <braunr> (if your goal really is to get something usable for real world
      tasks)
    <antrik> braunr: why would you want to strip down Linux? I think one of the
      major benefits of building a Linux-Frankenmach would be the ability to
      use standard Linux functionality alongside Hurd...
    <braunr> we could have a linux x86_64 based mach replacement in "little"
      time, with a compatible i386 interface for the hurd
    <braunr> antrik: well, many of the vfs and network subsystems would be hard
      to use
    <antrik> BTW, one of the talks at FOSDEM was about the possibility of using
      different kernels for Genode, and pariticularily focused on the
      possibilities with using Linux... unfortunately, I wasn't able to follow
      the whole talk; but they mentioned similar possibilities to what I'm
      envisioning here :-)


## IRC, freenode, #hurd, 2012-03-28

    <mel__> is there currently any work going on with respect to
      Mach-alternatives?
    <antrik> mel__: no
    <antrik> we have other priorities to take care of :-)
    <braunr> antrik: i still intend to try giving linux a "mach personality" :)
    <braunr> antrik: but i don't have much time for development currently :(
    <mel__> antrik: which means that the hope is that Mach can be turned into
      something really well working (i.e. secure/scalable/etc.)?
    <antrik> mel__: yes, that's the venue we are pursuing
    <antrik> (and IMHO the right one... though the Linux layer mentioned by
      braunr is also one of my favourite projects, that we should pursue *in
      parallel* to the existing Mach-based implementation)
    <mel__> what is this Linux Layer exactly?
    <mel__> a Linux instance running on top of Mach in parallel to Hurd
      serverS?
    <braunr> mel__: not exactly
    <braunr> mel__: it would involve adding a mach layer on top of linux
      actually
    <braunr> mel__: so that linux can be used as a mach kernel
    <mel__> Ah!
    <mel__> Running Hurd on top of Linux
    <mel__> :-D
    <mel__> Funny
    <braunr> ironic, yes
    <braunr> but very pragmatic
    <mel__> and THEN
    <antrik> yeah. I most like the name: Hurd Emulation Layer on
      Linux... i.e. HELL :-)
    <mel__> we use a device driver framework something so that we can use Linux
      device drivers in Hurd!
    <mel__> on top of Linux....
    <braunr> yes
    <braunr> i guess a transition phase would include using in kernel drivers
      directly for a while
    <mel__> and somebody is working on that?
    <antrik> mel__: well, for using Linux drivers we are persuing DDE, which
      allows us doing that with Mach as well
    <braunr> then grabbing them out of the kernel and into dde
    <braunr> not yet
    <antrik> (in fact we have been using Linux drivers since forever... they
      just haven't been updated for ages)
    <mel__> I would _guess_ that it is not that hard.
    <braunr> it's not
    <mel__> Basically one would need to implement the message passing interface
      thing in linux I guess.
    <braunr> and many exported kernel objects like tasks, threads, etc..
    <braunr> and implement all the RPCs normally implemented by the kernel
    <braunr> but it's doable
    <antrik> mel__: the IPC aspect is one part, but probably the less tricky
      one. the external pager interface is really the crucial element
    <mel__> uh
    <mel__> yeah
    <mel__> hooking into linux virtual memory stuff
    <mel__> sounds delicate
    <braunr> it's true that some interactions between the linux VM and file
      systems (the linux equivalent of our pagers) is synchronous
    <braunr> but i don't think it would be that hard considering the amount of
      facilities available in linux
    <braunr> it's just work, takes time, needs debugging, reviewing, testing,
      etc..
    <lcc> hurd on top of linux. how would that work?
    <braunr> 15:30 < braunr> antrik: i still intend to try giving linux a "mach
      personality" :)
    <braunr> lcc: 7 system calls and a few hundreds of RPCs on top, the
      internal magic of course, and voila ..
    <antrik> of course porting Mach still requires work
    <mel__> that would then be GNU/Hurd/Linux
    <mel__> :-)
    <antrik> hehe
    <braunr> eh
    <antrik> braunr: BTW, are you more thinking of a userspace solution on top
      of standard Linux mechanisms, or additions to Linux itself?...
    <antrik> (we probably talked about it already, but I don't remember all the
      details...)
    <braunr> antrik: adding to linux
    <antrik> do you think a pure userspace solution would be realistic at all?
      (at the expense of some performance of course)
    <mel__> it's probably comparable to the qemu vs. qemu/kvm thing
    <antrik> yeah, I guess that pretty much sums it up...
    <braunr> antrik: i don't know :/
    <antrik> OK
    <lcc> how challenging is it to port mach?
    <antrik> lcc: it requires good low-level knowledge of the platform in
      question. having that, I guess it shouldn't be too hard to add the
      necessary code in Mach...
    <antrik> TBH I'm not sure how much knowledge of Mach internals is required
    <braunr> the pmap module is the main thing to port
    <antrik> braunr: well, sartakov seemed to have most trouble with the
      scheduler when he attempted the ARM port...
    <braunr> that's strange
    <antrik> at least there was quite a long thread where he asked about how
      task switching works in Mach
    <braunr> ok
    <braunr> that would be interesting
    <braunr> i thought intereacting with the hardclock was enough


## IRC, freenode, #hurd, 2012-04-05

    <braunr> antrik: don't you think HELL is worth a try for the GSoC btw ?
    <antrik> braunr: definitely. I haven't managed to rework the project ideas
      list at all this year... but it's something I wanted there for a long
      time

    <youngrw> just out of curiousity, what is HELL ?
    <antrik> Hurd Emulation Layer on Linux
    <braunr> youngrw: it can be described in several ways
    <braunr> youngrw: basically, it's a mach interface on top of linux
    <youngrw> implementing I suppose both the IPC mechanism and memory
      management interface?
    <mel__> youngrw: basically that. more generally: implement everything in
      order to let Hurd run on that layer.
    <antrik> well, it's slightly more complicated in my view... it's basically
      anything that allows running a Hurdish environment on top of
      GNU/Linux. it might be simply an implementation/emulation of Mach
      mechanisms; but it also *might* work on a slightly higher level...
    <youngrw> antrik: how might HELL operate at the slighty higher level like
      you describe?
    <antrik> let's call it low-level HELL and high-level HELL ;-)
    <antrik> (a more descriptive name would be hurdenv... but HELL is so much
      more catchy :-) )
    <antrik> so, high-level HELL... basically, the idea would be not to emulate
      the kernel facilities and run all the real Hurd servers; but instead to
      use special servers implementing the Hurd interfaces, but on top of
      standard Linux facilities
    <antrik> hurdish programs could run in such an environment, as long as they
      aren't too low-level
    <antrik> I wonder whether generic RPC interfaces could be implemented with
      some side channel tunneled though the ordinary Linux FS interfaces...
    <antrik> so translators would be loaded as FUSE modules for example, but
      could still present generic interfaces
    <youngrw> That's actually pretty different from what I was expecting
    <antrik> what were you expecting?
    <youngrw> maybe something where the entire kernel interface is emulated by
      a running user process, like a kind of virtual machine
    <youngrw> I hope that makes sense--I may be using my words incorrectly.
    <antrik> youngrw: that would be in the low-level HELL category
    <youngrw> antrik: right; I had the misconception that the level was defined
      by how it made use of the underlying linux system
    <youngrw> and that different HELL designs would always implement the mach
      interface


## IRC, freenode, #hurd, 2012-04-06

    <braunr> antrik: i think we have diverging ideas about how to use linux for
      the hurd
    <braunr> antrik: what you seem to want are really emulation componants,
      like e.g. ext2fs and pfinet actually using the linux implementation
    <braunr> (correct me if i'm mistaken)
    <braunr> whereas my project is to make linux behave as a mach clone
    <antrik> braunr: as I said, I consider both variants -- either a high-level
      HELL or a low-level HELL
    <braunr> ok
    <antrik> (or perhaps a mix of both)
    <braunr> a mix would be best imho
    <antrik> yeah, probably
    <braunr> so we have the real hurd, the real mach interface, and a set of
      native translators (e.g. ext2fs) along some emulating their functionality
      using linux code (e.g. a hypothetical ext4fs)
    <antrik> I don't think we would have emulation servers for individual Linux
      filesystems. rather, a generic server interfacing with the Linux VFS
      tree...
    <braunr> ok

    <antrik> braunr: BTW, I think I mentioned a couple of years ago that the
      most realistic route towards a modern Mach in my opinion would be taking
      a modern BSD (or Linux), and redo what the original Mach developers did
      -- i.e. add the Mach-specific features, and drop the unnecessary UNIX
      stuff
    <braunr> antrik: :)
    <braunr> we had discussions about it some time ago yes
    <antrik> later I realised that it's better *not* to drop the UNIX
      interfaces, but rather to keep them in parallel :-)
    <braunr> antrik: for what purpose ?
    <braunr> (i can imagine a few, but i'd like to know your idea)
    <antrik> for the purpose of HELL :-)
    <braunr> well hell would be the implementation, but what do you keep these
      unix interfaces for ?
    <antrik> i.e. people being able to play around with a Hurd environment
      while staying with their existing system
    <braunr> yes, i see
    <braunr> i was considering doing that for development, yes
    <braunr> uml first, and then i realized i wouldn't need it :)
    <braunr> then i remembed netbsd and its syscall emulation layer
    <antrik> also we might leverage some "foreign" userspace infrastructure
      that way, such as udev
    <antrik> (in the case of Linux that is... not sure whether NetBSD has
      something similar at all ;-) )
    <braunr> i'll have to check, it's been a long time since i've really used
      it
    <braunr> they must use a pure devfs instance now


# IRC, freenode, #hurd, 2014-02-23

    <desrt> so crazy idea: would it be possible to have mach as a linux kernel
      module?
    <desrt> ie: some new binfmt type thing that could load mach binaries and
      implement the required kernel ABI for them
    <desrt> and then run the entire hurd under that....
    <braunr> desrt: that's an idea, yes
    <braunr> and not a new one
    * desrt did a bit of googling but didn't find any information about it
    <braunr> desrt: but why are you thinking of it ?
    <braunr> we talked about it here, informally
    <desrt> braunr: mostly because running hurd in a VM sucks
    <desrt> if we had mach-via-linux, we'd have:
    <desrt>  - no vm overhead
    <desrt>  - no device virtualisation
    <desrt>  - 64bit (physical at least) memory support
    <desrt>  - SMP
    <desrt>  - access to the linux drivers, natively
    <desrt> and maybe some other nice things
    <braunr> yes we talkbed about all this
    <braunr> but i still consider that to be an incomplete solution
    <desrt> i don't consider it to be running "the hurd" as your OS... but it
      would be a nice solution for development and virtualisation
    <braunr> we probably don't want to use drivers natively, since we want them
      to run in their own address space, with their own namespace context
    <braunr> it would, certainly
    <braunr> but it would require a lot of effort anyway
    <desrt> right