path: root/history/port_to_another_microkernel.mdwn

[[!meta copyright="Copyright © 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008,
2009, 2010, 2011 Free Software Foundation, Inc."]]

[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]

[[!meta title="Porting the Hurd to another microkernel"]]

At first, there was an effort to port the Hurd from the [[Mach
microkernel|microkernel/mach]] to the [[L4_microkernel_family|microkernel/l4]].
Then the story continued...

[[!toc levels=2]]


# L4

## Initial Idea

Encountering a number of fundamental design issues with the [[Mach
microkernel|microkernel/mach]] (mostly regarding [[resource
management|open_issues/resource_management_problems]]), some of the Hurd
developers began experimenting with using other microkernels for the Hurd
around the turn of the millenium.

The idea of using L4 as a [[microkernel]] for a Hurd system was initially
voiced in the [[community]] by Okuji Yoshinori, who, for discussing this
purpose, created the [[mailing_lists/l4-hurd]] mailing list in November 2000.

Over the years, a lot of discussion have been held on this mailing list, which
today is still the right place for [[next-generation Hurd|hurd/ng]]
discussions.


## Why?

Even though that said resource management issues constitute a broad research
topic, there was no hope that the original Mach project would work on these:
[[microkernel/Mach]] wasn't maintained by its original authors anymore.  Mach
had served its purpose as a research vehicle, and has been retired by its
stakeholders.

Thus, switching to a well-maintained current [[microkernel]] was expected to
yield a more solid foundation for a Hurd system than the [[decaying
Mach|microkernel/mach/history]] design and implementation was able to.

At that time, the [[L4 microkernel family|microkernel/L4]] was one obvious
choice.  Being a second-generation microkernel, it was deemed to provide for a
faster system kernel implementation, especially in the time-critical [[IPC]]
paths.  Also, as L4 was already implemented for a bunch of different
architectures (x86, Alpha, MIPS; also including SMP support), and the Hurd
itself being rather archtecture-agnostic, it was expected to be able to easily
support more platforms than with the existing system.


## Steps and Goals

At the same time, the idea was -- while mucking with the system's core anyway
-- to improve on some fundamental design issues, too -- like the resource
management problems, for example.

One goal of porting the Hurd to L4 was to make the Hurd independent of
[[microkernel/Mach]] interfaces, to make it somewhat microkernel-agnostic.

One idea was to first introduce a Mach-on-L4 emulation layer, to easily get a
usable (though slow) Hurd-using-Mach-interfaces-on-L4 system, and then
gradually move the Hurd servers to use L4 intefaces rather than Mach ones.

A design upon the lean L4 kernel would finally have made it feasible to move
devices drivers out of the kernel's [[TCB]].


# Implementation

The project itself then was mostly lead by Marcus Brinkmann and Neal Walfield.
Neal started the original Hurd/L4 port while visiting Karlsruhe university in
2002.  He explains:

> My intention was to adapt the Hurd to exploit L4's concepts and intended
> [[design_pattern]]s; it was not to simply provide a Mach
> [[compatibility_layer]] on top of L4.  When I left Karlsruhe, I no longer had
> access to [[microkernel/l4/Pistachio]] as I was unwilling to sign an NDA.
> Although the specification was available, the Karlsruhe group only [released
> their code in May
> 2003](https://lists.ira.uni-karlsruhe.de/pipermail/l4ka/2003-May/000345.html).
> Around this time, Marcus began hacking on Pistachio.  He created a relatively
> complete run-time.  I didn't really become involved again until the second
> half of 2004, after I complete by Bachelors degree.

Development of Hurd/L4 was done in the [CVS module
`hurd-l4`](http://savannah.gnu.org/cgi-bin/viewcvs/hurd/hurd-l4/).  The `doc`
directory contains a design document that is worth reading for anyone who
wishes to learn more about Hurd/L4.

Even though there was progress -- see, for example, the [[QEMU image for
L4|hurd/running/qemu/image_for_l4]] -- this port never reached a releasable
state.  Simple POSIX programs, such as `banner` could run, but for more complex
system interfaces, a lot more work was needed.

Eventually, a straight-forward port of the original Hurd's design wasn't deemed
feasible anymore by the developers, partly due to them not cosidering L4
suitable for implementing a general-purpose operating system on top of it, and
because of deficiencies in the original Hurd's design, which they discovered
along their way.  Neal goes on:

> Before Marcus and I considered [[microkernel/Coyotos]], we had already
> rejected some parts of the Hurd's design.  The
> [[open_issues/resource_management_problems]] were
> what prompted me to look at L4.  Also, some of the problems with
> [[hurd/translator]]s were already well-known to us.  (For a more detailed
> description of the problems we have identified, see our [[hurd/critique]] in the
> 2007 July's SIGOPS OSR.  We have also written a forward-looking
> [[hurd/ng/position_paper]].)

> We visited Jonathan Shapiro at Hopkins in January 2006.  This resulted in a
> number of discussions, some quite influential, and not always in a way which
> aligned our position with that of Jonathan's.  This was particularly true of
> a number of security issues.

A lange number of discussion threads can be found in the archives of the
[[mailing_lists/l4-hurd]] mailing list.

> Hurd-NG, as we originally called it, was an attempt to articulate the system
> that we had come to envision in terms of interfaces and description of the
> system's structure.  The new name was selected, if I recall correctly, as it
> clearly wasn't the Hurd nor the Hurd based on L4.


## Termination

As of 2005, development of Hurd/L4 has stopped.


# Coyotos

Following that, an attempt was started to use the kernel of the
[[microkernel/Coyotos]] system.  As Coyotos is an object capability system
througout, the microkernel would obviously be more suitable for this purpose;
and it looked pretty promising in the beginning.  However, further
investigations found that there are some very fundamental philosophical
differences between the Coyotos and Hurd designs; and thus this this attempt
was also abandonned, around 2006 / 2007.  (This time before producing any
actual code.)


# Viengoos

By now (that is, after 2006), there were some new [[microkernel/L4]] variants
available, which added protected [[IPC]] paths and other features necessary for
object capability systems; so it might be possible to implement the Hurd on top
of these.  However, by that time the developers concluded that microkernel
design and system design are interconnected in very intricate ways, and thus
trying to use a third-party microkernel will always result in trouble.  So Neal
Walfield created the experimental [[microkernel/Viengoos]] kernel instead --
based on the experience from the previous experiments with L4 and Coyotos --
for his [[research on resource
management|open_issues/resource_management_problems]].  Currently he works in
another research area though, and thus Viengoos is on hold.


# Intermediate Results

Note that while none of the microkernel work is active now, the previous
experiments already yielded a lot of experience, which will be very useful in
the further development / improvement of the mainline (Mach-based) Hurd
implementation.