adding in serverboot v2 draft RFC.

* hurd/bootstrap.mdwn: I inlined the what_is_an_os_bootstrap page, and
wrote that the current bootstrap page is out of date and does not
include pci-arbiter or rumpdisk.
* hurd/what_is_an_os_bootstrap.mdwn: a new web page that is not meant
to be viewed directly.  Instead hurd/bootstrap and
open_issues/serverbootv2 is meant to inline the content.
* open_issues/serverbootv2.mdwn: Sergey proposed this new bootstrap
for the Hurd.  This is a draft RFC document that explains the
reasoning behind it.  Not that "Serverboot V2" is a working name.  We
have yet to find a better name for it.
Message-ID: <20240527010924.2607-1-jbranso@dismail.de>

3 files changed, 930 insertions, 0 deletions

diff --git a/hurd/bootstrap.mdwn b/hurd/bootstrap.mdwn
index fbce3bc1..c77682b9 100644
--- a/hurd/bootstrap.mdwn
+++ b/hurd/bootstrap.mdwn
@@ -15,8 +15,15 @@ this text.  -->
 
 [[!toc]]
 
+[[!inline pagenames=hurd/what_is_an_os_bootstrap raw=yes feeds=no]]
+
 # State at the beginning of the bootstrap
 
+Please note that as of May 2024 this document is out of date.  It does
+not explain how rumpdisk or the pci-arbitor is started.  Also consider
+reading about [[Serverboot V2|open_issues/serverbootv2]], which
+is a new bootstrap proposal.
+
 After initializing itself, GNU Mach sets up tasks for the various bootstrap
 translators (which were loader by the GRUB bootloader). It notably makes
 variables replacement on their command lines and boot script function calls (see
diff --git a/hurd/what_is_an_os_bootstrap.mdwn b/hurd/what_is_an_os_bootstrap.mdwn
new file mode 100644
index 00000000..b2db2554
--- /dev/null
+++ b/hurd/what_is_an_os_bootstrap.mdwn
@@ -0,0 +1,24 @@
+[[!meta copyright="Copyright © 2020 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!meta title="What is an OS bootstrap"]]
+
+# What is an OS bootstrap?
+
+An operating system's bootstrap is the process that happens shortly
+after you press the power on button, as shown below:
+
+Power-on -> Bios -> Bootloader ->  **OS Bootstrap** -> service manager
+
+Note that in this context the OS bootstrap is not [building a
+distribution and packages from source
+code](https://guix.gnu.org/manual/en/html_node/Bootstrapping.html).
+The OS bootstrap has nothing to do with [reproducible
+builds](https://reproducible-builds.org/).
diff --git a/open_issues/serverbootv2.mdwn b/open_issues/serverbootv2.mdwn
new file mode 100644
index 00000000..9702183e
--- /dev/null
+++ b/open_issues/serverbootv2.mdwn
@@ -0,0 +1,899 @@
+[[!meta copyright="Copyright © 2024 Free Software
+Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+
+# ServerBootV2 RFC Draft
+
+[[!inline pagenames=hurd/what_is_an_os_bootstrap raw=yes feeds=no]]
+
+The Hurd's current bootstrap, [[Quiet-Boot|hurd/bootstrap]] (a biased
+and made-up name), is fragile, hard to debug, and complicated:
+
+* `Quiet-boot` chokes on misspelled or missing boot arguments.  When
+  this happens, the Hurd bootstrap will likely hang and display
+  nothing. This is tricky to debug.
+* `Quiet-Boot` is hard to change. For instance, when the Hurd
+  developers added `acpi`, the `pci-arbiter`, and `rumpdisk`, they
+  struggled to get `Quiet-Boot` working again.
+* `Quiet-Boot` forces each bootstrap task to include special bootstrap
+  logic to work.  This limits what is possible during the
+  bootstrap. For instance, it should be trivial for the Hurd to
+  support netboot, but `Quiet-Boot` makes it hard to add `nfs`,
+  `pfinet`, and `isofs` to the bootstrap.
+* `Quiet-Boot` hurts other Hurd distributions too. When Guix
+  developers updated their packaged version of the Hurd, that included
+  support for SATA drives, a simple misspelled boot argument halted
+  their progress for a few weeks.
+
+The alternative `ServerBoot V2` proposal (which was discussed on
+[irc](https://logs.guix.gnu.org/hurd/2023-07-18.log) and is similar to
+the previously discussed [bootshell
+proposal](https://mail-archive.com/bug-hurd@gnu.org/msg26341.html))
+aims to code all or most of the bootstrap specific logic into one
+single task (`/hurd/serverboot`).  `Serverboot V2` has a number
+of enticing advantages:
+
+* It simplifies the hierarchical dependency of translators during
+  bootstrap. Developers should be able to re-order and add new
+  bootstrap translators with minimal work.
+* It gives early bootstrap translators like `auth` and `ext2fs`
+  standard input and output which lets them display boot errors.  It
+  also lets signals work.
+* One can trivially use most Hurd translators during the
+  bootstrap. You just have to link them statically.
+* `libmachdev` could be simplified to only expose hardware to
+  userspace; it might even be possible to remove it entirely.  Also
+  the `pci-arbiter`, `acpi`, and `rumpdisk` could be simplified.
+* Developers could remove any bootstrap logic from `libdiskfs`, which
+  detects the bootstrap filesystem, starts the `exec` server, and
+  spawns `/hurd/startup`.  Instead,`libdiskfs` would only focus on
+  providing filesystem support.
+* If an error happens during early boot, the user could be dropped
+  into a REPL or mini-console, where he can try to debug the issue.
+  We might call this `Bootshell V2`, in reference to the original
+  proposal.  This could be written in lisp.  Imagine having an
+  extremely powerful programming language available during bootstrap
+  that is only [436 bytes!](https://justine.lol/sectorlisp2)
+* It would simplify the code for subhurds by removing the logic from
+  each task that deals with the OS bootstrap.
+
+Now that you know why we should use `Serverboot V2`, let's get more
+detailed.  What is `Serverboot V2` ?
+
+`Serverboot V2` would be an empty filesystem dynamically populated
+during bootstrap.  It would use a `netfs` like filesystem that will
+populate as various bootstrap tasks are started.  For example,
+`/servers/socket2` will be created once `pfinet` starts.  It also
+temporarily pretends to be the Hurd process server, `exec`, and `/`
+filesystem while providing signals and `stdio`.  Let's explain how
+`Serverboot V2` will bootstrap the Hurd.
+
+**FIXME The rest of this needs work.**
+
+Any bootstrap that the Hurd uses will probably be a little odd,
+because there is an awkward and circular startup-dance between
+`exec`, `ext2fs`, `startup`, `proc`, `auth`, the `pci-arbiter`,
+`rumpdisk`, and `acpi` in which each translator oddly depends on the
+other during the bootstrap, as this ascii art shows.
+
+
+       pci-arbiter
+           |
+          acpi
+           |
+        rumpdisk
+           |
+         ex2fs  -- storeio
+        /     \
+     exec     startup
+      /          \
+    auth         proc
+
+
+This means that there is no *perfect* Hurd bootstrap design.  Some
+designs are better in some ways and worse in others.  `Serverboot V2`
+would simplify other early bootstrap tasks, but all that complicated
+logic would be in one binary. One valid criticism of `Serverboot V2`
+is that it will may be a hassle to develop and maintain. In any case,
+trying to code the *best* Hurd bootstrap may be a waste of time. In
+fact, the Hurd bootstrap has been rewritten several times already.
+Our fearless leader, Samuel, feels that rewriting the Hurd bootstrap
+every few years may be a waste of time.  Now that you understand why
+Samuel's discourages a Hurd bootstrap rewrite, let's consider why we
+should develop `Serverboot V2`.
+
+# How ServerBoot V2 will work
+
+Bootstrap begins when Grub and GNU Mach start some tasks, and then GNU
+Mach resumes the not-yet-written
+`/hurd/serverboot`. `/hurd/serverboot` is the only task to accept
+special ports from the kernel via command line arguments like
+`--kernel-task`; `/hurd/serverboot` tries to implement/emulate as much
+of the normal Hurd environment for the other bootstrap translators.
+In particular, it provides the other translators with `stdio`, which
+lets them read/write without having to open the Mach console device.
+This means that the various translators will be able to complain about
+their bad arguments or other startup errors, which they cannot
+currently do.
+
+`/hurd/serverboot` will provide a basic filesystem with netfs, which
+gives the other translators a working `/` directory and `cwd`
+ports. For example, `/hurd/serverboot`, would store its port at
+`/dev/netdde`.  When `/hurd/netdde` starts, it will reply to its
+parent with `fsys_startup ()` as normal.
+
+`/hurd/serverboot` will also emulate the native Hurd process server to
+early bootstrap tasks.  This will allow early bootstrap tasks to get
+the privileged (device master and kernel task) ports via the normal
+glibc function `get_privileged_ports (&host_priv, &device_master).`
+Other tasks will register their message ports with the emulated
+process server.  This will allow signals and messaging during the
+bootstrap. We can even use the existing mechanisms in glibc to set and
+get init ports.  For example, when we start the `auth` server, we will
+give every task started thus far, their new authentication port via
+glibc's `msg_set_init_port ()`.  When we start the real proc server,
+we query it for proc ports for each of the tasks, and set them the
+same way. This lets us migrate from the emulated proc server to the
+real one.
+
+**Fix me: Where does storeio (storeio with**
+`device:@/dev/rumpdisk:wd0`**), rumpdisk, and the pci-arbiter come
+in?**
+
+Next, we start `ext2fs`.  We reattach all the running translators from
+our `netfs` bootstrap filesystem onto the new root.  We then send
+those translators their new root and cwd ports.  This should happen
+transparently to the translators themselves!
+
+# Supporting Netboot
+
+`Serverboot V2` could trivially support netboot by adding `netdde`,
+`pfinet` (or `lwip`), and `isofs` as bootstrap tasks. The bootstrap
+task will start the `pci-arbiter`, and `acpi` (FIXME add some more
+detail to this sentence). The bootstrap task starts `netdde`, which
+will look up any `eth` devices (using the device master port, which it
+queries via the fake process server interface), and sends its fsys
+control port to the bootstrap task in the regular `fsys_startup
+()`. The bootstrap task sets the fsys control port as the translator
+on the `/dev/netdde` node in its `netfs` bootstrap fs. Then
+`/hurd/serverboot` resumes `pfinet`, which looks up
+`/dev/netdde`. Then `pfinet` returns its `fsys` control port to the
+bootstrap task, which it sets on `/servers/socket/2`. Then bootstrap
+resumes `nfs`, and `nfs` just creates a socket using the regular glibc
+socket () call, and that looks up `/servers/socket/2`, and it just
+works. **FIXME where does isofs fit in here?**
+
+Then `nfs` gives its `fsys` control port to `/hurd/serverboot`, which
+knows it's the real root filesystem, so it take the netdde's and
+pfinet's fsys control ports.  Then it calls `file_set_translator ()`
+on the nfs on the same paths, so now `/dev/netdde` and
+`/servers/socket/2` exist and are accessible both on our bootstrap fs,
+and on the new root fs. The bootstrap can then take the root fs to
+broadcast a root and cwd port to all other tasks via a
+`msg_set_init_port ()`. Now every task is running on the real root fs,
+and our little bootstrap fs is no longer used.
+
+`/hurd/serverboot` can resume the exec server (which is the first
+dynamically-linked task) with the real root fs.  Then we just
+`file_set_translator ()` on the exec server to `/servers/exec`, so
+that `nfs` doesn't have to care about this. The bootstrap can now
+spawn tasks, instead of resuming ones loaded by Mach and grub, so it
+next spawns the `auth` and `proc` servers and gives everyone their
+`auth` and `proc` ports. By that point, we have enough of a Unix
+environment to call `fork()` and `exec()`. Then the bootstrap tasks
+would do the things that `/hurd/startup` used to do, and finally
+spawns (or execs) `init / PID 1`.
+
+With this scheme you will be able to use ext2fs to start to your root
+fs via as `/hurd/ext2fs.static /dev/wd0s1`.  This eliminates boot
+arguments like `--magit-port` and `--next-task`.
+
+This also simplifies `libmachdev`, which exposes devices to userspace
+via some Mach `device_*` RPC calls, which lets the Hurd contain device
+drivers instead of GNU Mach. Everything that connects to hardware can
+be a `machdev`.
+
+Additionally, during the `Quiet Boot` bootstrap,`libmachdev` awkwardly
+uses `libtrivfs` to create a transient `/` directory, so that the
+`pci-arbiter` can mount a netfs on top of it at bootstrap.
+`libmachdev` needs `/servers/bus` to mount `/pci,`and it also
+needs `/servers` and `/servers/bus` (and `/dev`, and
+`/servers/socket`). That complexity could be moved to `ServerbootV2`,
+which will create directory nodes at those locations.
+
+`libmachdev` provides a trivfs that intercepts the `device_open` rpc,
+which the `/dev` node uses. It also fakes a root filesystem node, so
+you can mount a `netfs` onto it. You still have to implement
+`device_read` and `device_write` yourself, but that code runs in
+userspace.  An example of this can be found in
+`rumpdisk/block-rump.c`.
+
+`libpciaccess` is a special case: it has two modes, the first time it
+runs via `pci-arbiter`, it acquires the pci config IO ports and runs
+as x86 mode. Every subsequent access of pci becomes a hurdish user of
+pci-arbiter.
+
+`rumpdisk` exposes `/dev/rumpdisk`:
+
+```
+$ showtrans /dev/rumpdisk
+  /hurd/rumpdisk
+```
+
+
+# FAQ
+
+## `Server Boot V2` looks like a ramdisk + a script...?
+
+Its not quite a ramdisk, its more a netfs translator that
+creates a temporary `/`.  Its a statically linked binary. I don't
+think it differs from a multiboot module.
+
+## How are the device nodes on the bootstrap netfs attached to each translator?
+## How does the first non-bootstrap task get invoked?
+## does bootstrap resume it?
+## Could we just use a ram disk instead?
+## One could stick an unionfs on top of it to load the rest of the system after bootstrap.
+
+It looks similar to a ramdisk in principle, i.e. it exposes a fs which
+lives only in ram, but a ramdisk would not help with early bootstrap.
+Namely during early bootstrap, there are no signals or console.
+Passing control from from one server to the next via a bootstrap port
+is a kludge at best. How many times have you seen the bootstrap
+process hang and just sit there?  `Serverboot V2` would solve that.
+Also, it would allow subhurds to be full hurds without special casing
+each task with bootstrap code.  It would also clean up `libmachdev`,
+and Damien, its author, is in full support.
+
+## A ramdisk could implement signals and stdio.  Isn't that more flexible?
+
+But if its a ramdisk essentially you have to provide it with a tar
+image.  Having it live inside a bootstrap task only is
+preferable. Also the task could even exit when its done whether you
+use an actual ramdisk or not. You still need to write the task that
+boots the system.  That is different than how it works currently. Also
+a ramdisk would have to live in mach, and we want to move things out
+of mach.
+
+Additionally, the bootstrap task will be loaded as the first multiboot
+module by grub.  It's not a ramdisk, because a ramdisk has to contain
+some fs image (with data), and we'd need to parse that format.  It
+might make sense to steer it more into that direction (and Samuel
+seems to have preferred it), because there could potentially be some
+config files, or other files that the servers may need to run. I'm not
+super fond of that idea. I'd prefer the bootstrap fs to be just a
+place where ports (translators) can be placed and looked up. Actually
+in my current code it doesn't even use `netfs`, it just implements the
+RPCs directly.  I'll possibly switch to `netfs` later, or if the
+implementation stays simple, I won't use `netfs`.
+
+## Serverboot V2 just rewrites proc and exec.  Why reimplement so much code?
+
+I don't want to exactly reimplement full `proc` and `exec` servers in the
+bootstrap task, it's more of providing very minimal emulation of some
+of their functions.  I want to implement the two RPCs from the
+`proc` interface, one to give a task the privileged ports on request and
+one to let the task give me its msg port.  That seems fairly simple to
+me.
+
+While we were talking of using netfs, my actual implementation doesn't
+even use that, it just implements the RPCs directly (not to suggest I
+have anything resembling a complete implementation). Here's some
+sample code to give you an idea of what it is like
+
+
+	error_t
+	S_proc_getprivports (struct bootstrap_task *task,
+                     mach_port_t *host_priv,
+                     mach_port_t *device_master)
+	{
+ 		if (!task)
+         return EOPNOTSUPP;
+
+      if (bootstrap_verbose)
+        fprintf (stderr, "S_proc_getprivports from %s\n", task->name);
+
+      *host_priv = _hurd_host_priv;
+      *device_master = _hurd_device_master;
+
+      return 0;
+    }
+
+	error_t
+	S_proc_setmsgport (struct bootstrap_task *task,
+                   mach_port_t reply_port,
+                   mach_msg_type_name_t reply_portPoly,
+                   mach_port_t newmsgport,
+                   mach_port_t *oldmsgport,
+                   mach_msg_type_name_t *oldmsgportPoly)
+	{
+		if (!task)
+			return EOPNOTSUPP;
+
+	    if (bootstrap_verbose)
+			fprintf (stderr, "S_proc_setmsgport for %s\n", task->name);
+
+	    *oldmsgport = task->msgport;
+	    *oldmsgportPoly = MACH_MSG_TYPE_MOVE_SEND;
+
+	    task->msgport = newmsgport;
+
+	    return 0;
+	    }
+
+Yes, it really is just letting tasks fetch the priv ports (so
+`get_privileged_ports ()` in glibc works) and set their message ports.
+So much for a slippery slope of reimplementing the whole process
+server :)
+
+
+## Let's bootstrap like this: initrd, proc, exec, acpi, pci, drivers,
+## unionfs+fs with every server executable included in the initrd tarball?
+
+I don't see how that's better, but you would be able to try something
+like that with my plan too.  The OS bootstrap needs to start servers
+and integrate them into the eventual full hurd system later when the
+rest of the system is up.  When early servers start, they're running
+on bare Mach with no processes, no `auth`, no files or file
+descriptors, etc.  I plan to make files available immediately (if not
+the real fs), and make things progressively more "real" as servers
+start up.  When we start the root fs, we send everyone their new root
+`dir` port.  When we start `proc`, we send everyone their new `proc`
+port.  and so on.  At the end, all those tasks we have started in
+early boot are full real hurd proceses that are not any different to
+the ones you start later, except that they're statically linked, and
+not actually `io map`'ed from the root fs, but loaded by Mach/grub
+into wired memory.
+
+# IRC Logs
+
+    <damo22> showtrans /dev/wd0 and you can open() that node and it will
+    act as a device master port, so you can then `device_open` () devices
+    (like wd0) inside of it, right?
+
+    oh it's a storeio, that's… cute. that's another translator we'd need
+    in early boot if we want to boot off /hurd/ext2fs.static /dev/wd0
+
+    <damo22> We implemented it as a storeio with
+	device:@/dev/rumpdisk:wd0
+
+	so the `@` sign makes it use the named file as the device master, right?
+
+	<damo22> the `@` symbol means it looks up the file as the device
+	master yes.  Instead of mach, but the code falls back to looking up
+	mach, if it cant be found.
+
+	I see it's even implemented in libstore, not in storeio, so it just
+	does `file_name_lookup ()`, then `device_open` on that.
+
+	<damo22> pci-arbiter also needs acpi because the only way to know the
+	IRQ of a pci device reliably is to use ACPI parser, so it totally
+	implements the Mach `device_*` functions. But instead of handling the
+	RPCs directly, it sets the callbacks into the
+	`machdev_device_emulations_ops` structure and then libmachdev calls
+	those. Instead of implementing the RPCs themselves, It abstracts them,
+	in case you wanted to merge drivers. This would help if you wanted
+	multiple different devices in the same translator, which is of course
+	the case inside Mach, the single kernel server does all the devices.
+
+	but that shouldn't be the case for the Hurd translators, right? we'd
+	just have multiple different translators like your thing with rumpdisk
+	and rumpusb.
+
+	`<damo22>`	i dont know
+
+	ok, so other than those machdev emulation dispatch, libmachdev uses
+	trivfs and does early bootstrap. pci-arbiter uses it to centralize the
+	early bootstrap so all the machdevs can use the same code. They chain
+	together. pci-arbiter creates a netfs on top of the trivfs. How
+	well does this work if it's not actually used in early bootstrap?
+
+	<damo22> and rumpdisk opens device ("pci"), when each task is resumed,
+	it inherits a bootstrap port
+
+	and what does it do with that? what kind of device "pci" is?
+
+	<damo22> its the device master for pci, so rumpdisk can call
+	pci-arbiter rpcs on it
+
+	hm, so I see from the code that it returns the port to the root of its
+	translator tree actually. Does pci-arbiter have its own rpcs? does it
+	not just expose an fs tree?
+
+	<damo22> it has rpcs that can be called on each fs node called
+	"config" per device: hurd/pci.defs. libpciaccess uses these.
+
+	how does that compare to reading and writing the fs node with regular read and write?
+
+	<damo22> so the second and subsequent instances of pciaccess end up
+	calling into the fs tree of pci-arbiter. you can't call read/write on
+	pci memory its MMIO, and the io ports need `inb`, `inw`, etc. They
+	need to be accessed using special accessors, not a bitstream.
+
+	but I can do $ hexdump /servers/bus/pci/0000/00/02/0/config
+
+	<damo22> yes you can on the config file
+
+	how is that different from `pci_conf_read` ?  it calls that.
+
+	<damo22> the `pci fs` is implemented to allow these things.
+
+	why is there a need for `pci_conf_read ()` as an RPC then, if you can
+	instead use `io_read` on the "config" node?
+
+	<damo22> i am not 100% sure. I think it wasn't fully implemented from
+	the beginning, but you definitely cannot use `io_read ()` on IO
+	ports. These have explicit x86 instructions to access them
+	MMIO. maybe, im not sure, but it has absolute physical addressing.
+
+	I don't see how you would do this via `pci.defs` either?
+
+	<damo22> We expose all the device tree of pci as a netfs
+	filesystem. It is a bus of devices. you may be right. It would be best
+	to implement pciaccess to just read/write from the filesystem once its
+	exposed on the netfs.
+
+	yes, the question is:
+
+	1 is there anything that you can do by using the special RPCs from
+	pci.defs that you cannot do by using the regular read/write/ls/map
+	on the exported filsystem tree,
+	2 if no, why is there even a need for `pci.defs`, why not always use
+	the fs? But anyway, that's irrelevant for the question of bootstrap
+	and libmachdev
+
+	<damo22> There is a need for rpcs for IO ports.
+
+	Could you point me to where rumpdisk does `device_open ("pci")`? grep
+	doesn't show anything. which rpcs are for the IO ports?
+
+	<damo22> They're not implemented yet we are using raw access I
+	think. The way it works, libmachdev uses the next port, so it all
+	chains together: `libmachdev/trivfs_server.c`.
+
+	but where does it call `device_open ("pci")` ?
+
+	<damo22> when the pci task resumes, it has a bootstrap port, which is
+	passed from previous task. There is no `device_open ("pci")`.  or if
+	its the first task to be resumed, it grabs a bootstrap port from
+	glibc? im not sure
+
+	ok, so if my plan is implemented how much of `libmachdev` functionality
+	will still be used / useful?
+
+	<damo22> i dont know.  The mach interface? device interface\*. maybe
+	it will be useless.
+
+	I'd rather you implemented the Mach device RPCs directly, without the
+	emulation structure, but that's an unrelated change, we can leave that
+	in for now.
+
+	<damo22> I kind of like the emulation structure as a list of function
+	pointers, so i can see what needs to be implemented, but that's
+	neither here nor there.  `libmachdev` was a hack to make the bootstrap
+	work to be honest.…and we'd no longer need that. I would be happy if
+	it goes away.  the new one would be so much better.
+
+	is there anything else I should know about this all? What else could
+	break if there was no libmachdev and all that?
+
+	<damo22> acpi, pci-arbiter, rumpdisk, rumpusbdisk
+
+	right, let's go through these
+
+	<damo22> The pci-arbiter needs to start first to claim the x86 config
+	io ports.  Then gnumach locks these ports.  No one else can use them.
+
+	so it starts and initializes **something** what does it need?  the
+	device master port, clearly, right?  that it will get through the
+	glibc function / the proc API
+
+	<damo22> it needs a /servers/bus and the device master
+
+	<solid_black>
+	right, so then it just does fsys_startup, and the bootstrap task
+	places it onto `/servers/bus` (it's not expected to do
+	`file_set_translator ()` itself, just as when running as a normal
+	translator)
+
+	<damo22> it exposes a netfs on `/servers/bus/pci`
+
+	<solid_black> so will pci-arbiter still expose mach devices? a mach
+	device master?  or will it only expose an fs tree + pci.defs?
+
+	<damo22> i think just fs tree and pci.defs. should be enough
+
+	<solid_black> ok, so we drop mach dev stuff from pci-arbiter
+	completely. then acpi starts up, right? what does it need?
+
+	<damo22> It needs access to `pci.defs` and the pci tree. It
+	accesses that via libpciaccess, which calls a new mode that
+	accesses the fstree. It looks up `servers/bus/pci`.
+
+	ok, but how does that work now then?
+
+	<damo22> It looks up the right nodes and calls pci.defs on them.
+
+	<solid_black> looks up the right node on what? there's no root
+	filesystem at that point (in the current scheme)
+
+	`<damo22>` It needs pci access
+
+	that's why I was wondering how it does `device_open ("pci")`
+
+	<damo22> I think libmachdev from pci gives acpi the fsroot. there is a
+	doc on this.
+
+	so does it set the root node of pci-arbiter as the root dir of acpi?
+	as in, is acpi effectively chrooted to `/servers/bus/pci`?
+
+	<damo22> i think acpi is chrooted to the parent of /servers. It shares
+	the same root as pci's trivfs.
+
+	i still don't quite understand how netfs and trivfs within pci-arbiter interact.
+
+	<damo22> you said there would be a fake /. Can't acpi use that?
+
+	<solid_black> yeah, in my plan / the new bootstrap scheme, there'll be
+	a / from the very start.
+
+	<damo22> ok so acpi can look up /servers/bus/pci, and it will exist.
+
+	and pci-arbiter can really sit on `/servers/bus/pci` (no need for
+	trivfs there at all) and acpi will just look up
+	`/servers/bus/pci`. And we do not need to change anything in acpi to
+	get it to do that.
+
+	And how does it do it now? maybe we'd need to remove some
+	no-longer-required logic from acpi then?
+
+	<damo22> it looks up device ("pci") if it exists, otherwise it falls
+	back to `/servers/bus/pci`.
+
+	Ah hold on, maybe I do understand now.  currently pci-arbiter exposes
+	its mach dev master as acpi-s mach dev master. So it looks up
+	device("pci") and finds it that way.
+
+	<damo22> correct, but it doesnt need that if the `/` exists.
+
+	yeah, we could remove this in the new bootstrap scheme, and just
+	always open the fs node (or leave it in for compatibility, we'll see
+	about that). acpi just sits on `/servers/acpi/tables`.
+
+	`rumpdisk` runs next and it needs `/servers/bus/pci`, `pci.defs`, and
+	`/servers/acpi/tables`, and `acpi.defs`. It exposes `/dev/rumpdisk`.
+
+	Would it make sense to make rumpdisk expose a tree/directory of Hurd
+	files and not Mach devices?  This is not necessary for anything, but
+	just might be a nice little cleanup.
+
+	<damo22> well, it could expose a tree of block devices, like
+	`/dev/rumpdisk/ide/1`.
+
+	<solid_black> and then `ln -s /rumpdisk/ide/1 /dev/wd1`.  and no need
+	for an intermediary storeio.  plus the Hurd file interface is much
+	richer than Mach device, you can do fsync for instance.
+
+	<damo22> the rump kernel is bsd under the hood, so needs to be
+	`/dev/rumpdisk/ide/wd0`
+
+	<solid_black> You can just convert "ide/0" to "/dev/wd0" when
+	forwarding to the rump part. Not that I object to ide/wd0, but we can
+	have something more hierarchical in the exposed tree than old-school
+	unix device naming?  Let's not have /dev/sda1.  Instead let's have
+	/dev/sata/0/1, but then we'd still keep the bsd names as symlinks into
+	the *dev/rumpdisk*…  tree
+
+	<damo22> sda sda1
+
+	<solid_black> good point
+
+	<damo22> 0 0/1
+
+	<solid_black> well, you can on the Hurd :D and we won't be doing that
+	either, rumpdisk only exposes the devices, not partitions
+
+	<damo22> well you just implement a block device on the directory?  but
+	that would be confusing for users.
+
+	<solid_black> I'd expect rumpdisk to only expose device nodes, like
+	/dev/rumpdisk/ide/0, and then we'd have /dev/wd0 being a symlink to
+	that. And /dev/wd0s1 being a storeio of type part:1:/dev/wd0 or
+	instead of using that, you could pass that as an option to your fs,
+	like ext2fs -T typed part:1/dev/wd0
+
+	<damo22> where is the current hurd bootstrap (QuietBoot) docs hosted?
+	here:
+	https://git.savannah.gnu.org/cgit/hurd/web.git/plain/hurd/bootstrap.mdwn
+
+	<solid_black> so yeah, you could do the device tree thing I'm
+	proposing in rumpdisk, or you could leave it exposing Mach devices and
+	have a bunch of storeios pointing to that. So anyway, let's say
+	rumpdisk keeps exposing a single node that acts as a Mach device
+	master and it sits on /dev/rumpdisk.
+
+	<solid_black> Then we either need a storeio, or we could make ext2fs
+	use that directly. So we start `/hurd/ext2fs.static -T typed
+	part:1:@/dev/rumpdisk:wd0`.
+
+	<solid_black> I'll drop all the logic in libdiskfs for detecting if
+	it's the bootstrap filesystem, and starting the exec server, and
+	spawning /hurd/startup. It'll just be a library to help create
+	filesystems.
+
+	<solid_black> After that the bootstrap task migrates all those
+	translator nodes from the temporary / onto the ext2fs, broadcasts the
+	root and cwd ports to everyone, and off we go to starting auth and
+	proc and unix.  sounds like it all would work indeed.  so we're just
+	removing libmachdev completely, right?
+
+	<damo22> netdde links to it too. I think it has libmachdevdde
+
+	<solid_black> Also how would you script this thing. Like ideally we'd
+	want the bootstrap task to follow some sort of script which would say,
+	for example,
+
+	mkdir /servers
+	mkdir /servers/bus
+	settrans /servers/bus/pci ${pci-task} --args-to-pci
+	mkdir /dev
+	settrans /dev/netdde ${netdde-task} --args-to-netdde
+	setroot ${ext2fs-task} --args-to-ext2fs
+
+	<solid_black> and ideally the bootstrap task would implement a REPL
+	where you'd be able to run these commands interactively (if the
+	existing script fails for instance). It can be like grub, where it has
+	a predefined script, and you can do something (press a key combo?) to
+	instead run your own commands in a repl.  or if it fails, it bails out
+	and drops you into the repl, yes. this gives you **so much more**
+	visibility into the boot process, because currently it's all scattered
+	across grub, libdiskfs (resuming exec, spawning /hurd/startup),
+	/hurd/startup, and various tricky pieces of logic in all of these
+	servers.
+
+	<solid_black> We could call the mini-repl hurdhelper? If something
+	fails, you're on your own, at best it prints an error message (if the
+	failing task manages to open the mach console at that point) Perhaps
+	we call the new bootstrap proposal Bootstrap.
+
+	<solid_black> When/if this is ready, we'll have to remove libmachdev
+	and port everything else to work without it.
+
+	<damo22> yes its a great idea.  I'm not a fan of lisp either.  If i
+	keep in mind that `/` is available early, then I can just clean up the
+	other stuff.  and assume i have `/`, and the device master can be
+	accessed with the regular glibc function, and you can printf freely
+	(no need to open the console). Do i need to run `fsys_startup` ?
+
+	yes, exactly like all translators always do. Well you probably run
+	netfs_startup or whatever, and it calls that. you're not supposed to
+	call fsys_getpriv or fsys_init
+
+	<damo22> i think my early attempts at writing translators did not use
+	these, because i assumed i had `/`. Then i realised i didn\`t. And
+	libmachdev was born.
+
+	<solid-black> Yes, you should assume you have /, and just do all the
+	regular things you would do. and if something that you would usually
+	do doesn't work, we should think of a way to make it work by adding
+	more stuff in the bootstrap task when it's reasonable to, of
+	course. and please consider exposing the file tree from rumpdisk,
+	though that's orthogonal.
+
+	<damo22> you mean a tree of block devices?
+
+	<solid_black> Yes, but each device node would be just a Hurd (device)
+	file, not a Mach device.  i.e. it'd support io_read and io_write, not
+	device_read and device_write.  well I guess you could make it support
+	both.
+
+	<damo22>	isnt that storeio's job?
+
+	<solid_black> if a node only implements the device RPCs, we need a
+	storeio to turn it into a Hurd file, yes.  but if you would implement
+	the file RPCs directly, there wouldn't be a need for the intermediary
+	storeio, not that it's important.
+
+	<damo22> but thats writing storeio again.  thing is, i dont know at
+	runtime which devices are exposed by rump.  It auto probes them and
+	prints them out but i cant tell programmatically which ones were
+	detected, becuause rump knows which devices exist but doesn't expose
+	it over API in any way. Because it runs as a kernel would with just
+	one driver set.
+
+	<damo22> Rump is a decent set of drivers. It does not have better
+	hardware support than Linux drivers (of modern Linux)? Instead Rump is
+	netbsd in a can, and it's essentially unmaintained upstream
+	too. However, it still is used it to test kernel modules, but it lacks
+	makefiles to separate all drivers into modules. BUT using rump is
+	better than updating / redoing the linux drivers port of DDE, because
+	netbsd internal kernel API is much much more stable than linux. We
+	would fall behind in a week with linux.  No one would maintain the
+	linux driver -> hurd port.  Also, there is a framework that lets you
+	compile the netbsd drivers as userspace unikernels: rump.  Such a
+	thing only does not exist for modern Linux. Rump is already good
+	enough for some things. It could replace netdde. It already works for
+	ide/sata.
+
+	<damo22> Rump it has its own /dev nodes on a rumpfs, so you can do
+	something like `rump_ls` it.
+
+	<damo22> Rump is a minimal netbsd kernel. It is just the device
+	drivers, and a bit of pthreading, and has only the drivers that you
+	link. So rumpdisk only has the ahci and ide drivers and nothing
+	else. Additionally rump can detect them off the pci bus.
+
+	<damo22> I will create a branch on
+	<http://git.zammit.org/hurd-sv.git> with cleaned translators.
+
+	<damo22> solid_black: i almost cleaned up acpi and pci-arbiter but
+	realised they are missing the shutdown notification when i strip out
+	libmachdev.
+
+	<solid-black>: "how are the device nodes on the bootstrap netfs attached to
+	each translator?" – I don't think I understand the question, please
+	clarify.
+
+	<damo22> I was wondering if the new bootstrap process can resume a fs
+	task and have all the previous translators wake up and serve their
+	rpcs.  without needing to resume them.  we have a problem with the
+	current design, if you implement what we discussed yesterday, the IO
+	ports wont work because they are not exposed by pci-arbiter yet.  I am
+	working on it, but its not ready.
+
+	<solid_black> I still don't understand the problem.  the bootstrap
+	task resumes others in order.  the root fs task too, eventually, but
+	not before everything that hash to come up before the root fs task is
+	ready.
+
+	<damo22> I don't think it needs to be a disk. Literally a trivfs is enough.
+
+	<solid_black> why are I/O ports not exposed by pci-arbiter? why isn't
+	that in issue with how it works currently then?
+
+	<damo22> solid_black: we are using ioperm() in userspace, but i want
+	to refactor the io port usage to be granularly accessed.  so one day
+	gnumach can store a bitmap of all io ports and reject more than one
+	range that overlaps ports that are in use.  since only one user of any
+	port at any time is allowed.  i dont know if that will allow users to
+	share the same io ports, but at least it will prevent users from
+	clobbering each others hw access.
+
+	<solid_black> damo22: (again, sorry for not understanding the hardware
+	details), so what would be the issue? when the pci arbiter starts,
+	doesn't it do all the things it has to do with the I/O ports?
+
+	<damo22> io ports are only accessed in raw method now. Any user can do
+	ioperm(0, 0xffff, 1) and get access to all of them
+
+	<solid_black> doesn't that require host priv or something like that?
+
+	<damo22> yeh probably.  maybe only root can.  But i want to allow
+	unprivileged users to access io ports by requesting exclusive access
+	to a range.
+
+	<solid_black> I see that ioperm () in glibc uses the device master
+	port, so yeah, root-only (good)
+
+	`<damo22>` first in locks the port range
+
+	<solid_black> but you're saying that there's someting about these I/O
+	ports that works today, but would break if we implemented what we
+	discussed yeasterday? what is it, and why?
+
+	`<damo22>` well it might still work.  but there's a lot of changes to
+	be done in general
+
+	<solid_black> let me try to ask it in a different way then
+
+	<damo22> i just know a few of the specifics because i worked on them.
+
+	<solid_black> As I understand it, you're saying that 1: currently any
+	root process can request access to any range of I/O ports, and you
+	also want to allow **unprivileged** processes to get access to ranges
+	of I/O ports, via a new API of the PCI arbiter (but this is not
+	implemented yet, right?)
+
+	<damo22> yes
+
+	<solid_black> 2: you're saying that something about this would break /
+	be different in the new scheme, compared to the current scheme.  i
+	don't understand the 2, and the relation between 1 and 2.
+
+	<damo22> 2 not really, I may have been mistaken it probably will
+	continue working fine.  until i try to implement 1.  ioperm calls
+	`i386_io_perm_create` and `i386_io__perm_modify` in the same system
+	call. I want to seperate these into the arbiter so the request goes
+	into pci-arbiter and if it succeeds, then the port is returned to the
+	caller and the caller can change the port access.
+
+	<solid_black> yes, so what about 2 will break 1 when you try to implement it?
+
+	<damo22> with your new bootstrap, we need `i386_io_perm_*` to be
+	accessible.  im not sure how.  is that a mach rpc?
+
+	<solid_black> these are mach rpcs. i386_io_perm_create is an rpc that
+	you do on device master.
+
+	<damo22> should be ok then
+
+	<solid_black> i386_io_perm_modify you do on you task port.  yes, I
+	don't see how this would be problematic.
+
+	<damo22>: you might find this branch useful
+	<http://git.zammit.org/hurd-sv.git/log/?h=feat-simplify-bootstrap>
+
+	<solid_black> although:
+
+	1. I'm not sure whether the task itself should be wiring its memory,
+	or if the bootstrap task should do it.
+	2. why do you request startup notifications if you then never do
+	anything in `S_startup_dosync`?
+
+	<solid_black> same for essential tasks actaully, that should probably
+	be done by the bootstrap task and not the translator itself (but we'll
+	see)
+
+	<solid_black> 1. don't `mach_print`, just `fprintf (stderr, "")`
+	<solid_black> 2. please always verify the return result of
+	`mach_port_deallocate` (and similar functions),
+	typically like this:
+
+	err = mach_port_deallocate (…);
+	assert_perror_backtrace (err);
+
+	this helps catch nasty bugs.
+
+	<solid_black> 3. I wonder why both acpi and pci have their own
+	`pcifs_startup` and `acpifs_startup`; can't they use `netfs_startup
+	()`?
+
+	`<damo22>` 1. no idea, 2. rumpdisk needed it, but these might
+	not 3. ACK, 4.ACK, 5. I think they couldnt use the `netfs_startup ()`
+	before but might be able to now.  Anyway, this should get you booting
+	with your bootstrap translator (without rumpdisk).  Rumpdisk seems to
+	use the `device_* RPC` from `libmachdev` to expose its device.
+	whereas pci and acpi dont use them for anything except `device_open`
+	to pass their port to the next translator.  I think my latest patch
+	for io ports will work.  but i need to rebuild glibc and libpciaccess
+	and gnumach. Why does libhurduser need to be in glibc?  It's quite
+	annoying to add an rpc.
+
+	I think i have done gnumach io port locking, and pciaccess, but hurd
+	part needs work and then to merge it needs a rebuild of glibc because
+	of hurduser
+
+	<damo22> Why cant libhurduser be part of the hurd package?
+
+	I don't think I understnad enough of this to do a review, but I'd
+	still like to see the patch if it's available anywhere.
+
+	<damo22> ok i can push to my repos
+
+	<solid_black> glibc needs to use the Hurd RPCs (and implement some,
+	too), and glibc cannot depend on the Hurd package because the Hurd
+	package depends on glibc.
+
+	<damo22> lol ok
+
+	<solid_black> As things currently stand, glibc depends on the Hurd
+	**headers** (including mig defs), but not any Hurd binaries.  still,
+	the cross build process is quite convoluted.  I posted about it
+	somewhere: https://floss.social/@bugaevc/109383703992754691
+
+	<jpoiret> the manual patching of the build system that's needed to
+	bootstrap everything is a bit suboptimal.
+
+	<damo22> what if you guys submit patches upstream to glibc to add a
+	build target to copy the headers or whatever is needed?  solid_black:
+	see
+	[http://git.zammit.org/{libpciaccess.git,gnumach.git](http://git.zammit.org/%7Blibpciaccess.git,gnumach.git)}
+	on fix-ioperm branches