diff options
Diffstat (limited to 'microkernel/mach')
47 files changed, 1555 insertions, 529 deletions
diff --git a/microkernel/mach/concepts.mdwn b/microkernel/mach/concepts.mdwn index 04dbb1c6..0f7cbf00 100644 --- a/microkernel/mach/concepts.mdwn +++ b/microkernel/mach/concepts.mdwn @@ -1,6 +1,33 @@ -[[Mach]] is a first-generation [[microkernel]]. Mach's basic abstractions -include [[address_space]]s in the form of [[task]]s, execution contexts in the -form of [[thread]]s, [[IPC]], [[capabilities|capability]] in the form of [[port]]s, and -[[memory_object]]s, which enable Mach's [[external_pager_mechanism]]. +[[!meta copyright="Copyright © 2002, 2003, 2007, 2010 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[Mach]] is a first-generation [[microkernel]]. + +Mach's basic abstractions include [[virtual_address_space]]s in the form of +[[task]]s, execution contexts in the form of [[thread]]s, [[IPC]], +[[capabilities|capability]] in the form of [[port]]s, and [[memory_object]]s, +which enable Mach's [[external_pager_mechanism]]. + +Controlling [[task]]s, their [[virtual_address_space]], [[thread]]s, and other +system objects in Mach is implemented by using [[port]]s, as opposed to other +[[kernel]]s' [[system_call]] interface: almost all of the Mach API is +implemented by sending [[message]]s to [[port]]s. Device drivers that reside +in kernel space are controlled by ports, too. Mach's [[API]] is well-[[documented|documentation]]. + +[[!toggleable id=mach_kernel_principles text="""[[!template id=note +text="*[[mach\_kernel\_principles|documentation]]*: +{{$mach#kernel_principles}}"]]"""]] + +In particular the [[!toggle id=mach_kernel_principles +text="[mach\_kernel\_principles]"]] book further elaborates on Mach's concepts +and principles. diff --git a/microkernel/mach/continuation.mdwn b/microkernel/mach/continuation.mdwn new file mode 100644 index 00000000..7a3267f3 --- /dev/null +++ b/microkernel/mach/continuation.mdwn @@ -0,0 +1,24 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[Mach]] internally uses *continuation*s for kernel [[thread]] management. + +The advantage is that not a full kernel thread stack has to be preserved in +case that a thread is about to enter a blocking state. This saves space. It +is not clear this is still worthwhile given today's RAM offerings. (How many +kernel threads are there, typically?) + +And, this would no longer be possible in case Mach were be made a +[[preemptive|preemtion]] kernel. In the latter case, the kernel itself, that +is, kernel threads can be preempted, and then their full state needs to be +preserved. + +[[!tag open_issue_documentation]] <!-- Not linked to from any Mach page. Move +to GNU Mach pages, as this is only an implementation detail? --> diff --git a/microkernel/mach/discussion.mdwn b/microkernel/mach/discussion.mdwn deleted file mode 100644 index 589e302d..00000000 --- a/microkernel/mach/discussion.mdwn +++ /dev/null @@ -1,23 +0,0 @@ -## <a name="Maintenance_of_the_Mach_web"> Maintenance of the Mach web </a> - -**_Old discussions:_** [[WIKIHOMEURLMachTOPICrev13]] - -Interesting, for consistency sake I'll think about making your changes you made on the right hand side to the other web WebHome pages. I guess it's not critical that they are identical, but I was trying to keep them identical if possible. I also wanted it to be "light" enough feature wise that it doesn't overpower the page. You've added back a few of the features, so we obviously differ in how important you and I think these features are. That's OK, I'll think about it some more and we'll see what happens. - -Oh, I see you added back [[WebTopicList]] and [[WebPreferences]]. I purposely removed [[WebPreferences]] from the lists on the right because it has nothing to do with navigation. I also didn't think that people actually use topic names to navigate. If they do they could search for them. Keeping the number to four items instead of six and keeping the descriptions concise makes a big difference when I view the page. - -(goes off to think more...) - -and eat... ;-) - --- [[Main/GrantBow]] - 29 Dec 2002 - -**_Reasons for my change:_** - -1. [[WebTopicList]] is a lot quicker than the [[WebIndex]] - brings down the load times and the load of the server -2. [[WebPreferences]] - users might be curious to see what can be modified. Changes should of course only be made in their home topics, like in %WIKIUSERNAME%. However, the [[WebPreferences]] can serve as an inspiration. Therefore we should perhaps make sure only the [[Main/TWikiAdminGroup]] members can alter the \*Preferences topics. -3. If you look closely I've also reordered the links. Shorter names first and long ones last, I tried to keep the descriptions brief and in proportional length as well. - -I don't know about you, but keeping the number of items to four rather than six doesn't really matter to me. The text is quite small and if it's the space we're after the [[WebStatistics]] does take up more than the navigation links. - --- [[Main/JoachimNilsson]] - 29 Dec 2002 diff --git a/microkernel/mach/documentation.mdwn b/microkernel/mach/documentation.mdwn index 83e2d0e2..cc880ab6 100644 --- a/microkernel/mach/documentation.mdwn +++ b/microkernel/mach/documentation.mdwn @@ -1,17 +1,49 @@ -[[meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008, +2010 Free Software Foundation, Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] - - [Meet Mach](http://www.stepwise.com/Articles/Technical/MeetMach.html), a - summary of Mach's history and main concepts. + * Mach's [[concepts]]. - - OSF's [Kernel - Interface](ftp://ftp.cs.cmu.edu/afs/cs/project/mach/public/doc/osf/kernel_interface.ps) - - OSF's [Kernel - Principles](ftp://ftp.cs.cmu.edu/afs/cs/project/mach/public/doc/osf/kernel_principles.ps) + * [*Meet Mach* by James + Scott](http://beefchunk.com/documentation/macosx-programming/Meet_Mach.pdf), + a summary of Mach's history and main concepts. + + * *[[The_GNU_Mach_Reference_Manual|gnumach/reference_manual]]*. + + * {{$mach#kernel_foundation_unix}} + + * {{$mach#vm}} + + * {{$mach#kernel_principles}} + + * {{$mach#kernel_interface}} + + * {{$mach#server_writer}} + + * {{$mach#server_interface}} + + * [*The Unofficial GNU Mach IPC beginner's + guide*](http://hurdextras.nongnu.org/ipc_guide/), an easy introduction to + Inter Process Comunication in the Mach microkernel by Manuel Pavón + Valderrama. + + * [*Mach IPC without + MIG*](http://walfield.org/pub/people/neal/papers/hurd-misc/mach-ipc-without-mig.txt), + an exercise by Neal Walfield *to understand Mach IPC at one of its lowest + application levels*. + + * [*ipc-hello.c*](http://walfield.org/pub/people/neal/papers/hurd-misc/ipc-hello.c): + *Hello world à la mach ipc*. + + - [Porting and Modifying the Mach 3.0 Microkernel](http://shakthimaan.com/downloads/hurd/Porting%20and%20Modifying%20the%20Mach%203.0%20Microkernel.pdf) + + - [An IO System for Mach](http://shakthimaan.com/downloads/hurd/An%20IO%20System%20for%20Mach.pdf) + + - [A Programmers' Guide to Mach System Call](http://shakthimaan.com/downloads/hurd/A.Programmers.Guide.to.the.Mach.System.Calls.pdf) diff --git a/microkernel/mach/external_pager_mechanism.mdwn b/microkernel/mach/external_pager_mechanism.mdwn index 7ed1ca57..05a6cc56 100644 --- a/microkernel/mach/external_pager_mechanism.mdwn +++ b/microkernel/mach/external_pager_mechanism.mdwn @@ -1,91 +1,93 @@ -[[meta copyright="Copyright © 2002, 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2002, 2007, 2008, 2010, 2011 Free Software +Foundation, Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] -Mach provides a so-called external pager [[mechanism]]. This +Mach provides a so-called *external pager [[mechanism]]*. This mechanism serves to separate *managing memory* from *managing -content*. Mach does the former while user space tasks do the +content*. Mach does the former while user-space processes do the latter. +[[!tag open_issue_documentation]] <!-- Should probably refer to {{$mach#vm}}. +--> + # Introduction -In Mach, a task's [[Mach/AddressSpace]] consists of references -to [[Mach/MemoryObjects]]. A memory object is designated using -a [[port]] (a port is just a [[capability]]) and -implemented by a normal process. +In Mach, a [[task]]'s [[virtual_address_space]] consists of references to +[[memory_object]]s. To associate a memory object with a portion of a task's -address space, vm\_map is invoked a capability designating +address space, `vm_map` is invoked on a capability designating the task and passing a reference to the memory object and the offset at which to install it. (The first time a task maps an object, Mach sends an initialization message to the server including a control capability, which it uses to supply pages to the kernel.) This is essentially -the same as mapping a file into an address space on Unix -using mmap. +the same as mapping a file into an address space on [[UNIX]] +using `mmap`. -When a task faults, Mach checks to see if there is a memory +When a task [[faults|page_fault]], Mach checks to see if there is a memory object associated with the fault address. If not, the task -is sent an exception, which is normally further propagated +is sent an [[exception]], which is normally further propagated as a segmentation fault. If there is an associated memory -object, Mach checks whether the corresponding page is in core. -If it is, it installs the page and resumes the task. Mach -then invokes the memory object with the memory\_object\_request +object, Mach checks whether the corresponding [[page]] is in core. +If it is, it installs the page and resumes the task. Mach +then invokes the memory object with the `memory_object_request` method and the page to read. The memory manager then fetches or creates the content as appropriate and supplies it to -Mach using the memory\_object\_supply method. +Mach using the `memory_object_supply` method. # Creating and Mapping a Memory Object The following illustrates the basic idea: -> ________ -> / \ -> | Mach | -> \________/ -> /| / |\ \ -> (C) vm_map / / m_o_ready (E)\ \ (D) memory_object_init -> / |/ (F) return \ \| -> ________ ________ -> / \ -----> / \ -> | Client | (A) open | Server | -> \________/ <----- \________/ -> (B) memory_object - -(A) The client sends an "open" rpc to the server. + ________ + / \ + | Mach | + \________/ + /| / |\ \ + (C) vm_map / / m_o_ready (E)\ \ (D) memory_object_init + / |/ (F) return \ \| + ________ ________ + / \ -----> / \ + | Client | (A) open | Server | + \________/ <----- \________/ + (B) memory_object + +(A) The client sends an `open` [[RPC]] to the server. (B) The server creates a memory object (i.e., a port receive right), adds it to the port set that it is listening on and returns a capability (a port send right) to the client. (C) The client attempts to map the object into its address space using -the vm\_map rpc. It passes a reference to the port that the server gave +the `vm_map` RPC. It passes a reference to the port that the server gave it to the vm server (typically Mach). (D) Since Mach has never seen the object before, it queues a -memory\_object\_init on the given port along with a send right (the +`memory_object_init` on the given port along with a send right (the memory control port) for the manager to use to send messages to the kernel and also as an authentication mechanism for future interactions: the port is supplied so that the manager will be able to -identify from which kernel a given memory\_object\_* IPC is from. +identify from which kernel a given `memory_object_*` IPC is from. (E) The server dequeues the message, initializes internal data structures to manage the mapping and then invokes the -memory\_object\_ready method on the control object. +`memory_object_ready` method on the control object. (F) The kernel sees that the manager is ready, sets up the appropriate -mappings in the client and then replies to the vm\_map rpc indicating +mappings in the client's address space and then replies to the `vm_map` RPC indicating success. -There is nothing stopping others from playing "the kernel." This is +There is nothing stopping others from playing *the kernel*. This is not a security problem: clients must [[trust]] the server from whom they obtain memory objects and also the servers with whom they share the object. Multiple memory managers are a reality that should be @@ -95,37 +97,37 @@ mappings etc. # Resolving Page Faults -> (G) Client ________ -> resumed / \ -> | Mach | -> (A) Fault +----|------+ | \ (B) m_o_request (C) store_read -> ____|___ \_____|__/ |\ \| ________ _________ -> / +---\-------+ \ / \ / \ -> | Client | (F) | Server |<===>| storeio | -> \________/ m_o_supply \________/ \_________/ -> (E) return data | ^ -> | | (D) device_read -> v | -> ________ -> / Device \ -> | Driver | -> \________/ -> | ^ -> | | -> v -> ____________ -> / Hardware \ - -(A) The client does a memory access and faults. The kernel catches + (G) Client ________ + resumed / \ + | Mach | + (A) Fault +----|------+ | \ (B) m_o_request (C) store_read + ____|___ \_____|__/ |\ \| ________ _________ + / +---\-------+ \ / \ / \ + | Client | (F) | Server |<===>| storeio | + \________/ m_o_supply \________/ \_________/ + (E) return data | ^ + | | (D) device_read + v | + ________ + / Device \ + | Driver | + \________/ + | ^ + | | + v + ____________ + / Hardware \ + +(A) The client does a memory access and [[faults|page_fault]]. The kernel catches the fault and maps the address to the appropriate memory object. It -then invokes the memory\_object\_request method on the associated +then invokes the `memory_object_request` method on the associated capability. (In addition to the page to supply, it also supplies the control port so that the server can determine which kernel sent the message.) -(B) The manager dequeues the message. On the Hurd, this is translated -into a store\_read: a function in the libstore library which is used to -transparently manage block devices. The storeio server starts off as +(B) The manager dequeues the message. On the [[Hurd]], this is translated +into a `store_read`: a function in the [[hurd/libstore]] library which is used to +transparently manage block devices. The [[hurd/storeio]] server starts off as a separate process, however, if the server has the appropriate permission, the backing object can be contacted directly by the server. This layer of indirection is desirable when, for instance, a @@ -133,37 +135,37 @@ storeio running as root may want to only permit read only access to a resource, yet it cannot safely transfer its handle to the client. In this case, it would proxy the requests. -(C) The storeio server contacts, for instance, a device driver to do +(C) The storeio server contacts, for instance, a [[device_driver]] to do the read. This could also be a network block device (the NBD server in GNU/Linux), a file, a memory object, etc. -(D) The device driver allocates an anonymous page from the default -pager and reads the data into it. Once all of the operations are +(D) The device driver allocates an [[anonymous_page]] from the +[[default_pager]] and reads the data into it. Once all of the operations are complete, the device returns the data to the client unmapping it from its own address space at the same time. -(E) The storeio transfers the page to the server. The page is still +(E) The storeio server transfers the page to the server. The page is still anonymous. -(F) The manager does a memory\_object\_supply transferring the page to +(F) The manager does a `memory_object_supply` transferring the page to the kernel. Only now is the page not considered to be anonymous but managed. (G) The kernel caches the page, installs it in the client's virtual -address space and finally, resumes the client. +[[address_space]] and finally, resumes the client. # Paging Data Out -> Change manager Pager m_o_return store_write -> \ _________ (B) __(A)__ (C) ________ (D) _______ -> S | / Default \ / \ / \ / \ -> W |<=>| Pager |<=>| Mach |==>| server |<=>| storeio |<=> -> A | \_________/ \________/ \________/ \_______/ -> P | -> / + Change manager Pager m_o_return store_write + \ _________ (B) __(A)__ (C) ________ (D) _______ + S | / Default \ / \ / \ / \ + W |<=>| Pager |<=>| Mach |==>| server |<=>| storeio |<=> + A | \_________/ \________/ \________/ \_______/ + P | + / -(A) The paging [[policy]] is implemented by Mach: servers just implement +(A) The [[paging]] [[policy]] is implemented by Mach: servers just implement the [[mechanism]]. (B) Once the kernel has selected a page that it would like to evict, it @@ -172,10 +174,22 @@ if the server does not deallocate the page quickly enough, it cannot cause a denial of service: the kernel will just later double page it to swap (the default pager is part of the [[tcb]]). -(C) Mach then invokes memory\_object\_return method on the control -object. The server is expected to save the page free it in a timely +(C) Mach then invokes `memory_object_return` <!-- doesn't exist --> method on the control +object. The server is expected to save the page free <!-- ? --> it in a timely fashion. The server is not required to send a response to the kernel. -(D) The manager then transfers the data to the storeio which +(D) The manager then transfers the data to the storeio server which eventually sends it to disk. The device driver consumes the memory -doing the equivalent of a vm\_deallocate. +doing the equivalent of a `vm_deallocate`. + + +# Issues + + * [[open_issues/performance/io_system/read-ahead]] + + * [[open_issues/performance/io_system/clustered_page_faults]] + + +# GNU Hurd Usage + +Read about the [[Hurd's I/O path|hurd/io_path]]. diff --git a/microkernel/mach/gnumach.mdwn b/microkernel/mach/gnumach.mdwn index 71681eed..d9ff6535 100644 --- a/microkernel/mach/gnumach.mdwn +++ b/microkernel/mach/gnumach.mdwn @@ -1,16 +1,18 @@ -[[meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2001, 2002, 2007, 2008, 2011 Free Software +Foundation, Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] -GNU Mach is currently used by the GNU [[Hurd]]. +GNU Mach is the microkernel that the GNU/Hurd system is based on. -GNU Mach remains compatible with [[Mach]] 3.0. +It is maintained by the Hurd developers for the GNU project and remains +compatible with [[Mach]] 3.0. The majority of GNU Mach's [[device_driver]]s are from Linux 2.0. They were added using glue code, i.e., a Linux [[emulation]] layer in Mach. @@ -20,10 +22,60 @@ GNU Mach runs on x86 machines. See the [[ports]] to other architectures. +# Advantages of GNU Mach + +GNU Mach is not the most advanced [[microkernel]] known to the planet, nor is +it the fastest or smallest, but it has a rich set of [[interface]]s and some +features which make it useful as the base of the [[Hurd]] system. + + * **it's free software** + + Anybody can use, modify, and redistribute it under the terms of the + [[GNU_General_Public_License_(GPL)|gpl]]. + + * **it's built to survive** + + As a [[microkernel]], GNU Mach doesn't implement a lot of the features + commonly found in an operating system, but only the bare minimum that is + required to implement a full operating system on top of it. This means + that a lot of the operating system code is maintained outside of GNU Mach, + and while this code may go through a complete redesign, the code of the + microkernel can remain comparatively stable. + + * **it's scalable** + + Mach is particularly well suited for SMP and network cluster techniques. + Thread support is provided at the kernel level, and the kernel itself takes + advantage of that. Network transparency at the [[IPC]] level makes + resources of the system available across machine boundaries (with NORMA + IPC, currently not available in GNU Mach). + + * **it exists** + + The Mach microkernel is real software that works Right Now. It is not a + research or a proposal. You don't have to wait at all before you can start + using and developing it. Mach has been used in many operating systems in + the past, usually as the base for a single UNIX server. In the GNU system, + Mach is the base of a functional multi-server operating system, the + [[Hurd]]. + + +# Booting + +To actually use the kernel and boot the GNU operating system, you need a boot +loader. Not all boot loaders are capable to boot the GNU system, you need one +that supports the multiboot standard. The bootloader of the GNU system is +[[GNU_GRUB|grub]], which supports a broad range of operating systems including +GNU/Hurd. + + # Development -* [[Building]] -* [[Debugging]] -* [[Boot_Trace]] -* [[Projects]] - * [[Rules]] + * [[Reference_Manual]] + * [[Building]] + * [[Debugging]] + * [[Boot_Trace]] + * [[Memory_Management]] + * [[Projects]] + * [[Rules]] + * [[Open Issues|tag/open_issue_gnumach]] diff --git a/microkernel/mach/gnumach/boot_trace.mdwn b/microkernel/mach/gnumach/boot_trace.mdwn index a08384f0..1badf712 100644 --- a/microkernel/mach/gnumach/boot_trace.mdwn +++ b/microkernel/mach/gnumach/boot_trace.mdwn @@ -1,12 +1,13 @@ -[[meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2011 Free Software Foundation, +Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] `if NCPUS > 1` stuff is not being considered so far. @@ -215,6 +216,12 @@ is included in the section entitled >> kern/bootstrap.c: bootstrap\_create +>>> The [[grub/multiboot]] modules have been put somewhere into memory by +>>> [[GRUB]]. The boot scripts are parsed. The modules' ELF image's `PT_LOAD` +>>> sections are \`\`read'' (that is, `vm_allocate` and `copyout`) and turned +>>> into real [[task]]s. The multiboot modules' memory regions can be +>>> deallocated then. + >> [...] >> vm\_pageout diff --git a/microkernel/mach/gnumach/building.mdwn b/microkernel/mach/gnumach/building.mdwn index 27573b64..427fb083 100644 --- a/microkernel/mach/gnumach/building.mdwn +++ b/microkernel/mach/gnumach/building.mdwn @@ -1,9 +1,17 @@ -Additional to the following text, a further [[example]] has be posted. +[[!meta copyright="Copyright © 2006, 2007, 2008, 2011 Free Software Foundation, +Inc."]] +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] -# Building [[GNUMach]] from Source +# Building [[GNU_Mach|gnumach]] from Source -If you want to build the [[GNUMach]] kernel yourself instead of just using a +If you want to build the [[GNU_Mach|gnumach]] kernel yourself instead of just using a pre-built binary, follow these instructions. The unpacked source tree is around 20 MiB, and the build tree (with all drivers @@ -11,104 +19,101 @@ enabled) is around 50 MiB. ## Getting the Source Code -### Developers's RCS +You can either use the git repository (see <http://git.savannah.gnu.org/cgit/hurd/>), -See [here](http://www.gnu.org/software/hurd/gnumach-download.html#cvs). + $ git clone http://git.savannah.gnu.org/cgit/hurd/gnumach.git/ - $ cvs -z3 -d:pserver:anonymous@cvs.savannah.gnu.org:/sources/hurd co -r gnumach-1-branch gnumach - -(Most probably you want to get hold of the *GNU Mach 1 branch* and not the -trunk, which is also what we've done above.) - -You then have to create the automatically generatable files: - - $ ( cd gnumach && autoreconf --install ) - -### What Debian is currently using - -See [here](http://packages.debian.net/source/unstable/gnumach). +... or get the Debian sources, if you're using Debian. (See +[here](http://packages.debian.net/source/unstable/gnumach).) $ apt-get source gnumach Please see the Debian [[running/debian/FAQ]] before using `apt-get source`. -## Preparing for the Build +## On Debian Systems: -### ... on Debian systems +### Preparing for the Build -Building GNU Mach requires the *build-essential* and *fakeroot* packages, their -dependencies and additional packages that are specified by the source gnumach -package: +Building GNU Mach requires the *build-essential* and *fakeroot* packages, +and some additional dependencies specified by the gnumach source package: # apt-get install build-essential fakeroot # apt-get build-dep gnumach -### ... on non-Debian systems - -Apart from the case that you only want to install GNU Mach's header files (see -below), building GNU Mach requires you to have the Mach Interface Generator -installed. See [[building_MIG|mig/building]] about how to do that, then come -back here. +### Building and Installing ... Debian `.deb` files -Additionally, building GNU Mach requires a C compiler, a standard C library and -your favourite flavor of awk (gawk) and GNU make. +Change into the directory with the downloaded / unpacked GNU Mach sources, -## Building and Installing - -### ... Debian `.deb` files - -Change into the directory with the downloaded / unpacked GNU Mach sources, e.g. - - $ cd gnumach-20050801 + $ cd gnumach-XXXXXXXX Start the build process with $ dpkg-buildpackage -us -uc -b -rfakeroot -[[GNUMach]] is now building. To use the new kernel, you must install the +[[GNU_Mach|gnumach]] is now building. To use the new kernel, you must install the resulting `.deb` package which is located one directory above the build -directory and has a similar name as the build directory, e.g. +directory and has a similar name as the build directory: - # dpkg -i ../gnumach_20050801-4_hurd-i386.deb + # dpkg -i ../gnumach_XXXXXXXX-X_hurd-i386.deb You can now reboot your computer and enjoy the new kernel. -### [TODO] +## On non-Debian Systems: -GNU Mach should be built in a separate directory: +### Preparing for the Build - $ mkdir gnumach-build - $ cd gnumach-build +Building GNU Mach requires a C compiler, a _static_ 32 bit standard C library, +your favourite flavor of awk (gawk) and GNU make. -Find the path to your GNU Mach sources (`[...]/gnumach-1-branch`) and configure -it: +First, create the configuartion files: - $ [...]/gnumach-1-branch/configure [TODO] + $ cd gnumach + $ autoreconf --install -Build the kernel image: +GNU Mach (and the associated headers) need be built in a separate build directory: - $ make gnumach.gz + $ mkdir build + $ cd build -Optionally run the (tiny) test suite: +Run configure: - $ make check + $ ../configure --prefix= -You can then install and use `gnumach.gz`. +If building on a 64 bit host system, +you need a number of additional settings to force a 32 bit build: -[TODO.] + $ CPP='gcc -m32 -E -x c -undef -ansi' CC='gcc -m32' LD='ld -melf_i386' ../configure --prefix= --host=i686-unknown-linux-gnu -### Installing only the Header Files +### Installing the Header Files First -GNU Mach should be built in a separate directory: +In order to build GNU Mach, you will need a working MIG. +Building MIG in turn requires the GNU Mach header files to be already present. +So for bootstrapping MIG, you have to install the Mach headers first, +for example into `~/gnu/include/`: - $ mkdir gnumach-build - $ cd gnumach-build + $ make DESTDIR=~/gnu install-data -Find the path to your GNU Mach sources (`[...]/gnumach-1-branch`) and configure -it: +Now you can [[build_MIG|mig/gnu_mig/building]]. +Once you are done with that, come back here to finish the Mach build. - $ [...]/gnumach-1-branch/configure --prefix= +### Building and Installing -Install the header files into e.g. `~/gnu/include/`: +With MIG present, now build the kernel image: + + $ make gnumach.gz + +Optionally run the (tiny) test suite: + + $ make check + +It's a good idea to make a backup of the previously installed kernel, in case +you can't boot using the new one. That way, you can restore it after booting +from a rescue media (or mounting the disk image used by your vm). + + # cp /boot/gnumach.gz /boot/gnumach.gz.bak + +GNU Mach can now be moved into place, typically `/boot/gnumach.gz`, so that you +can boot your system with the new kernel. + + # cp gnumach.gz /boot - $ make DESTDIR=~/gnu install-data diff --git a/microkernel/mach/gnumach/building/example.mdwn b/microkernel/mach/gnumach/building/example.mdwn deleted file mode 100644 index 6da05c5b..00000000 --- a/microkernel/mach/gnumach/building/example.mdwn +++ /dev/null @@ -1,54 +0,0 @@ -[[meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] - -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] - -## Compiling GNU Mach microkernel - -Host development system is IBM T41 running Debian Sarge 3.1r0a GNU/Linux. - -* gcc version: 3.3.5 -* GNU sed version: 4.1.2 -* GNU make version: 3.8 -* mig version: 1.3-4 - -Obtained gnumach-1-branch sources from cvs: - - export CVS_RSH="ssh" - cvs -z3 -d:ext:anoncvs@ savannah.gnu.org:/cvsroot/hurd co -r gnumach-1-branch gnumach - -Obtained mig_1.3-4_i386.deb from -http://www.hadrons.org/~guillem/debian/pool/main/mig/. Installed it using dpkg: - - dpkg -i mig_1.3-4_i386.deb - -Entered into the gnumach sources and did the following for compilation: - - mkdir build - cd build - ../configure --host=i386-unknown-gnu0.2 --build=i586-pc-linux-gnu \ - --enable-kdb --enable-ide - make - -The kernel file is created in the build directory. Move it to /boot on the -testing x86 system Hurd partition. Rename it as gnumach1 and compress it: - - mv kernel gnumach1 - gzip gnumach1 - -Add a new entry on the testing machine /boot/grub/menu.lst to boot the new -kernel. - - title GNU Hurd K10 Compiled gnumach - kernel (hd0,3)/boot/gnumach1.gz root=device:hd2s4 -s - module (hd0,3)/hurd/ext2fs.static--multiboot-command-line=${kernel-command-line} \\ - --host-priv-port=${host-port} --device-master-port=${device-port} \\ - --exec-server-task=${exec-task} -T typed ${root} $(task-create)$(task-resume) - module (hd0,3)/lib/ld.so.1 /hurd/exec $(exec-task=task-create) - -Reboot into the new compiled mygnumach1.gz kernel! diff --git a/microkernel/mach/gnumach/debugging.mdwn b/microkernel/mach/gnumach/debugging.mdwn index fa2a9d42..f657e7cc 100644 --- a/microkernel/mach/gnumach/debugging.mdwn +++ b/microkernel/mach/gnumach/debugging.mdwn @@ -1,22 +1,78 @@ -[[meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2009, 2011 Free Software Foundation, +Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +Here are some hints to debug with GNU Mach. + +[[!toc levels=2]] + + +# Kernel Debugger Mach has a built-in kernel debugger. [Manual](http://www.gnu.org/software/hurd/gnumach-doc/Kernel-Debugger.html). +First, make sure to enable it. Either by using a pre-packaged gnumach-image-something-dbg, or by passing --enable-kdb to the ./configure invocation. + +Then, reproduce the issue again. If something like a kernel trap happens, you will end up in the GNU Mach debugger. Otherwise, type control-alt-d to make Mach enter it by hand. + +If you are running in kvm or qemu, it is convenient to use the curses frontend to be able to copy/paste. + +To get the register values, type + + show registers + +To get a backtrace, type trace, which will print both function return addresses and function parameters, such as + + 0x107cf1(8088488,5e,40000008,2aa008,0) + 0x1071bc(0,0,0,0,0) + 0x106831(24fe00,2000,b,800,0) + +Run the addr2line tool on the return addresses: + + $ addr2line -i -f -e /boot/gnumach 0x107cf1 0x1071bc 0x106831 + +This will print the source code lines of the backtrace. + +To examine the backtrace of some given thread, use + + show all thread/u + +to get the whole listing of all tasks and threads. You can then use trace/t to trace a specific thread. + +Unfortunately, userland and kernelland use the same range of addresses, so one can not get userland traces easily. The Xen port uses different ranges, and in that case one can use trace/u to also get the userland trace. + +To examine a variable, use nm /boot/gnumach to get the address of the variable (e.g. 0x123400), and use + + x 0x123400 + +to read it. One can also write to it by using + + w 0x123400 + +Another interesting feature is watching a variable, by using + + watch 0x123400 + +and then type continue, to let Mach continue execution. The debugger will be entered again on any change in that variable. The watch is implemented in hardware, so it does not disturb or slow down execution at all. + + +# GDB in QEMU When you're [[running_a_system_in_QEMU|hurd/running/qemu]] you can directly [use GDB on the running -kernel](http://fabrice.bellard.free.fr/qemu/qemu-doc.html#SEC36). +kernel](http://www.nongnu.org/qemu/qemu-doc.html#SEC48). +# Code Inside the Kernel + Alternatively you can use an approach like this one: add the following code snippet to `device/ds_routines.c`'s `ds_device_open` function, right at the top of the function, and modify the code as needed. @@ -55,6 +111,8 @@ This is especially useful if you need to manually trigger some stuff inside the running kernel, as with the *D1* example. +## Writing to the Screen Buffer + If you're doing real low level debugging, you might want to put variations of the following snipped into the code, this code will write a `#` character at line `[LINE]`, column `[COLUMN]` on the screen: @@ -66,3 +124,14 @@ The call of `halt_cpu` will -- as the name suggests -- halt the system afterwards. This might be what you want or it might not, but it is needed at some place when running the kernel inside QEMU, as QEMU somehow decides not to update its display buffer anymore under certain conditions. + + +# Halting the CPU and Examining Registers + +IRC, freenode, #hurd, 2011-07-14: + + <braunr> one ugly trick i use when printf isn't available is to halt the + cpu + <braunr> then use info registers to know where the cpu is halted + <braunr> and you'll know if you reached that code or not + <braunr> (info registers is a qemu command) diff --git a/microkernel/mach/gnumach/hardware_compatibility_list.mdwn b/microkernel/mach/gnumach/hardware_compatibility_list.mdwn index 09882467..874f5f07 100644 --- a/microkernel/mach/gnumach/hardware_compatibility_list.mdwn +++ b/microkernel/mach/gnumach/hardware_compatibility_list.mdwn @@ -1,12 +1,13 @@ -[[meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2009, 2011 Free Software Foundation, +Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] # CPU Architecture @@ -28,7 +29,7 @@ Read about further [[ports]]. # Memory -GNU Mach will use a maximum of 1 GiB of RAM. If your system has more, +GNU Mach will use a maximum of 1.7 GiB of RAM. If your system has more, the surplus will silently be ignored. (In past times, this would hinder GNU Mach from booting at all, but this has been fixed, so you no longer need to apply GRUB's `uppermem` directive.) @@ -64,14 +65,14 @@ IEEE 1394 is not supported at this time # Storage All common IDE drives should work. Some drive geometries do not work, -e.g. drives with hundreds of GiB of storage space. If you find a specific IDE -drive that does not work, make a note of the model and technical specifications -here. +e.g. drives with hundreds of GiB of storage space, see [[!GNU_Savannah_bug +26425]]. -[[toggle id="SATA" text="SATA drives may work in compatibility mode."]] -<!-- Sure? --[[tschwinge]] --> -[[toggleable id="SATA" text=""" +## SATA + +SATA drives may work in compatibility mode. + This is how booting a [[GNU/Hurd_system|hurd]] will typically fail if GNU Mach couldn't connect to the hard disk, e.g., in a SATA system without IDE compatibility mode: @@ -81,7 +82,7 @@ compatibility mode: There *may* be an option in the system's BIOS setup to configure enabling such a compatibility mode. -"""]] + # Device Drivers diff --git a/microkernel/mach/gnumach/hardware_compatibility_list/discussion.mdwn b/microkernel/mach/gnumach/hardware_compatibility_list/discussion.mdwn index 69ca3190..2b65956a 100644 --- a/microkernel/mach/gnumach/hardware_compatibility_list/discussion.mdwn +++ b/microkernel/mach/gnumach/hardware_compatibility_list/discussion.mdwn @@ -1,4 +1,33 @@ +[[!meta copyright="Copyright © 2007, 2008, 2011 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_documentation]] + Further information may still be found on <http://www.nongnu.org/thug/gnumach_hardware.html> and could perhaps be incorporated into that page. --[[tschwinge]] + + +# SATA + +IRC, freenode, +hurd, 2011-07-24 + + <braunr> youpi: concerning the ide compatibility problem, it seems some + bioses provide several modes + <braunr> youpi: "legacy ide" and "native ide" + <braunr> i don't know what native ide really means, but when debugging ide + probing in gnumach, it just looks like there is nothing to detect + <braunr> and even in this mode, linux uses the ahci driver + <youpi> apparently native means it still uses the IDE protocol, but + possibly with other IRQs + <youpi> i.e. you need a PCI driver to handle that + <braunr> ok diff --git a/microkernel/mach/gnumach/hardwarecompatibilitylist.mdwn b/microkernel/mach/gnumach/hardwarecompatibilitylist.mdwn deleted file mode 100644 index 8c67e3e0..00000000 --- a/microkernel/mach/gnumach/hardwarecompatibilitylist.mdwn +++ /dev/null @@ -1,11 +0,0 @@ -[[meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] - -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] - -[[meta redir=hardware_compatibility_list]] diff --git a/microkernel/mach/gnumach/memory_management.mdwn b/microkernel/mach/gnumach/memory_management.mdwn new file mode 100644 index 00000000..43b99d83 --- /dev/null +++ b/microkernel/mach/gnumach/memory_management.mdwn @@ -0,0 +1,82 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_documentation]] + +IRC, freenode, #hurd, 2011-02-15 + + <braunr> etenil: originally, mach had its own virtual space (the kernel + space) + <braunr> etenil: in order to use linux 2.0 drivers, it now directly maps + physical memory, as linux does + <braunr> etenil: but there is nothing similar to kmap() or vmalloc() in + mach, so the kernel is limited to its 1 GiB + <braunr> (3 GiB userspace / 1 GiB kernelspace) + <braunr> that's the short version, there is a vmalloc() in mach, but this + trick made it behave almost like a kmalloc() + <antrik> braunr: the direct mapping is *only* for the benefit of Linux + drivers?... + <braunr> also, the configuration of segments limits the kernel space + <braunr> antrik: i'm not sure, as i said, this is the short version + <braunr> antrik: but there is a paper which describes the integration of + those drivers in mach + <etenil> you mean the linux 2.0 drivers? + <antrik> braunr: I read it once, but I don't remember anything about the + physical mapping in there... + <antrik> etenil: well, originally it was 1.3, but essentially that's the + same... + <braunr> i don't see any other reason why there would be a direct mapping + <braunr> except for performance (because you can use larger - even very + lage - pages without resetting the mmu often thanks to global pages, but + that didn't exist at the time) + +IRC, freenode, #hurd, 2011-02-15 + + <antrik> however, the kernel won't work in 64 bit mode without some changes + to physical memory management + <braunr> and mmu management + <braunr> (but maybe that's what you meant by physical memory) + +IRC, freenode, #hurd, 2011-02-16 + + <braunr> antrik: youpi added it for xen, yes + <braunr> antrik: but you're right, since mach uses a direct mapped kernel + space, the true problem is the lack of linux-like highmem support + <braunr> which isn't required if the kernel space is really virtual + + +--- + +IRC, freenode, #hurd, 2011-06-09 + + <braunr> btw, how can gnumach use 1 GiB of RAM ? did you lower the + user/kernel boundary address ? + <youpi> I did + <braunr> 2G ? + <youpi> yes + <braunr> ok + <youpi> it doesn't make so much sense to let processes have 3G addressing + space when there can't be more that 1G physical memory + <braunr> that's sad for an operating system which does most things by + mapping memory eh + <youpi> well, if a process wants to map crazy things, 3G may be tight + already + <youpi> e.g. ext2fs + <braunr> yes + <youpi> so there's little point in supporting them + <braunr> we need hurd/amd64 + <youpi> and there's quite some benefit in shrinking them to 2G + <youpi> yes + <youpi> actually even 2G may become a bit tight + <youpi> webkit linking needs about 1.5-2GiB + <youpi> things become really crazy + <braunr> wow + <braunr> i remember the linux support for 4G/4G split when there was enough + RAM to fill the kernel space with struct page entries diff --git a/microkernel/mach/gnumach/ports.mdwn b/microkernel/mach/gnumach/ports.mdwn index 00cdee8c..f114460c 100644 --- a/microkernel/mach/gnumach/ports.mdwn +++ b/microkernel/mach/gnumach/ports.mdwn @@ -1,15 +1,24 @@ -[[meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2009, 2011 Free Software Foundation, +Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] * x86. This is the main port. + + * [[Xen]] + * [PowerPC](http://www.pjbruin.dds.nl/hurd/). Is not in a usable state. - * Alpha. Was once started, but isn't in a usable state either. - * [[Xen]] + * Alpha: [project I](http://savannah.nongnu.org/projects/hurd-alpha), and + [project II](http://savannah.nongnu.org/projects/gnumach-alpha). Was once + started, but isn't in a usable state either. + + * MIPS. Status completely unknown. + + * [[open_issues/Mach_on_Top_of_POSIX]]. Status unknown. diff --git a/microkernel/mach/gnumach/ports/xen.mdwn b/microkernel/mach/gnumach/ports/xen.mdwn index 1c1cc40f..5fe73c06 100644 --- a/microkernel/mach/gnumach/ports/xen.mdwn +++ b/microkernel/mach/gnumach/ports/xen.mdwn @@ -1,29 +1,73 @@ -[[meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2009, 2011 Free Software Foundation, +Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] -/!\ Since GNU Mach doesn't handle PAE yet, you'll need a PAE-disabled hypervisor. +[[!toc]] -/!\ You need an already installed GNU/Hurd system. -This also means that you'll currently need a PAE-disabled `dom0`. -[[Stefan_Siegl|stesie]] is providing a PAE-disabled Linux kernel image at -<http://brokenpipe.de/GnuHurd/XEN/>. +# Xen dom0, hypervisor +/!\ Now that GNU Mach handles PAE you can use a PAE-enabled hypervisor. You can either get binaries at <http://youpibouh.thefreecat.org/hurd-xen/> or build them yourself. -- Copy `gnumach-xen` and `hurd-modules` to your dom0 /boot. +- Copy `gnumach-xen-pae` and `hurd-modules` to your dom0 /boot. If you still have a non-PAE hypervisor, use `gnumach-xen-nonpae` instead. - Copy `hurd` into `/etc/xen`, edit it for fixing access to your hurd / and swap -- Run `xm create -c hurd`, gnumach should get started. + +# GNU/Hurd system + +/!\ You need an already installed [[GNU/Hurd_system|hurd/running]]. + +If you have a free partition, you can fdisk to type 0x83, create a filesystem using: + + sudo mke2fs -b 4096 -I 128 -o hurd /dev/sda4 + +Replace /dev/sda4 with your partition. Install and use crosshurd to setup a GNU/Hurd system on this partition. + + +# /etc/xen/hurd configuration + +Here is a sample /etc/xen/hurd configuration + + kernel = "/boot/gnumach-xen-pae" + memory = 256 + disk = ['phy:sda4,hda,w'] + extra = "root=device:hd0" + vif = [ '' ] + ramdisk = "/boot/hurd-modules" + +Do not give more than 580MB memory (due to bootstrap limitations, it's not easy +to map more). + +Suggestions about [[networking_configuration]] are available. + +If you need stable MAC addresses, use a syntax like `vif = [ +'mac=00:16:3e:XX:XX:XX, bridge=br0' ]`. + + +# Running Hurd with Xen + +To run Hurd with Xen, use: + + xm create -c hurd + +and gnumach should get started. Proceed with native-install. + + export TERM=mach + ./native-install + - If `xm` complains about networking (`vif could not be connected`), it's Xen scripts' fault, see Xen documentation for how to configure the network. The simplest way is network-bridge with fixed IPs (note that you need the bridge-utils package for this). You can also just disable networking by commenting the vif line in the config. -- If `xm` complains `Error: (2, 'Invalid kernel', 'xc_dom_compat_check: guest type xen-3.0-x86_32 not supported by xen kernel, sorry\n')`, you most probably have a PAE-enabled hypervisor, and you just need to install and boot non-PAE hypervisor and kernel. +- If `xm` complains `Error: (2, 'Invalid kernel', 'xc_dom_compat_check: guest type xen-3.0-x86_32 not supported by xen kernel, sorry\n')`, you most probably have a PAE-enabled hypervisor and a non-PAE gnumach. Either install and boot non-PAE hypervisor and kernel, or rebuilt gnumach in PAE mode. + + +# Building from sources If you want to generate these images, first get the `gnumach-1-branch-Xen-branch` branch from gnumach CVS. Then look for "Ugly" in `kern/bootstrap.c`, how to generate `hurd-modules` is explained there, and you'll have to fix `EXT2FS_SIZE` and `LD_SO_SIZE` by hand. @@ -33,10 +77,51 @@ Then use make The current `hurd-modules` was built from the debian packages `hurd 20070606-2` and `libc0.3 2.6.1-1`. -/!\ This means that when using this image, your GNU/Hurd system also needs to be a glibc version 2.6-based one! +/!\ This means that when using this image, your GNU/Hurd system also needs to be a glibc version 2.6 or later-based one! + +# `pv-grub` + +From Xen 4.0 on you can run the GNU Hurd directly using `pv-grub`, +without the need to [prepare a special bootstrap +image](http://youpibouh.thefreecat.org/hurd-xen/build_hurd-modules) (like an +initrd). + +Download http://youpibouh.thefreecat.org/hurd-xen/pv-grub.gz into /boot, and use the following for instance: + + kernel = "/boot/pv-grub.gz" + memory = 256 + disk = ['phy:sda4,hda,w'] + extra = "(hd0,1)/boot/grub/menu.lst" + vif = [ '' ] + +extra is now the path to the grub config file. + +# Partitions + +You will need the following notation for the gnumach root= parameter: ---- +root=part:2:device:hd0 + +to access the second partition of hd0, for instance. + +You will also need to use the parted storeio module for the /dev entries, for instance: + +settrans -fgap /dev/hd0s1 /hurd/storeio -T typed part:1:device:hd0 + +# Miscellaneous [[Internals]]. -[[GNU_Savannah_task 5468]], [[GNU_Savannah_task 6584]]. +[[!GNU_Savannah_task 5468]], [[!GNU_Savannah_task 6584]]. + + +# Host-side Writeback Caching + +Optimization possible as it is with [[QEMU|hurd/running/qemu/discussion]], +*Host-side Writeback Caching*? + +IRC, freenode, #hurd, 2011-06-08 + + <braunr> youpi: does xen provide disk caching options ? + <youpi> through a blktap, probably + <braunr> ok diff --git a/microkernel/mach/gnumach/ports/xen/discussion.mdwn b/microkernel/mach/gnumach/ports/xen/discussion.mdwn new file mode 100644 index 00000000..2980e3b2 --- /dev/null +++ b/microkernel/mach/gnumach/ports/xen/discussion.mdwn @@ -0,0 +1,14 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_documentation]] + +Stuff from <http://youpibouh.thefreecat.org/hurd-xen> should be merged into +these pages here. diff --git a/microkernel/mach/gnumach/ports/xen/internals.mdwn b/microkernel/mach/gnumach/ports/xen/internals.mdwn index 22aac75e..eae9d9a8 100644 --- a/microkernel/mach/gnumach/ports/xen/internals.mdwn +++ b/microkernel/mach/gnumach/ports/xen/internals.mdwn @@ -1,12 +1,14 @@ -[[meta copyright="Copyright © 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008 Free Software Foundation, Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +[[GNU Free Documentation License|/fdl]]."]]"""]] The port does use Xen's para-virtualized interface for device (ide, network, etc.) access. + +[[Virtualization]]. diff --git a/microkernel/mach/gnumach/ports/xen/networking_configuration.mdwn b/microkernel/mach/gnumach/ports/xen/networking_configuration.mdwn new file mode 100644 index 00000000..52e6db87 --- /dev/null +++ b/microkernel/mach/gnumach/ports/xen/networking_configuration.mdwn @@ -0,0 +1,105 @@ +[[!meta copyright="Copyright © 2008 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled +[[GNU Free Documentation License|/fdl]]."]]"""]] + +[[!toc]] + +The Xen dom0 infrastructure provides for a bridged networking setup using shell +scripts to configure the bridging device properly and attach the domUs' virtual +interfaces to the bridge. However, we've [seen +problems](http://lists.gnu.org/archive/html/bug-hurd/2008-08/msg00023.html) +when using this approach, so to [solve these +issues](http://lists.gnu.org/archive/html/bug-hurd/2008-09/msg00071.html), +instead suggest the following configuration method (to achieve the same thing). + +This is for a Debian dom0. + +# */etc/network/interfaces* + +Comment out everything referencing your physical devices. Add this: + + auto br0 + iface br0 inet dhcp + bridge_ports regex (eth|vif).* noregex + +... or if you want to do the manual configuration dance: + + auto br0 + iface br0 inet static + bridge_ports regex (eth|vif).* noregex + address 192.168.10.60 + netmask 255.255.255.0 + [...] + +This needs a version of the `bridge-utils` package more recent than the current +Debian stable one ([[!debbug 405215]]). (It's trivial to rebuild the `dpkg` of, +e.g., the Debian testing one on Debian stable.) + +# */etc/xen/xend-config.sxp* + +Make sure that only `(network-script network-dummy)` and `(vif-script +vif-bridge)` are activated and all other `(network-script network-WHATEVER)`, +respective `(vif-script vif-WHATEVER)` are commented out. + + +# Sample configuration files on Debian Lenny + +## /etc/xen/hurd on dom0 + + kernel = "/boot/gnumach-xen" + memory = 256 + disk = ['phy:sda5,hda,w'] + extra = "root=device:hd0" + vif = [ 'mac=00:16:3e:00:00:00, bridge=br0' ] + ramdisk = "/boot/hurd-modules" + +/dev/sda5 is an extended partition. br0 is bridge interface on dom0. + +## /etc/xen/xend-config.sxp on dom0 + + (network-script 'network-bridge netdev=br0') + (dom0-min-mem 196) + (dom0-cpus 0) + (vncpasswd '') + +## /etc/network/interfaces on dom0 + + auto br0 + iface br0 inet static + address 192.168.1.211 + network 192.168.1.0 + netmask 255.255.255.0 + broadcast 192.168.1.255 + gateway 192.168.1.1 + bridge_ports eth1 + +eth1 is the interface that is connected to the Internet on the LAN: + +## Doing settrans on domU + + settrans -fgap /servers/socket/2 /hurd/pfinet -i eth0 -a 192.168.1.210 -g 192.168.1.1 -m 255.255.255.0 + +## /sbin/ifconfig on dom0 + + br0 Link encap:Ethernet HWaddr 00:19:d1:2e:06:33 + inet addr:192.168.1.211 Bcast:192.168.1.255 Mask:255.255.255.0 + inet6 addr: fe80::219:d1ff:fe2e:633/64 Scope:Link + UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 + RX packets:14187 errors:0 dropped:0 overruns:0 frame:0 + TX packets:9214 errors:0 dropped:0 overruns:0 carrier:0 + collisions:0 txqueuelen:0 + RX bytes:936563 (914.6 KiB) TX bytes:746184 (728.6 KiB) + + eth1 Link encap:Ethernet HWaddr 00:19:d1:2e:06:33 + inet6 addr: fe80::219:d1ff:fe2e:633/64 Scope:Link + UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 + RX packets:34339 errors:0 dropped:0 overruns:0 frame:0 + TX packets:18526 errors:0 dropped:0 overruns:0 carrier:0 + collisions:0 txqueuelen:1000 + RX bytes:3019251 (2.8 MiB) TX bytes:1453672 (1.3 MiB) diff --git a/microkernel/mach/gnumach/projects.mdwn b/microkernel/mach/gnumach/projects.mdwn index 35b5c2ef..f4ef192a 100644 --- a/microkernel/mach/gnumach/projects.mdwn +++ b/microkernel/mach/gnumach/projects.mdwn @@ -1,13 +1,13 @@ -[[meta copyright="Copyright © 2005, 2006, 2007, 2008 - Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2005, 2006, 2007, 2008, 2011 Free Software +Foundation, Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] This page is a place to keep track of ideas about things that may be improved in GNU Mach, so that it'll evolve to a reliable microkernel for The Hurd, both @@ -15,7 +15,7 @@ in terms of stability and performance. If you find anything missing here, please feel free to add a entry with a short description. If you want to help with any of the task (thanks!), please send a mail to -*[[mailing_lists/bug-hurd]]* stating what task you wish to work on, +*[[mailing lists/bug-hurd]]* stating what task you wish to work on, so that no duplicate efforts end up. # Active Branches @@ -33,6 +33,8 @@ so that no duplicate efforts end up. * [[Clean_up_the_code]] + * [[Open Issues|tag/open_issue_gnumach]] + * Update the core architecture and drivers * Check what NetBSD, FreeBSD and Linux do with their host specific code @@ -56,7 +58,8 @@ so that no duplicate efforts end up. * Improve the external pagers interface - * Implement read-ahead (huge I/O improvements expected). + * Implement [[open_issues/performance/io_system/read-ahead]] (huge I/O + improvements expected). * Making this interface synchronous should improve I/O performance significantly, without (almost) any drawbacks (we also get some @@ -99,7 +102,7 @@ so that no duplicate efforts end up. See [[GDB_stubs]]. - * Make it run as a UNIX/Linux executable. + * Make it run as a [[UNIX]]/Linux executable. * Neal: diff --git a/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn b/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn index 875bb8cd..2a9b4b60 100644 --- a/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn +++ b/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn @@ -1,13 +1,15 @@ -[[meta copyright="Copyright © 2005, 2006, 2007, 2008 - Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2005, 2006, 2007, 2008, 2010 Free Software +Foundation, Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +[[GNU Free Documentation License|/fdl]]."]]"""]] + +[[!tag open_issue_gnumach]] # Restructure the tree in a sane way @@ -39,80 +41,80 @@ So, additionally to the list given below, there may actually be a bunch of further files (also exported ones) that serve no real value, but are being `#include`d through one way or another. -* [[source_gnumach-1-branch ddb/db_expr.h]] +* [[!source_gnumach-1-branch ddb/db_expr.h]] Currently used, but copyright violation? Rewrite? -* [[source_gnumach-1-branch ddb/db_print.h]] +* [[!source_gnumach-1-branch ddb/db_print.h]] Copyright violation? Currently unused, but could be used in principle (or be rewritten, to avoid the copyright oddity). -* [[source_gnumach-1-branch ddb/tr.h]] +* [[!source_gnumach-1-branch ddb/tr.h]] Copyright violation. Unused. Remove. -* [[source_gnumach-1-branch device/dev_master.h]] +* [[!source_gnumach-1-branch device/dev_master.h]] Might be usable for SMP? Remove otherwise. -* [[source_gnumach-1-branch i386/i386/kttd_machdep.h]] +* [[!source_gnumach-1-branch i386/i386/kttd_machdep.h]] -* [[source_gnumach-1-branch i386/i386/sched_param.h]] +* [[!source_gnumach-1-branch i386/i386/sched_param.h]] -* [[source_gnumach-1-branch i386/include/mach/i386/cthreads.h]] +* [[!source_gnumach-1-branch i386/include/mach/i386/cthreads.h]] Was probably once exported, but is no longer. -* [[source_gnumach-1-branch i386/include/mach/i386/ioccom.h]] +* [[!source_gnumach-1-branch i386/include/mach/i386/ioccom.h]] Exported. -* [[source_gnumach-1-branch include/device/audio_status.h]] +* [[!source_gnumach-1-branch include/device/audio_status.h]] Exported. -* [[source_gnumach-1-branch include/device/tape_status.h]] +* [[!source_gnumach-1-branch include/device/tape_status.h]] Exported. -* [[source_gnumach-1-branch include/mach/alert.h]] +* [[!source_gnumach-1-branch include/mach/alert.h]] Exported. -* [[source_gnumach-1-branch include/mach/boot.h]] +* [[!source_gnumach-1-branch include/mach/boot.h]] Exported. -* [[source_gnumach-1-branch include/mach/macro_help.h]] +* [[!source_gnumach-1-branch include/mach/macro_help.h]] Exported. -* [[source_gnumach-1-branch include/mach/multiboot.h]] +* [[!source_gnumach-1-branch include/mach/multiboot.h]] Exported. -* [[source_gnumach-1-branch include/mach/profil.h]] +* [[!source_gnumach-1-branch include/mach/profil.h]] Exported. -* [[source_gnumach-1-branch include/mach/profilparam.h]] +* [[!source_gnumach-1-branch include/mach/profilparam.h]] Exported. -* [[source_gnumach-1-branch include/mach/exec/a.out.h]] +* [[!source_gnumach-1-branch include/mach/exec/a.out.h]] Exported. -* [[source_gnumach-1-branch include/mach_debug/pc_info.h]] +* [[!source_gnumach-1-branch include/mach_debug/pc_info.h]] Currently not exported, but was probably once meant to be. -* [[source_gnumach-1-branch kern/act.h]] +* [[!source_gnumach-1-branch kern/act.h]] -* [[source_gnumach-1-branch kern/refcount.h]] +* [[!source_gnumach-1-branch kern/refcount.h]] -* [[source_gnumach-1-branch kern/shuttle.h]] +* [[!source_gnumach-1-branch kern/shuttle.h]] # Remove dead functions, variables, etc. from source files diff --git a/microkernel/mach/gnumach/projects/gdb_stubs.mdwn b/microkernel/mach/gnumach/projects/gdb_stubs.mdwn index 9a11a82b..064da7bf 100644 --- a/microkernel/mach/gnumach/projects/gdb_stubs.mdwn +++ b/microkernel/mach/gnumach/projects/gdb_stubs.mdwn @@ -1,13 +1,19 @@ -[[meta copyright="Copyright © 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2010 Free Software Foundation, Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +[[GNU Free Documentation License|/fdl]]."]]"""]] + +[[!tag open_issue_gnumach open_issue_gdb]] * <http://lists.gnu.org/archive/html/bug-hurd/2008-04/msg00103.html> * [ChangeLog.gdb](http://cvs.savannah.gnu.org/viewvc/gnumach/ChangeLog.gdb?root=hurd&view=markup&pathrev=gnumach-1-branch-gdb-branch) + +This may be another follow-up project: [*Linux Kernel GDB tracepoint +module*](http://thread.gmane.org/gmane.comp.gdb.devel/29369), Hui Zhu, +2010-10-09. diff --git a/microkernel/mach/gnumach/reference_manual.mdwn b/microkernel/mach/gnumach/reference_manual.mdwn new file mode 100644 index 00000000..95d11517 --- /dev/null +++ b/microkernel/mach/gnumach/reference_manual.mdwn @@ -0,0 +1,26 @@ +[[!meta copyright="Copyright © 2001, 2002, 2003, 2004, 2005, 2006, 2007, 2008 +Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled +[[GNU Free Documentation License|/fdl]]."]]"""]] + +*The GNU Mach Reference Manual* documents the architecture, the usage and the +programming of the GNU Mach microkernel. At the moment, the manual documents +the interface completely, but is not very useful as a tutorial or introduction +into the Mach architecture. + + * [HTML version](http://www.gnu.org/software/hurd/gnumach-doc/index.html) + for browsing online, + * [PostScript + version](http://www.gnu.org/software/hurd/gnumach-doc/mach.ps) [around + 900KiB], + * [gzipped PostScript + version](http://www.gnu.org/software/hurd/gnumach-doc/mach.ps.gz) + [around 300KiB], + * [PDF version](http://www.gnu.org/software/hurd/gnumach-doc/mach.pdf) + [around 700KiB]. diff --git a/microkernel/mach/history.mdwn b/microkernel/mach/history.mdwn index a27595c0..5a3608cd 100644 --- a/microkernel/mach/history.mdwn +++ b/microkernel/mach/history.mdwn @@ -1,14 +1,14 @@ -# <a name="Table_of_Contents"> Table of Contents </a> - -%TOC% - # <a name="Early_beginnings"> Early beginnings </a> Mach has quite a history. Everything actually started at the University of Rochester in 1975. It was invented to demonstrate how operating systems could be built using a modular design where processes communicated using message passing, even across networks. The system was called the Rochester Intelligent Gateway and ran on a 16 bit mini computer called Eclipse from Data General. One of the engineers behind the project was Richard Rashid. In 1979 he moved his socks to Carnegie Mellon University to continue his research on message passing operating systems. The result emerged in 1981 and was called Accent. -Accent kept running at CMU until 1984 but was by then being run over by UNIX. Rashid then decided to use an early embrace strategy and started designing the third generation OS project called Mach. By making Mach compatible with UNIX, Mach was supposed to gain a lot of available UNIX software. +Accent kept running at CMU until 1984 but was by then being run over by +[[UNIX]]. Rashid then decided to use an early embrace strategy and started +designing the third generation OS project called Mach. By making Mach +compatible with UNIX, Mach was supposed to gain a lot of available UNIX +software. Mach was a vast improvement over Accent. It incorporated parts commonly used today, such as: threads, better IPC, multiprocessor support and an advanced VM system. diff --git a/microkernel/mach/ipc.mdwn b/microkernel/mach/ipc.mdwn index 889fac2f..1bb44b59 100644 --- a/microkernel/mach/ipc.mdwn +++ b/microkernel/mach/ipc.mdwn @@ -1,22 +1,21 @@ -[[meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2010 Free Software Foundation, +Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] -[[General_information|/ipc]] about IPC. +Read about the [[general concept of *inter-process communication* (IPC)|/ipc]]. -An IPC is sent by invoking a [[port]]. <!-- Isn't this wording a bit strange? -``IPC is sent'' --tschwinge --> +On Mach, an IPC is done by invoking a [[port]]. + +The two fundamental operations, to *send* and *receive* [[message]]s, are used +to implement a [[RPC]] system. [[Sequence_numbering]]. [The Unofficial GNU Mach IPC beginner's guide](http://www.nongnu.org/hurdextras/ipc_guide/ipc_guide.html) - -# See Also - -* [[RPC]] diff --git a/microkernel/mach/ipc/sequence_numbering.mdwn b/microkernel/mach/ipc/sequence_numbering.mdwn index 7c1f68e4..eb94d662 100644 --- a/microkernel/mach/ipc/sequence_numbering.mdwn +++ b/microkernel/mach/ipc/sequence_numbering.mdwn @@ -1,12 +1,12 @@ -[[meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +[[GNU Free Documentation License|/fdl]]."]]"""]] Mach's [[IPC]] mechanism allows for getting access to a message's sequence number. diff --git a/microkernel/mach/logo.png b/microkernel/mach/logo.png Binary files differdeleted file mode 100644 index 94951acf..00000000 --- a/microkernel/mach/logo.png +++ /dev/null diff --git a/microkernel/mach/memory_object.mdwn b/microkernel/mach/memory_object.mdwn new file mode 100644 index 00000000..f32fe778 --- /dev/null +++ b/microkernel/mach/memory_object.mdwn @@ -0,0 +1,33 @@ +[[!meta copyright="Copyright © 2002, 2003, 2010, 2011 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +Mach's [[virtual_memory]] subsystem uses *memory objects* for supplying the +content of regions of virtual memory in an [[virtual_address_space]]. + +All of these objects are managed by *memory manager*s, that are also called +*pager*s. These can be implemented as user-space processes. + +Both the memory objects, and their managers are kernel objects, and are +accessed by [[port]]s. + +A system's physical memory is conceived as a *memory cache* that contains +*memory cache objects*. So when a [[thread]] accesses a page in its task's +address space, the memory object that includes this page is *cached* in the +memory cache. Memory objects are [[paged out and paged +in|external_pager_mechanism]] by the aforementioned memory managers. The +decision when they should be paged in or paged out is left to [[Mach]]. Each +memory object has an ordered list of memory managers that provide paging. The +last one tried is the *default memory manager* that resides in the microkernel, +in contrast to most of the others. The default memory manager is needed +because the microkernel can't wait infinitely for someone else to free the +memory cache: it just calls the next memory manager hoping it to succeed. + +Read about [[GNU Mach's memory management|gnumach/memory_management]]. diff --git a/microkernel/mach/memory_object/discussion.mdwn b/microkernel/mach/memory_object/discussion.mdwn new file mode 100644 index 00000000..a2a1514b --- /dev/null +++ b/microkernel/mach/memory_object/discussion.mdwn @@ -0,0 +1,67 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_documentation open_issue_gnumach]] + +IRC, freenode, #hurd, 2011-08-05: + + < neal> braunr: For instance, memory objects are great as they allow you to + specify the mapping policy in user space. + < neal> braunr: But, the policy for determining the eviction order is + realized by Mach + < neal> braunr: And user-space has no control + < braunr> are you referring to the page cache lru approximation and stuff + like resource containers ? + < neal> I'm not sure what you mean by page cache lru appoximateion + < braunr> the kernel eviction policy :) + < neal> that's an implementation detail + +IRC, freenode, #hurd, 2011-09-05: + + <braunr> mach isn't a true modern microkernel, it handles a lot of + resources, such as high level virtual memory and cpu time + <braunr> for example, the page replacement mechanism can't be implemented + outside the kernel + <braunr> yet, it provides nothing to userspace server to easily allocate + resources on behalf of clients + <braunr> so, when a thread calls an RPC, the cpu time used to run that RPC + is accounted on the server task + <braunr> the hurd uses lots of external memory managers + +[[external_pager_mechanism]]. + + <braunr> but they can't decide how to interact with the page cache + <braunr> the kernel handles the page cache, and initiates the requests to + the pagers + <cjuner> braunr, why can't they decide that? + <braunr> because it's implemented in the kernel + <braunr> and there is nothing provided by mach to do that some other way + <slpz_> braunr: you probably already know this, but the problem with client + requests being accounted on behalf the server, is fixed in Mach with + Migrating Threads + +[[open_issues/mach_migrating_threads]]. + + <braunr> slpz_: migrating threads only fix the issue for the resources + managed by mach, not the external servers + <braunr> slpz_: but it's a (imo necessary) step to completely solve the + issue + <braunr> in addition to being a great feature for performance (lighter + context switchers, less state to track) + <braunr> it also helps priority inversion problems + <slpz_> braunr: I was referring just to cpu-time, but I agree with you an + interface change is needed for external pagers + <braunr> slpz_: servers in general, not necessarily pagers + <slpz_> as a way to mitigate the effect of Mach paging out to external + pagers, the folks at OSF implemented an "advisory pageout", so servers + are "warned" that they should start paging out, and can decide which + pages are going to be flushed by themselves + +[[open_issues/resource_management_problems]]. diff --git a/microkernel/mach/message.mdwn b/microkernel/mach/message.mdwn new file mode 100644 index 00000000..ba47671e --- /dev/null +++ b/microkernel/mach/message.mdwn @@ -0,0 +1,31 @@ +[[!meta copyright="Copyright © 2002, 2003, 2010 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +*Messages* are collections of typed data, with a defined layout. + +They are used for [[IPC]], and are sent to and received from [[port]]s. + +These messages are not only opaque data. They can also contain [[port +rights|port]] to be passed to another [[task]]. Port rights are either +*copied* or *moved*. Notice that port receive right must be moved but not +copied because there can't be more than one task that holds the receive right +to a port. The receiving task creates new local port name to the port rights +it received. + +Some data in the message can be *out-of-line data*. In the message, these are +*references* to memory regions ([[memory_object]]s) that are *virtually +copied*. When the message is received in a task, these virtual copies become +part of the task by mapping them into the receiver's [[virtual_address_space]]. +Another key concept that is applied is using *copy-on-write*, which means that +data is not copied immediately, but only when it is changed. This is primarily +used to send large blocks of data efficiently, as it is too expensive to store +them in the kernel address space: extra copied need only be made at the moment +that the memory regions begin to diverge, by threads modifying them. diff --git a/microkernel/mach/mig.mdwn b/microkernel/mach/mig.mdwn index f47eadfb..331b3bf4 100644 --- a/microkernel/mach/mig.mdwn +++ b/microkernel/mach/mig.mdwn @@ -1,12 +1,35 @@ -[[img logo.png]] +[[!meta copyright="Copyright © 2001, 2002, 2003, 2006, 2007, 2008, 2010 Free +Software Foundation, Inc."]] -The Mach Interface Generator (MIG) is an [[IDL]] compiler. Based on an -interface definition, it creates stubs to [[invoke]] object methods -and to demultiplex incoming messages. These stubs conveniently hide -the details of Mach's [[IPC]] machinery. +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] -GNU MIG is fully compatible with OSF MIG. +The *Mach Interface Generator* (*MIG*) is an [[IDL]] compiler. Based on an +interface definition, it creates stub code to [[invoke]] object methods and to +demultiplex incoming messages. These stub functions conveniently hide the +details of Mach's [[IPC]] and [[port]] machinery and make it easy to implement +and use Mach [[interface]]s as [[remote procedure calls (RPC)|rpc]]: by using +the stub functions, the client programs can call remote procedures more or less +like any other C function. -* MIG's [homepage](http://www.gnu.org/software/hurd/mig.html) -* [[Documentation]] -* [[Building]] - Building (and obtaining) MIG +These functions encode arguments into [[message]]s' format (*marshalling*), +wait for a result on a newly created [[reply port|port]], decode return +arguments from the reply message (*demarshalling*, or *unmarshalling*) and pass +them to the client program. Similar actions are provided in the skeletons that +are linked to server programs. + +MIG allows very precise semantics to be specified about what the arguments are +and how to be passed. + + + * [[Documentation]] + + +# Implementations + + * [[GNU_MIG]] diff --git a/microkernel/mach/mig/building.mdwn b/microkernel/mach/mig/building.mdwn deleted file mode 100644 index ee299166..00000000 --- a/microkernel/mach/mig/building.mdwn +++ /dev/null @@ -1,71 +0,0 @@ -# <a name="Building_the_Mach_Interface_Gene"> Building the Mach Interface Generator from Source </a> - -If you want to build the Mach Interface Generator yourself instead of just using a pre-built package, follow these instructions. - -## <a name="Getting_the_Source_Code"> Getting the Source Code </a> - -You can chose between getting the [sources from the developers's rcs](http://www.gnu.org/software/hurd/mig-download.html#cvs): - - $ cvs -z3 -d:pserver:anonymous@cvs.savannah.gnu.org:/sources/hurd co mig - -... or (if you are working on a Debian system) the ones that are used for the [current Debian mig package](http://packages.debian.net/source/unstable/mig): - - $ apt-get source mig - -Please see the Debian [[running/debian/FAQ]] before using _apt-get source_. - -The unpacked source tree is around 1 MiB, and the build tree also is around 1 MiB. - -## <a name="Preparing_for_the_Build"> Preparing for the Build </a> - -### <a name="_on_Debian_systems"> ... on Debian systems </a> - -Building the Mach Interface Generator requires the _build-essential_ and _fakeroot_ packages, their dependencies and additional packages that are specified by the source mig package: - - # apt-get install build-essential fakeroot - # apt-get build-dep mig - -### <a name="_on_non_Debian_systems"> ... on non-Debian systems </a> - -Building the Mach Interface Generator requires a C compiler, a standard C library (with corresponding header files) and your favourite flavor of awk (gawk), yacc (bison), lex (flex) and make. - -Additionally, you need to have GNU Mach's header files installed. See [[mach/gnumach/building]] about how to do that, then come back here. - -## <a name="Building_and_Installing"> Building and Installing </a> - -### <a name="_a_deb_file"> ... a _.deb_ file </a> - -Change into the directory with the downloaded / unpacked MIG sources (_mig-1.3.1.99_): - - $ cd mig-1.3.1.99 - -Start the build process: - - $ dpkg-buildpackage -us -uc -b -rfakeroot - -You can then install / distribute the _.deb_ file which will drop out one directory above the current one. - -### <a name="_TODO_"> [TODO] </a> - -The Mach Interface Generator has to be built in a separate directory: - - $ mkdir mig-build - $ cd mig-build - -Find the root directory where you installed GNU Mach's header files and where you now intend to install the Mach Interface Generator (_~/gnu_) and the path to your Mach Interface Generator sources (\_[...]/mig) and configure it: - - $ GNU=~/gnu - $ TARGET_CPPFLAGS=-I"$GNU"/include [...]/mig/configure --prefix="$GNU" - -Build and install the Mach Interface Generator into _$GNU_, i.e. _~/gnu/_ in our example: - - $ make all install - -To make your _mig_ binary easily available, you should append something like the following to e.g. your _~/.bash\_profile_: - - PATH=~/gnu/bin:$PATH - export PATH - -If you already have e.g. _~/bin_ in your _$PATH_, you could also create a symbolic link: - - $ ln -s ~/gnu/bin/mig ~/bin/ diff --git a/microkernel/mach/mig/dealloc.mdwn b/microkernel/mach/mig/dealloc.mdwn deleted file mode 100644 index 1ed9d52b..00000000 --- a/microkernel/mach/mig/dealloc.mdwn +++ /dev/null @@ -1,11 +0,0 @@ -[[meta copyright="Copyright © 2008 Free Software Foundation, Inc."]] - -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] - -[[meta redir=documentation#dealloc]] diff --git a/microkernel/mach/mig/discussion.mdwn b/microkernel/mach/mig/discussion.mdwn deleted file mode 100644 index fdab3a45..00000000 --- a/microkernel/mach/mig/discussion.mdwn +++ /dev/null @@ -1,17 +0,0 @@ -Created - --- [[Main/JoachimNilsson]] - 29 Oct 2002 - -The logo seems very programmer friendly as this web topic is intended. - --- [[Main/GrantBow]] - 15 Nov 2002 - -There's little traffic here and little content. Perhaps we should just remove this web? It seemed like a good idea to create it when we split the others off... - --- [[Main/GrantBow]] - 22 Dec 2002 - -Maybe, but not yet. Let's keep it for a while longer - say, three months. 1st April 2003. If the traffic still is low then we move the Mig topics to the Mach web ... - -...Mig = **Mach** Interface Generator. - --- [[Main/JoachimNilsson]] - 22 Dec 2002 diff --git a/microkernel/mach/mig/documentation.mdwn b/microkernel/mach/mig/documentation.mdwn index 8c977e55..7d4f1eca 100644 --- a/microkernel/mach/mig/documentation.mdwn +++ b/microkernel/mach/mig/documentation.mdwn @@ -1,13 +1,13 @@ -[[meta copyright="Copyright © 2002, 2003, 2005, 2007, 2008 Free Software -Foundation, Inc."]] +[[!meta copyright="Copyright © 2002, 2003, 2005, 2007, 2008, 2009, 2010 Free +Software Foundation, Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] This is a small collection of links to external documents describing the *Mach Interface Generator* used by GNU Mach. @@ -17,23 +17,23 @@ Interface Generator* used by GNU Mach. A tutorial which demonstrates the use of the C Threads library primitives in writing a multithreaded program and the use of the Mach Interface Generator -(MIG) to generate remote procedure calls for interprocess communication. Like +(MIG) to generate remote procedure calls for inter-process communication. Like its companion tutorial, it is based on the Mach 2.5 system. However, the concepts are applicable to Mach 3.0 user level programming. Linda R. Walmer and Mary R. Thompson. *A Programmer's Guide to the Mach User Environment*. [PostScript -](ftp://ftp.cs.cmu.edu/afs/cs/project/mach/public/doc/unpublished/machuse.ps), -[Doc](ftp://ftp.cs.cmu.edu/afs/cs/project/mach/public/doc/unpublished/machuse.doc). +](http://www.cs.cmu.edu/afs/cs/project/mach/public/doc/unpublished/machuse.ps), +[Doc](http://www.cs.cmu.edu/afs/cs/project/mach/public/doc/unpublished/machuse.doc). February 1988. School of Computer Science, Carnegie Mellon University. An ftp directory containing the [mig programming -examples](ftp://ftp.cs.cmu.edu/afs/cs/project/mach/public/doc/unpublished/mig_example) +examples](http://www.cs.cmu.edu/afs/cs/project/mach/public/doc/unpublished/mig_example) for this tutorial. Slides to Rich Drave's talk on MIG, on November 21, 1991: -[PostScript](ftp://ftp.cs.cmu.edu//afs/cs/project/mach/public/doc/unpublished/internals_slides/Mig/root.ps), -[TeX](ftp://ftp.cs.cmu.edu//afs/cs/project/mach/public/doc/unpublished/internals_slides/Mig/slides.tex). +[PostScript](http://www.cs.cmu.edu//afs/cs/project/mach/public/doc/unpublished/internals_slides/Mig/root.ps), +[TeX](http://www.cs.cmu.edu//afs/cs/project/mach/public/doc/unpublished/internals_slides/Mig/slides.tex). # Roots @@ -41,15 +41,15 @@ Slides to Rich Drave's talk on MIG, on November 21, 1991: Mig is an implementation of a subset of the Matchmaker **language**. "Matchmaker is a language for specifying and automating the generation of -multilingual interprocess communication interfaces. MIG is an interim +multilingual inter-process communication interfaces. MIG is an interim implementation of a subset of the Matchmaker language that generates C and C++ -remote procedure call interfaces for interprocess communication between Mach +remote procedure call interfaces for inter-process communication between Mach tasks." Richard P. Draves, Michael B. Jones, Mary R. Thompson, *MIG - THE MACH INTERFACE GENERATOR*. -[ps](ftp://ftp.cs.cmu.edu/afs/cs/project/mach/public/doc/unpublished/mig.ps), -[doc](ftp://ftp.cs.cmu.edu/afs/cs/project/mach/public/doc/unpublished/mig.doc). +[ps](http://www.cs.cmu.edu/afs/cs/project/mach/public/doc/unpublished/mig.ps), +[doc](http://www.cs.cmu.edu/afs/cs/project/mach/public/doc/unpublished/mig.doc). November 1989. Department of Computer Science, Carnegie-Mellon University. @@ -66,12 +66,19 @@ pp. 67--77." # Further Relevant Documentation - * The [GNU Mach Reference - Manual](http://www.gnu.org/software/hurd/docs.html#manuals), espacially + * The [[GNU_Mach_Reference_Manual|gnumach/reference_manual]], espacially [Chapter 4, Inter Process - Communication](http://www.gnu.org/software/hurd/gnumach-doc/Inter-Process-Communication.html), - which, for example, explains how the <a name="dealloc">`dealloc` flag</a> - works in [Chapter 4.2.4, - Memory](http://www.gnu.org/software/hurd/gnumach-doc/Memory.html) + Communication](http://www.gnu.org/software/hurd/gnumach-doc/Inter-Process-Communication.html). + + * OSF's [Server Writer's Guide (ps)](http://www.cs.cmu.edu/afs/cs/project/mach/public/doc/osf/server_writer.ps) + [Server Writer's Guide (pdf)](http://shakthimaan.com/downloads/hurd/server_writer.pdf) + + * OSF's [Server Writer's Interfaces (ps)](http://www.cs.cmu.edu/afs/cs/project/mach/public/doc/osf/server_interface.ps) + [Server Writer's Interfaces (pdf)](http://shakthimaan.com/downloads/hurd/server_interface.pdf) + + * Flags: + + * [[dealloc_and_dealloc[]|dealloc]] + * [[ServerCopy]] * MIG *in action*: [[hurd/io_path]]. diff --git a/microkernel/mach/mig/documentation/dealloc.mdwn b/microkernel/mach/mig/documentation/dealloc.mdwn new file mode 100644 index 00000000..b627b532 --- /dev/null +++ b/microkernel/mach/mig/documentation/dealloc.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled +[[GNU Free Documentation License|/fdl]]."]]"""]] + +The [[GNU_Mach_Reference_Manual|gnumach/reference_manual]] describes the +`dealloc` flag in [Chapter 4.2.4, +Memory](http://www.gnu.org/software/hurd/gnumach-doc/Memory.html). + +What exactly is `dealloc[]` (`hurd/fs.defs:dir_readdir`)? diff --git a/microkernel/mach/mig/documentation/servercopy.mdwn b/microkernel/mach/mig/documentation/servercopy.mdwn new file mode 100644 index 00000000..8abf9b07 --- /dev/null +++ b/microkernel/mach/mig/documentation/servercopy.mdwn @@ -0,0 +1,23 @@ +[[!meta copyright="Copyright © 2008 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled +[[GNU Free Documentation License|/fdl]]."]]"""]] + +For IN args. If set it... + + * removes the `__mig_deallocate` for OOL IN data, which is usually done after + the implementation has been called; + + * adds a `boolean_t NAMESCopy` for the IN arg `NAME` to indicate whether the + data will persist nevertheless (OOL case) or has to be copied by the + implementation (inline case). + +Cf., [[translator/exec]] server, `exec.defs`. + +I.e., the IN args' memory region (OOL case) persists after the implementation +has returned. diff --git a/microkernel/mach/mig/gnu_mig.mdwn b/microkernel/mach/mig/gnu_mig.mdwn new file mode 100644 index 00000000..0de1bd67 --- /dev/null +++ b/microkernel/mach/mig/gnu_mig.mdwn @@ -0,0 +1,28 @@ +[[!meta copyright="Copyright © 2001, 2006, 2008, 2009, 2010 Free Software +Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +GNU MIG is the GNU distribution of the +[[Mach_3.0_interface_generator_*MIG*|mig]], as maintained by the GNU Hurd +developers for the GNU project. + +You need this tool to compile the GNU Mach and GNU Hurd distributions, and to +compile the GNU C library for the Hurd. Also, you will need it for other +software in the GNU system that uses Mach-based +[[inter-process_communication|ipc]]. + +GNU MIG is fully compatible with [[OSF_MIG|mig]]. + +Like its predecessor, it can only generate C code, that has to be compiled and +linked to client and server programs respectively ([[!taglink +open_issue_mig]]). + + * [[Building]] - building (and obtaining) GNU MIG + * [[Open Issues|tag/open_issue_mig]] diff --git a/microkernel/mach/mig/gnu_mig/building.mdwn b/microkernel/mach/mig/gnu_mig/building.mdwn new file mode 100644 index 00000000..e7d3c150 --- /dev/null +++ b/microkernel/mach/mig/gnu_mig/building.mdwn @@ -0,0 +1,103 @@ +[[!meta copyright="Copyright © 2006, 2007, 2008, 2011 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +# <a name="Building_the_Mach_Interface_Gene"> Building the Mach Interface Generator from Source </a> + +If you want to build the Mach Interface Generator yourself instead of just +using a pre-built package, follow these instructions. + +## <a name="Getting_the_Source_Code"> Getting the Source Code </a> + +You can chose between getting the [sources from the developers' +RCS](http://git.savannah.gnu.org/cgit/hurd/): + + $ git clone http://git.savannah.gnu.org/cgit/hurd/mig.git/ + +... or (if you are working on a Debian system) get the sources that are used for the +[current Debian mig package](http://packages.debian.net/source/unstable/mig): + + $ apt-get source mig + +Please see the Debian [[hurd/running/debian/FAQ]] before using _apt-get source_. + +The unpacked source tree is around 1 MiB, and the build tree also is around 1 MiB. + +## <a name="_on_Debian_systems"> On Debian Systems: </a> + +### <a name="Preparing_for_the_Build"> Preparing for the Build </a> + +Building MIG requires the *build-essential* and *fakeroot* packages, +and some additional dependencies specified by the mig source package: + + # apt-get install build-essential fakeroot + # apt-get build-dep mig + +### <a name="Building_and_Installing"> Building and Installing </a> <a name="_a_deb_file"> ... a _.deb_ file </a> + +Change into the directory with the downloaded / unpacked MIG sources: + + $ cd mig-X.X.X.XX + +Start the build process: + + $ dpkg-buildpackage -us -uc -b -rfakeroot + +This will create a _.deb_ package in the parent directory, +which you can then install on your system. + +## <a name="_on_non_Debian_systems"> On non-Debian Systems: </a> + +### <a name="Preparing_for_the_Build"> Preparing for the Build </a> + +Building the Mach Interface Generator requires a C compiler, a standard 32 bit +C library (with corresponding header files), your favourite flavor of awk +(gawk), yacc (bison), lex (flex) and make. + +Additionally, you need to have GNU Mach's header files installed. See +[[building GNU Mach|mach/gnumach/building]] about how to do that, then come back here. + +### <a name="Building_and_Installing"> Building and Installing </a> + +First, generate the configuration files: + + $ cd mig + $ autoreconf --install + +The Mach Interface Generator has to be built in a separate build directory: + + $ mkdir build + $ cd build + +Find the base directory where you installed GNU Mach's header files and where +you now intend to install the Mach Interface Generator (e.g. _~/gnu_), and run +configure: + + $ GNU=~/gnu + $ TARGET_CPPFLAGS=-I"$GNU"/include ../configure --prefix="$GNU" + +If you are building on a 64 bit machine, you need to add a --host option: + + $ GNU=~/gnu + $ TARGET_CPPFLAGS=-I"$GNU"/include ../configure --prefix="$GNU" --host=i686-unknown-linux-gnu + +Build and install the Mach Interface Generator into _$GNU_ (i.e. _~/gnu/_ in our example): + + $ make all install + +To make your _mig_ binary easily available, you should append something like +the following to e.g. your _~/.bash\_profile_: + + PATH=~/gnu/bin:$PATH + export PATH + +If you already have e.g. _~/bin_ in your _$PATH_, you could also create a symbolic link: + + $ ln -s ~/gnu/bin/mig ~/bin/ diff --git a/microkernel/mach/mig/gnu_mig/building/discussion.mdwn b/microkernel/mach/mig/gnu_mig/building/discussion.mdwn new file mode 100644 index 00000000..d7636158 --- /dev/null +++ b/microkernel/mach/mig/gnu_mig/building/discussion.mdwn @@ -0,0 +1,16 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +# Non-cross-compiling + +[[!tag open_issue_mig]] + +[[samuelthibault]] mentioned that I should make clear what compiler options, etc. are only needed if compiling on a 64 bit computer. However, I don't know if the --host=i686... option is needed, here and when making gnumach, in case there may be some other default on 32 bit computers? --[[sudoman]] + diff --git a/microkernel/mach/mig/logo.png b/microkernel/mach/mig/logo.png Binary files differdeleted file mode 100644 index cdfec179..00000000 --- a/microkernel/mach/mig/logo.png +++ /dev/null diff --git a/microkernel/mach/port.mdwn b/microkernel/mach/port.mdwn index 77b1372f..26b55456 100644 --- a/microkernel/mach/port.mdwn +++ b/microkernel/mach/port.mdwn @@ -1,41 +1,89 @@ -[[meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2002, 2003, 2007, 2008, 2010, 2011 Free Software +Foundation, Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] - -Mach ports are [[capabilities|capability]]. - -A Mach port is a kernel queue. Each port has associated with -it a receive right and one or more send and send-once rights. -A queue can hold a number of messages. Once the queue is full, -the send blocks until their is space to enqueue the message -(this is interruptible via a timeout mechanism). - -A receive right designates a queue and authorizes the holder to -dequeue messages from the queue, and to create send and send-once -rights. - -Send and send-once rights designate a queue and authorize the -hold to enqueue messages (in the case of a send-once right, -a single message). Enqueuing a message is equivalent to -[[invoke|invoking]] a capability. - -Send and receive rights are named using local names. Each -task has associated with it a port [[address_space]]. A ports -are addressed via this table. Each task thus has its own -private [[naming_context]] for ports. - -Ports can be [[delegate]]d in an [[IPC]] message. When the -receiver dequeues the message, the right is made available -to it. - -A [[thread]] can only block receiving on a single port. To work -around this, the concept of a port set was introduced. A receive -right can be added to (at most) one port set. When a thread -receives from a port set, it dequeues from any of the ports that -has a message available. +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[Mach]] *port*s are [[capabilities|capability]], and are also essentially +similar to [[UNIX]] pipes. They are unforgeable communication channels, +implemented by kernel queues. + +Each port has associated with it one *receive right* and one or more *send +right*s and *send-once right*s. That is, there is one receiver and one or more +senders -- a unidirectional communication channel. Only with the corresponding +port right, access to a port is possible; this is enforced by Mach. + +The kernel queue can hold a number of [[message]]s. Once the queue is full, +the send blocks until there is space to enqueue the message (this is +interruptible via a timeout mechanism). + +A receive right [[designates|designation]] a queue and authorizes the holder to +dequeue messages from the queue, and to create send and send-once rights. + +Send and send-once rights designate a queue and authorize the hold to enqueue +messages (in the case of a send-once right, a single message). Enqueuing a +message is equivalent to [[invoke|invoking]] a capability. + +Ports are automatically destroyed when there is no associated port right to +them. + +Mach knows what port rights belong to each task, but [[thread]]s that running +in the context of a task refer to ports by means of send and receive rights +that are named using local *port names*. These port names are plain integers, +like [[UNIX file descriptors|unix/file_descriptor]]. Only these local names +can be used by [[thread]]s for invoking operations on ports, threads do not +deal with port rights directly. + +For that, each task has associated with it a *port address space*, or *port +name space*. All ports are addressed via this table. Each task thus has its +own private [[naming_context]] for port rights. + +So, the picture is that after obtaining a port send right, the client uses a +port name to send [[message]]s to the port, or exactly one message if it's a +send-once right. These messages are (probably) queued and when the server task +tries to receive messages by having a [[thread]] use its port receive right, it +gets the message(s). This is called [[IPC]]. + +Port rights themselves can be [[delegate]]d in a [[message]], too. When the +receiver dequeues the message, the right is made available to it. + +The delivery of [[message]]s is reliable and strictly ordered. When a +[[thread]] sends messages *1* and *2*, it is guaranteed that the receiving +[[task]] will catch them in the same order. Of course, there can be +intermediate messages that are sent by other threads. + +Ports are objects that are implemented by the [[kernel]], and they are +kernel-protected resources: they are unforgeable, and there is no way for a +[[task]] to do anything with a port unless it have corresponding port right. + +Due to this, ports are globally unique. This makes them ideal for constituting +system-wide *object references*. (Fruther reading: +{{$capability#wikipedia_object-capability_model}}.) For example, the [[RPC]] +system as used by the GNU Hurd works by invoking *methods* on such object +references. The available methods are defined in [[hurd/interface]] files, and +are processes by the [[MIG]] tool. + +Invoking an operation on a port does not transfer the current execution control +to the receiver, but instead is an asynchronous operation. For this, and +especially in a [[RPC]] system, the sender may include a *reply port* using a +send-once right, and synchronize (block) on that one. + + +# Port Set + +A [[thread]] can only block receiving on a single port. To work around this, +the concept of a *port set* was introduced. A receive right can be added to +(at most) one port set. These port sets look like port receive rights, but +cannot be passed to other tasks, and there are additional operations for adding +and removing port receive rights. + +When a server process' thread receives from a port set, it dequeues exactly one +message from any of the ports that has a message available in its queue. + +This concept of port sets is also the facility that makes convenient +implementation of [[UNIX]]'s `select` [[system_call]] possible. diff --git a/microkernel/mach/rpc.mdwn b/microkernel/mach/rpc.mdwn index ecef86ee..422e0441 100644 --- a/microkernel/mach/rpc.mdwn +++ b/microkernel/mach/rpc.mdwn @@ -1,15 +1,28 @@ -[[meta copyright="Copyright © 2007, 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2002, 2003, 2007, 2008, 2010, 2011 Free Software +Foundation, Inc."]] -[[meta license="""[[toggle id="license" text="GFDL 1.2+"]][[toggleable +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU_Free_Documentation_License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] -[[General_information|/rpc]] about RPC. +Read about the [[general concept of a *remote procedure call* (RPC)|/rpc]]. Uses Mach's [[IPC]] [[mechanism]]. -Stub code generated by [[MIG]]. +The [[port]] abstraction allows RPCs to be executed on another computer +transparently. This can be implemented with user [[task]]s, but there is an +implementation in the kernel possible, too, which is called *NORMA*, but is not +avilable in [[GNU Mach|gnumach]]. + +The RPC stub code generated by [[MIG]]. + + +# Tracing + + * [[hurd/debugging/rpctrace]] + + * [[open_issues/librpci]] diff --git a/microkernel/mach/rpc/discussion.mdwn b/microkernel/mach/rpc/discussion.mdwn new file mode 100644 index 00000000..00e4a012 --- /dev/null +++ b/microkernel/mach/rpc/discussion.mdwn @@ -0,0 +1,117 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_documentation]] + + +# IRC, freenode, #hurd, 2011-06-11 + + <antrik> I don't think we have a precendence case of Mach initiating RPCs + to userspace tasks + <braunr> well mach regularly sends RPCs to external pagers + <antrik> hm, right + <antrik> anyways, the ds_ in device.defs is for use *inside* Mach, not for + the userspace interface + <braunr> what makes you think so ? + <antrik> several things + <antrik> not least the fact that without zhengda's modifications, the + device handling never calls out to userspace for all I know + <braunr> hm, it does + <braunr> for async I/O + <braunr> when the kernel has finished its I/O, it calls + ds_device_read_reply/ds_device_write_reply + <antrik> I see + <antrik> I never quite understood the _reply stuff + <braunr> although i wonder how mig is supposed to forge those names + <antrik> braunr: it isn't + <antrik> braunr: there is a separate device_reply.defs + <antrik> braunr: and it sets a *userprefix* of ds_ + <antrik> rather than a serverprefix + <braunr> i saw, yes + <braunr> ah right + <antrik> so ds still refers to the in-Mach device server, not anything + userspace + <braunr> so this is where the patch is supposed to introduce the + device_intr_notify RPC + <antrik> or at least that's my understanding... + <braunr> no, it doesn't refer to in-mach servers + <braunr> it really forges the right rpcs to be called by mach + <antrik> the definition of "RPC" is rather unclear here + <braunr> why ? + <braunr> mach has its own mach_msg() call for kernel-to-user messaging + <antrik> yes, but this is used only to send the reply message for the RPC + earlier initiated by userspace AIUI + <antrik> it doesn't look like there is any special RPC for async I/O + <braunr> yes, because this is the only use case they had + <braunr> hence the name "reply" + <braunr> intr_notify isn't a reply, but it uses the same mechanism + <braunr> these are declared as simpleroutine + <antrik> sure. but the fact that it isn't a reply message, but rather + initiates a new RPC, changes things from MiG point of view I believe + <antrik> right, as there is no reply to the reply :-) + <braunr> :) + <braunr> a simpleroutine is how to turn an rpc into a simple ipc + <antrik> I know + <antrik> so in _reply, we pretend that the reply is actually a new RPC, + with server and client roles reversed, and no reply + <antrik> (this is actually rather kludgy... apparently MIG has no real + notion of async replies) + <braunr> i don't understand what you mean + <braunr> simpleroutine is the explicit solution for async replies + <braunr> as stated in + http://www.cs.cmu.edu/afs/cs/project/mach/public/doc/unpublished/mig.ps + <braunr> it's not a new rpc with roles reversed + <braunr> it's not a reply either + <antrik> it might be an explicit solution for that, but it still seems + kludgy :-) + <braunr> i don't see why :/ + <braunr> would you have expected something like an option to create both + sync and async versions ? + <antrik> because it requires an extra .defs file + <antrik> yes + <braunr> ok + <braunr> well this seems cumbersome to me :) + <braunr> i prefer the simpleroutine approach + <braunr> but i agree this seems odd since mach has a high level ipc api + <antrik> anyways, my point is that the ds_ in device_reply.defs still + refers to the Mach side of things + <braunr> npnth: which package fails to build ? + <antrik> though a userspace process that actually handles the replies in an + async fashion will of course need some kind of device server too, just + like the DDE stuff... + <antrik> though naming it ds_ is confusing IMHO, because of the name clash + with the device server in Mach + <braunr> hm again, i fail to see why + <braunr> ds_ just means device_server + <braunr> and as most things in mach, it can be in kernel or not + <braunr> i mean, this is an interface prefix, i don't refer to an actual + single instance of a "device server" out there + <antrik> oh, right... DDE implements the Mach device protocol, so it *does* + do the ds_ part... but that makes the interrupt notification stuff even + more confusing + <braunr> hm + <braunr> because it provides a ds_device_intr_notify() which will never be + used, just to completely implement the interface ? + <antrik> yeah, that's what I suspect... + <braunr> sounds likely + <antrik> the device interface actually has two parts: one for "generic" + RPCs on the master device port, and one for device-specific RPCs. DDE + implements the latter, and uses the former... + <antrik> they live in separate places though I think: the individual device + RPCs are implemented in libmachdev, while the intr_ stuff is used in + libddekit probably + <braunr> it would be hairy to build otherwise + <antrik> so we *really* need to know what component npnth gets the error + with + <antrik> braunr: nah, not really. that's why we always have a separate + prefix for the server routines in Hurd RPCs + <braunr> right, i really need to read about mig again + <antrik> it's pretty normal for a translator to both implement and use an + interface diff --git a/microkernel/mach/task.mdwn b/microkernel/mach/task.mdwn new file mode 100644 index 00000000..c03c6a14 --- /dev/null +++ b/microkernel/mach/task.mdwn @@ -0,0 +1,23 @@ +[[!meta copyright="Copyright © 2002, 2003, 2010 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +A Mach *task* is a collection of resources, a [[virtual_address_space]], and a +[[port name space|port]]. They depend on [[thread]]s for executing program +code: a task alone has no means to do so. + +Switching from one task to another one involves doing a *context switch*, which +is usually not a cheap operation, as it involves switching the hardware's idea +of the memory layout ([[virtual_address_space]]), amongst others. + +Mach tasks are distinct from [[UNIX processes|unix/process]] in that they +provide less facilities. In processes, there are [[unix/signal]]s, process / +group / session IDs, [[unix/file_descriptor]]s and many other things. Tasks +are used for resource allocation and sharing; they are *resource container*s. diff --git a/microkernel/mach/thread.mdwn b/microkernel/mach/thread.mdwn new file mode 100644 index 00000000..e27bb117 --- /dev/null +++ b/microkernel/mach/thread.mdwn @@ -0,0 +1,37 @@ +[[!meta copyright="Copyright © 2002, 2003, 2010 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +A Mach *thread* belongs to exactly one [[task]], and is the means of execution. +The task supplies the resources. + +Mach threads are implemented inside the [[kernel]], as opposed to other +systems' user-level thread packages. + +A thread (theoretically) runs concurrently with all the other threads of a +system. If the system provides several processors, they can be used for +simultaneously running either several threads of the same task, or several +threads of different tasks. <!-- This is called SMP; the processors use +*shared memory*. --> [[!tag open_issue_documentation]] <!-- This needs a new +page, also covering Mach's `processor_set`s, and non-SMP, but still +multiprocessor systems. --> (But this is currently not support in [[GNU +Mach|gnumach]].) + +It is easy for the kernel to switch execution from one thread to another one +inside the same task: essentially, it only involves exchanging a few processor +registers' state. + +Threads have scheduling parameters and maintain various statistics about +themselves. + +On GNU/Hurd, APIs for Mach threads and thereabouts are provided by the +[[hurd/libthreads]] (cthreads), and [[libpthread]] (POSIX Threads) packages. + +A task backing a thread is the basis for a [[UNIX process|unix/process]]. diff --git a/microkernel/mach/virtual_address_space.mdwn b/microkernel/mach/virtual_address_space.mdwn new file mode 100644 index 00000000..97bc5f6b --- /dev/null +++ b/microkernel/mach/virtual_address_space.mdwn @@ -0,0 +1,36 @@ +[[!meta copyright="Copyright © 2002, 2003, 2010 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +*Virtual address space*s in Mach define the valid virtual addresses that can be +used by [[thread]]s under execution in the [[task]] that owns that address +space. Each task has only one address space and each address space belongs to +only one task. So when we want to name an address space (for example, in the +Mach API) we name it by the task it belongs to. + +These address spaces are divided into *pages*. Each page has individual +properties like *access rights* (*read* / *write* / *execute*), *inheritance +attributes* (*no inheritance* / *copy* / *share*) and some other system +properties. Page manipulation is optimized to help moving large blocks of data +from one address space to another, for example when one thread provides data to +another thread -- *client / server* technology. + +Memory ranges of pages that can be controlled as a whole are called +*[[memory_object]]*s. + +*Wired pages* are those that cannot be [[paged out|external_pager_mechanism]]. +For example, Mach itself is a task with its own address space and threads, and +all of its pages are wired. + +*Precious pages* are those that must not be discarded silently when they are +clean and memory is needed. For example, a memory manager that shares memory +across a network could not restore a page if it is silently discarded because +it is unmodified. This is not valid for the well-known [[pager +managers|external_pager_mechanism]] that use disks as backing store. |
