summaryrefslogtreecommitdiff
path: root/microkernel
diff options
context:
space:
mode:
Diffstat (limited to 'microkernel')
-rw-r--r--microkernel/genode.mdwn17
-rw-r--r--microkernel/genode/rpc.mdwn65
-rw-r--r--microkernel/l4.mdwn10
-rw-r--r--microkernel/mach/deficiencies.mdwn262
-rw-r--r--microkernel/mach/external_pager_mechanism.mdwn14
-rw-r--r--microkernel/mach/gnumach/memory_management.mdwn14
-rw-r--r--microkernel/mach/gnumach/ports.mdwn9
-rw-r--r--microkernel/mach/gnumach/projects/clean_up_the_code.mdwn11
-rw-r--r--microkernel/mach/history.mdwn20
-rw-r--r--microkernel/mach/interface/vm_map.mdwn16
-rw-r--r--microkernel/mach/message.mdwn8
-rw-r--r--microkernel/mach/message/msgh_id.mdwn254
-rw-r--r--microkernel/mach/port.mdwn6
-rw-r--r--microkernel/mach/rpc.mdwn6
14 files changed, 691 insertions, 21 deletions
diff --git a/microkernel/genode.mdwn b/microkernel/genode.mdwn
new file mode 100644
index 00000000..66dd6b38
--- /dev/null
+++ b/microkernel/genode.mdwn
@@ -0,0 +1,17 @@
+[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+ * [[RPC]]
+
+
+# IRC, freenode, #hurd, 2012-08-02
+
+ <mcsim> If someone interested, there is a channel with lectures about
+ Genode and L4: http://www.youtube.com/user/drsartakov?feature=watch
diff --git a/microkernel/genode/rpc.mdwn b/microkernel/genode/rpc.mdwn
new file mode 100644
index 00000000..4f68a9f6
--- /dev/null
+++ b/microkernel/genode/rpc.mdwn
@@ -0,0 +1,65 @@
+[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+
+# IRC, freenode, #hurd, 2012-07-20
+
+ <braunr> for the curious, genode has a very interesting way to provide IPC
+ <braunr> i think they're on the right path at least (synchronous rpc,
+ shared memory, signals, no IDL)
+ <braunr> (i just don't like their choosing C++ at the system core)
+ <kilobug> braunr: hum, how do you write the rpc if there is no IDL ?
+ <kilobug> braunr: in a dynamic language like Python or Ruby, you can have
+ "transparent" RPC with no IDL, but in a language like C or C++ ?
+ <kilobug> when you call them I mean
+ <braunr> kilobug: they call this dynamic marshalling based on C++ streams
+ <braunr> http://genode-labs.com/publications/dynrpc-2007.pdf
+ <kilobug> sounds quite ugly to use :s but that may because I'm not fond of
+ C++ itself ;)
+ <braunr> same for me
+ <braunr> they say inheritance in RPC interfaces is "a must"
+ <braunr> makes me skeptical
+ <braunr> other than that, it's very promising
+ <kilobug> from the caller side, having the RPC appearing to be normal
+ function calls (like you do with Mig or Corba) is quite pleasant, even if
+ writing IDL is burdersome, you write IDL only once while calling RPC is
+ done very often
+ <braunr> oh but they have that as well
+ <braunr> there is just an additional, thin layer of hand written code to
+ provide that
+ <kilobug> ok
+ <braunr> basically, interfaces are C++ virtual classes, which are then
+ inherited in client and server classes
+ <braunr> (although they're changing that with recursive templates)
+ <braunr> but i really like to idea of not relying on an IDL
+ <kilobug> recursive templates :s
+ <braunr> yeah :>
+ <braunr> must be some tricky code, but i guess once it's there, it must be
+ practical
+ <braunr> see
+ http://genode.org/documentation/release-notes/11.05#New_API_for_type-safe_inter-process_communication
+ <braunr> they also added typed capabilities, but i don't really like that
+ idea
+ <antrik> braunr: shared memory for what?
+ <braunr> antrik: for uh.. sharing ? :)
+ <braunr> antrik: these systems don't provide ipc primitives able to share
+ memory directly
+ <braunr> messages are always copied (although zero copy can be used)
+ <braunr> so sharing must be done separately
+ <antrik> hm... I realise that I have no idea how the map operation in L4 is
+ actually done...
+ <braunr> iirc, privileged threads handle that
+ <antrik> I guess you have to explicitly map before an RPC and revoke
+ afterwards, which is some overhead...
+ <braunr> so i guess it's separated as well
+ <braunr> i have one question in mind for now, maybe you can help me with
+ that :
+
+[[open_issues/synchronous_ipc]].
diff --git a/microkernel/l4.mdwn b/microkernel/l4.mdwn
index 7af5e6fc..de311497 100644
--- a/microkernel/l4.mdwn
+++ b/microkernel/l4.mdwn
@@ -1,5 +1,5 @@
-[[!meta copyright="Copyright © 2004, 2006, 2007, 2008, 2010, 2011 Free Software
-Foundation, Inc."]]
+[[!meta copyright="Copyright © 2004, 2006, 2007, 2008, 2010, 2011, 2012 Free
+Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -24,6 +24,12 @@ There was a GNU/Hurd [[port to L4|history/port_to_another_microkernel]], which
is now stalled.
+# IRC, freenode, #hurd, 2012-08-02
+
+ <mcsim> If someone interested, there is a channel with lectures about
+ Genode and L4: http://www.youtube.com/user/drsartakov?feature=watch
+
+
[[!ymlfront data="""
sel4:
diff --git a/microkernel/mach/deficiencies.mdwn b/microkernel/mach/deficiencies.mdwn
index f2f49975..e1f6debc 100644
--- a/microkernel/mach/deficiencies.mdwn
+++ b/microkernel/mach/deficiencies.mdwn
@@ -258,3 +258,265 @@ License|/fdl]]."]]"""]]
working on research around mach
<antrik> braunr: BTW, I have little doubt that making RPC first-class would
solve a number of problems... I just wonder how many others it would open
+
+
+# IRC, freenode, #hurd, 2012-09-04
+
+X15
+
+ <braunr> it was intended as a mach clone, but now that i have better
+ knowledge of both mach and the hurd, i don't want to retain mach
+ compatibility
+ <braunr> and unlike viengoos, it's not really experimental
+ <braunr> it's focused on memory and cpu scalability, and performance, with
+ techniques likes thread migration and rcu
+ <braunr> the design i have in mind is closer to what exists today, with
+ strong emphasis on scalability and performance, that's all
+ <braunr> and the reason the hurd can't be modified first is that my design
+ relies on some important design changes
+ <braunr> so there is a strong dependency on these mechanisms that requires
+ the kernel to exists first
+
+
+## IRC, freenode, #hurd, 2012-09-06
+
+In context of [[open_issues/multithreading]] and later [[open_issues/select]].
+
+ <gnu_srs> And you will address the design flaws or implementation faults
+ with x15?
+ <braunr> no
+ <braunr> i'll address the implementation details :p
+ <braunr> and some design issues like cpu and memory resource accounting
+ <braunr> but i won't implement generic resource containers
+ <braunr> assuming it's completed, my work should provide a hurd system on
+ par with modern monolithic systems
+ <braunr> (less performant of course, but performant, scalable, and with
+ about the same kinds of problems)
+ <braunr> for example, thread migration should be mandatory
+ <braunr> which would make client calls behave exactly like a userspace task
+ asking a service from the kernel
+ <braunr> you have to realize that, on a monolithic kernel, applications are
+ clients, and the kernel is a server
+ <braunr> and when performing a system call, the calling thread actually
+ services itself by running kernel code
+ <braunr> which is exactly what thread migration is for a multiserver system
+ <braunr> thread migration also implies sync IPC
+ <braunr> and sync IPC is inherently more performant because it only
+ requires one copy, no in kernel buffering
+ <braunr> sync ipc also avoids message floods, since client threads must run
+ server code
+ <gnu_srs> and this is not achievable with evolved gnumach and/or hurd?
+ <braunr> well that's not entirely true, because there is still a form of
+ async ipc, but it's a lot less likely
+ <braunr> it probably is
+ <braunr> but there are so many things to change i prefer starting from
+ scratch
+ <braunr> scalability itself probably requires a revamp of the hurd core
+ libraries
+ <braunr> and these libraries are like more than half of the hurd code
+ <braunr> mach ipc and vm are also very complicated
+ <braunr> it's better to get something new and simpler from the start
+ <gnu_srs> a major task nevertheless:-D
+ <braunr> at least with the vm, netbsd showed it's easier to achieve good
+ results from new code, as other mach vm based systems like freebsd
+ struggled to get as good
+ <braunr> well yes
+ <braunr> but at least it's not experimental
+ <braunr> everything i want to implement already exists, and is tested on
+ production systems
+ <braunr> it's just time to assemble those ideas and components together
+ into something that works
+ <braunr> you could see it as a qnx-like system with thread migration, the
+ global architecture of the hurd, and some improvements from linux like
+ rcu :)
+
+
+### IRC, freenode, #hurd, 2012-09-07
+
+ <antrik> braunr: thread migration is tested on production systems?
+ <antrik> BTW, I don't think that generally increasing the priority of
+ servers is a good idea
+ <antrik> in most cases, IPC should actually be sync. slpz looked at it at
+ some point, and concluded that the implementation actually has a
+ fast-path for that case. I wonder what happens to scheduling in this case
+ -- is the receiver sheduled immediately? if not, that's something to
+ fix...
+ <braunr> antrik: qnx does something very close to thread migration, yes
+ <braunr> antrik: i agree increasing the priority isn't a good thing, but
+ it's the best of the quick and dirty ways to reduce message floods
+ <braunr> the problem isn't sync ipc in mach
+ <braunr> the problem is the notifications (in our cases the dead name
+ notifications) that are by nature async
+ <braunr> and a malicious program could send whatever it wants at the
+ fastest rate it can
+ <antrik> braunr: malicious programs can do any number of DOS attacks on the
+ Hurd; I don't see how increasing priority of system servers is relevant
+ in that context
+ <antrik> (BTW, I don't think dead name notifications are async by
+ nature... just like for most other IPC, the *usual* case is that a server
+ thread is actively waiting for the message when it's generated)
+ <braunr> antrik: it's async with respect to the client
+ <braunr> antrik: and malicious programs shouldn't be able to do that kind
+ of dos
+ <braunr> but this won't be fixed any time soon
+ <braunr> on the other hand, a higher priority helps servers not create too
+ many threads because of notifications, and that's a good thing
+ <braunr> gnu_srs: the "fix" for this will be to rewrite select so that it's
+ synchronous btw
+ <braunr> replacing dead name notifications with something like cancelling a
+ previously installed select request
+ <antrik> no idea what "async with respect to the client" means
+ <braunr> it means the client doesn't wait for anything
+ <antrik> what is the client? what scenario are you talking about? how does
+ it affect scheduling?
+ <braunr> for notifications, it's usually the kernel
+ <braunr> it doesn't directly affect scheduling
+ <braunr> it affects the amount of messages a hurd server has to take care
+ of
+ <braunr> and the more messages, the more threads
+ <braunr> i'm talking about event loops
+ <braunr> and non blocking (or very short) selects
+ <antrik> the amount of messages is always the same. the question is whether
+ they can be handled before more come in. which would be the case if be
+ default the receiver gets scheduled as soon as a message is sent...
+ <braunr> no
+ <braunr> scheduling handoff doesn't imply the thread will be ready to
+ service the next message by the time a client sends a new one
+ <braunr> the rate at which a message queue gets filled has nothing to do
+ with scheduling handoff
+ <antrik> I very much doubt rates come into play at all
+ <braunr> well they do
+ <antrik> in my understanding the problem is that a lot of messages are sent
+ before the receive ever has a chance to handle them. so no matter how
+ fast the receiver is, it looses
+ <braunr> a lot of non blocking selects means a lot of reply ports
+ destroyed, a lot of dead name notifications, and what i call message
+ floods at server side
+ <braunr> no
+ <braunr> it used to work fine with cthreads
+ <braunr> it doesn't any more with pthreads because pthreads are slightly
+ slower
+ <antrik> if the receiver gets a chance to do some work each time a message
+ arrives, in most cases it would be free to service the next request with
+ the same thread
+ <braunr> no, because that thread won't have finished soon enough
+ <antrik> no, it *never* worked fine. it might have been slighly less
+ terrible.
+ <braunr> ok it didn't work fine, it worked ok
+ <braunr> it's entirely a matter of rate here
+ <braunr> and that's the big problem, because it shouldn't
+ <antrik> I'm pretty sure the thread would finish before the time slice ends
+ in almost all cases
+ <braunr> no
+ <braunr> too much contention
+ <braunr> and in addition locking a contended spin lock depresses priority
+ <braunr> so servers really waste a lot of time because of that
+ <antrik> I doubt contention would be a problem if the server gets a chance
+ to handle each request before 100 others come in
+ <braunr> i don't see how this is related
+ <braunr> handling a request doesn't mean entirely processing it
+ <braunr> there is *no* relation between handoff and the rate of incoming
+ message rate
+ <braunr> unless you assume threads can always complete their task in some
+ fixed and low duration
+ <antrik> sure there is. we are talking about a single-processor system
+ here.
+ <braunr> which is definitely not the case
+ <braunr> i don't see what it changes
+ <antrik> I'm pretty sure notifications can generally be handled in a very
+ short time
+ <braunr> if the server thread is scheduled as soon as it gets a message, it
+ can also get preempted by the kernel before replying
+ <braunr> no, notifications can actually be very long
+ <braunr> hurd_thread_cancel calls condition_broadcast
+ <braunr> so if there are a lot of threads on that ..
+ <braunr> (this is one of the optimizations i have in mind for pthreads,
+ since it's possible to precisely select the target thread with a doubly
+ linked list)
+ <braunr> but even if that's the case, there is no guarantee
+ <braunr> you can't assume it will be "quick enough"
+ <antrik> there is no guarantee. but I'm pretty sure it will be "quick
+ enough" in the vast majority of cases. which is all it needs.
+ <braunr> ok
+ <braunr> that's also the idea behind raising server priorities
+ <antrik> braunr: so you are saying the storms are all caused by select(),
+ and once this is fixed, the problem should be mostly gone and the
+ workaround not necessary anymore?
+ <braunr> yes
+ <antrik> let's hope you are right :-)
+ <braunr> :)
+ <antrik> (I still think though that making hand-off scheduling default is
+ the right thing to do, and would improve performance in general...)
+ <braunr> sure
+ <braunr> well
+ <braunr> no it's just a hack ;p
+ <braunr> but it's a right one
+ <braunr> the right thing to do is a lot more complicated
+ <braunr> as roland wrote a long time ago, the hurd doesn't need dead-name
+ notifications, or any notification other than the no-sender (which can be
+ replaced by a synchronous close on fd like operation)
+ <antrik> well, yes... I still think the viengoos approach is promising. I
+ meant the right thing to do in the existing context ;-)
+ <braunr> better than this priority hack
+ <antrik> oh? you happen to have a link? never heard of that...
+ <braunr> i didn't want to do it initially, even resorting to priority
+ depression on trhead creation to work around the problem
+ <braunr> hm maybe it wasn't him, i can't manage to find it
+ <braunr> antrik:
+ http://lists.gnu.org/archive/html/l4-hurd/2003-09/msg00009.html
+ <braunr> "Long ago, in specifying the constraints of
+ <braunr> what the Hurd needs from an underlying IPC system/object model we
+ made it
+ <braunr> very clear that we only need no-senders notifications for object
+ <braunr> implementors (servers)"
+ <braunr> "We don't in general make use of dead-name notifications,
+ <braunr> which are the general kind of object death notification Mach
+ provides and
+ <braunr> what serves as task death notification."
+ <braunr> "In the places we do, it's to serve
+ <braunr> some particular quirky need (and mostly those are side effects of
+ Mach's
+ <braunr> decouplable RPCs) and not a semantic model we insist on having."
+
+
+### IRC, freenode, #hurd, 2012-09-08
+
+ <antrik> The notion that seemed appropriate when we thought about these
+ issues for
+ <antrik> Fluke was that the "alert" facility be a feature of the IPC system
+ itself
+ <antrik> rather than another layer like the Hurd's io_interrupt protocol.
+ <antrik> braunr: funny, that's *exactly* what I was thinking when looking
+ at the io_interrupt mess :-)
+ <antrik> (and what ultimately convinced me that the Hurd could be much more
+ elegant with a custom-tailored kernel rather than building around Mach)
+
+
+## IRC, freenode, #hurd, 2012-09-24
+
+ <braunr> my initial attempt was a mach clone
+ <braunr> but now i want a mach-like kernel, without compability
+ <lisporu> which new licence ?
+ <braunr> and some very important changes like sync ipc
+ <braunr> gplv3
+ <braunr> (or later)
+ <lisporu> cool 8)
+ <braunr> yes it is gplv2+ since i didn't take the time to read gplv3, but
+ now that i have, i can't use anything else for such a project: )
+ <lisporu> what is mach-like ? (how it is different from Pistachio like ?)
+ <braunr> l4 doesn't provide capabilities
+ <lisporu> hmmm..
+ <braunr> you need a userspace for that
+ <braunr> +server
+ <braunr> and it relies on complete external memory management
+ <lisporu> how much work is done ?
+ <braunr> my kernel will provide capabilities, similar to mach ports, but
+ simpler (less overhead)
+ <braunr> i want the primitives right
+ <braunr> like multiprocessor, synchronization, virtual memory, etc..
+
+
+### IRC, freenode, #hurd, 2012-09-30
+
+ <braunr> for those interested, x15 is now a project of its own, with no
+ gnumach compability goal, and covered by gplv3+
diff --git a/microkernel/mach/external_pager_mechanism.mdwn b/microkernel/mach/external_pager_mechanism.mdwn
index 05a6cc56..54492b71 100644
--- a/microkernel/mach/external_pager_mechanism.mdwn
+++ b/microkernel/mach/external_pager_mechanism.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2002, 2007, 2008, 2010, 2011 Free Software
+[[!meta copyright="Copyright © 2002, 2007, 2008, 2010, 2011, 2012 Free Software
Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
@@ -23,8 +23,8 @@ latter.
In Mach, a [[task]]'s [[virtual_address_space]] consists of references to
[[memory_object]]s.
-To associate a memory object with a portion of a task's
-address space, `vm_map` is invoked on a capability designating
+To associate a memory object with a portion of a task's address space,
+[[`vm_map`|interface/vm_map]] is invoked on a capability designating
the task and passing a reference to the memory object
and the offset at which to install it. (The first time
a task maps an object, Mach sends an initialization message
@@ -69,8 +69,8 @@ it to the port set that it is listening on and returns a capability (a port
send right) to the client.
(C) The client attempts to map the object into its address space using
-the `vm_map` RPC. It passes a reference to the port that the server gave
-it to the vm server (typically Mach).
+the [[`vm_map`|interface/vm_map]] RPC. It passes a reference to the port that
+the server gave it to the vm server (typically Mach).
(D) Since Mach has never seen the object before, it queues a
`memory_object_init` on the given port along with a send right (the
@@ -84,8 +84,8 @@ structures to manage the mapping and then invokes the
`memory_object_ready` method on the control object.
(F) The kernel sees that the manager is ready, sets up the appropriate
-mappings in the client's address space and then replies to the `vm_map` RPC indicating
-success.
+mappings in the client's address space and then replies to the
+[[`vm_map`|interface/vm_map]] RPC indicating success.
There is nothing stopping others from playing *the kernel*. This is
not a security problem: clients must [[trust]] the server from whom they
diff --git a/microkernel/mach/gnumach/memory_management.mdwn b/microkernel/mach/gnumach/memory_management.mdwn
index c630af05..3e158b7c 100644
--- a/microkernel/mach/gnumach/memory_management.mdwn
+++ b/microkernel/mach/gnumach/memory_management.mdwn
@@ -48,6 +48,7 @@ License|/fdl]]."]]"""]]
<braunr> and mmu management
<braunr> (but maybe that's what you meant by physical memory)
+
## IRC, freenode, #hurd, 2011-02-16
<braunr> antrik: youpi added it for xen, yes
@@ -119,3 +120,16 @@ License|/fdl]]."]]"""]]
<braunr> there is issue with watch ./slabinfo which turned in a infinite
loop, but it didn't affect the stability of the system
<braunr> actually with a 64-bits kernel, we could use a 4/x split
+
+
+# IRC, freenode, #hurd, 2012-08-10
+
+ <braunr> all modern systems embed the kernel in every address space
+ <braunr> which allows reduced overhead when making a system call
+ <braunr> sometimes there is no context switch at all
+ <braunr> on i386, there are security checks to upgrade the privilege level
+ (switch to ring 0), and when used, kernel page tables are global, so
+ they're not flushed
+ <braunr> using sysenter/sysexit makes it even faster
+
+[[open_issues/system_call_mechanism]].
diff --git a/microkernel/mach/gnumach/ports.mdwn b/microkernel/mach/gnumach/ports.mdwn
index f114460c..e7fdb446 100644
--- a/microkernel/mach/gnumach/ports.mdwn
+++ b/microkernel/mach/gnumach/ports.mdwn
@@ -1,5 +1,5 @@
-[[!meta copyright="Copyright © 2007, 2008, 2009, 2011 Free Software Foundation,
-Inc."]]
+[[!meta copyright="Copyright © 2007, 2008, 2009, 2011, 2012 Free Software
+Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -13,6 +13,11 @@ License|/fdl]]."]]"""]]
* [[Xen]]
+ * [[open_issues/64-bit_port]]. There is some preliminary work for a
+ x86\_64 port.
+
+ * [[open_issues/ARM_port]]. Is not in a usable state.
+
* [PowerPC](http://www.pjbruin.dds.nl/hurd/). Is not in a usable state.
* Alpha: [project I](http://savannah.nongnu.org/projects/hurd-alpha), and
diff --git a/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn b/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn
index 2a9b4b60..89a27b01 100644
--- a/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn
+++ b/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2005, 2006, 2007, 2008, 2010 Free Software
+[[!meta copyright="Copyright © 2005, 2006, 2007, 2008, 2010, 2012 Free Software
Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
@@ -121,3 +121,12 @@ further files (also exported ones) that serve no real value, but are being
# Rewrite ugly code
+
+
+# IRC, freenode, #hurd, 2012-09-06
+
+ <mcsim> hello. Why size parameter of rpc device_read has type
+ "mach_msg_type_number_t *"? Why not just "vm_size_t *"?
+ <mcsim> this parameter has name data_count
+ <braunr> that's one of the reasons mach is confusing
+ <braunr> i can't really tell you why, it's messy :/
diff --git a/microkernel/mach/history.mdwn b/microkernel/mach/history.mdwn
index 5a3608cd..776bb1d7 100644
--- a/microkernel/mach/history.mdwn
+++ b/microkernel/mach/history.mdwn
@@ -58,3 +58,23 @@ Verbatim copying and distribution of this entire article is permitted in any med
Apple's Macintosh OSX (OS 10.x) is based on [Darwin](http://www.apple.com/macosx/technologies/darwin.html). _"Darwin uses a monolithic kernel based on [[TWiki/FreeBSD]] 4.4 and the OSF/mk Mach 3."_ Darwin also has a [Kernel Programming](http://developer.apple.com/techpubs/macosx/Darwin/General/KernelProgramming/About/index.html) Book.
-- [[Main/GrantBow]] - 22 Oct 2002
+
+IRC, freenode, #hurd, 2012-08-29:
+
+ <pavlx> was moved the page from apple.com about darwin kernel programming
+ as described on the
+ https://www.gnu.org/software/hurd/microkernel/mach/history.html
+ <pavlx> i found the page and it's
+ https://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/About/About.html
+ <pavlx> it's not anymore the old page
+ http://developer.apple.com/techpubs/macosx/Darwin/General/KernelProgramming/About/index.html
+ <pavlx> and the link about darwin does noit exists anymore ! the new one
+ could be https://ssl.apple.com/science/profiles/cornell
+ <pavlx> the old one was
+ http://www.apple.com/macosx/technologies/darwin.html
+ <pavlx> the link to Darwin is changed i suppose that the nw one it's
+ https://ssl.apple.com/science/profiles/cornell
+ <pavlx> and the link to Kern Programming it's
+ https://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/About/About.html
+ <pavlx> can't be anymore
+ http://developer.apple.com/techpubs/macosx/Darwin/General/KernelProgramming/About/index.html
diff --git a/microkernel/mach/interface/vm_map.mdwn b/microkernel/mach/interface/vm_map.mdwn
new file mode 100644
index 00000000..7232209b
--- /dev/null
+++ b/microkernel/mach/interface/vm_map.mdwn
@@ -0,0 +1,16 @@
+[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+Used in [[glibc]] for implementing [[glibc/mmap]], brk and sbrk, and *attempt
+to map page zero redzoned* (`[glibc]/hurd/hurdstartup.c:_hurd_startup`).
+
+[[!tag open_issue_glibc open_issue_gnumach]]It has an awkward mention in
+`[glibc]/sysdeps/mach/hurd/dl-sysdep.c:fmh`: *XXX loser kludge for vm\_map
+kernel bug*.
diff --git a/microkernel/mach/message.mdwn b/microkernel/mach/message.mdwn
index ba47671e..4c49af17 100644
--- a/microkernel/mach/message.mdwn
+++ b/microkernel/mach/message.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2002, 2003, 2010 Free Software Foundation,
+[[!meta copyright="Copyright © 2002, 2003, 2010, 2012 Free Software Foundation,
Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
@@ -9,9 +9,11 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
-*Messages* are collections of typed data, with a defined layout.
+*Messages* are collections of typed data, with a defined layout, including an
+[[ID|msgh_id]].
-They are used for [[IPC]], and are sent to and received from [[port]]s.
+They are used for [[IPC]], and are sent to and received from [[port]]s using
+the `mach_msg` interface.
These messages are not only opaque data. They can also contain [[port
rights|port]] to be passed to another [[task]]. Port rights are either
diff --git a/microkernel/mach/message/msgh_id.mdwn b/microkernel/mach/message/msgh_id.mdwn
new file mode 100644
index 00000000..986fcbc7
--- /dev/null
+++ b/microkernel/mach/message/msgh_id.mdwn
@@ -0,0 +1,254 @@
+[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_mig]]
+
+Every [[message]] has an ID field, which is defined in the [[RPC]] `*.defs`
+files.
+
+
+# IRC, freenode, #hurd, 2012-07-12
+
+[Extending an existing RPC.]
+
+ <antrik> create a new call, either with a new variant of vm_statistics_t,
+ or a new structure with only the extra fields
+ <braunr> that seems cleaner indeed
+ <braunr> but using different names for the same thing seems so tedious and
+ unnecessary :/
+ <antrik> it's extra effort, but it pays off
+ <braunr> i agree, it's the right way to do it
+ <braunr> but this implies some kind of versioning
+ <braunr> which is currently more or less done using mig subsystem numbers,
+ and skipping obsolete calls in rpc definition files
+ <braunr> and a subsystem is like 100 calls (200 with the replies)
+ <braunr> at some point we should recycle them
+ <braunr> or use truely huge ranges
+ <antrik> braunr: that's not something we need to worry about until we get
+ there -- which is not likely to happen any time soon :-)
+ <braunr> "There is no more room in this interface for additional calls."
+ <braunr> in mach.defs
+ <braunr> i'll use the mach4.defs file
+ <braunr> but it really makes no sense at all to do such things just because
+ we want to be compatible with 20 year old software nobody uses any more
+ <braunr> who cares about the skips used to keep us from using the old mach
+ 2.5 interface ..
+ <braunr> (and this 100 arbitrary limit is really ugly too)
+ <antrik> braunr: I agree that we don't want to be compatible with 20 years
+ old software. just Hurd stuff from the last few years is perfectly fine.
+ <tschwinge> braunr, antrik: I agree with the approach of using a new
+ RPC/data structure for incompatible changes, and I also agree that
+ recycling RPC slots that have been unused (skipped) for some years is
+ fine.
+ <antrik> tschwinge: well, we probably shouldn't just reuse them
+ arbitrarily; but rather do a mass purge if the need really arises...
+ <antrik> it would be confusing otherwise IMHO
+ <tschwinge> antrik: What do you understand by doing a mass purge?
+ <tschwinge> My idea indeed was to replace arbitrary "skip"s by new RPC
+ definitions.
+ <braunr> a purge would be good along with a mig change to make subsystem
+ and routines identifier larger
+ <braunr> i guess 16-bits width should do
+ <tschwinge> But what do you unterstand by a "purge" in this context.
+ <braunr> removing all the skips
+ <tschwinge> But that moves the RPC ids following after?
+ <braunr> yes
+ <braunr> that's why i think it's not a good thing, unless we also change
+ the numbering
+ <tschwinge> ... which is a incompatible change for all clients.
+ <braunr> yes
+ <tschwinge> OK, so you'd propose a new system and deprecate the current
+ one.
+ <braunr> not really new
+ <braunr> just larger numbers
+ <braunr> we must acknowledge interfaces change with time
+ <tschwinge> Yes, that's "new" enough. ;-)
+ <tschwinge> New in the sense that all clients use new iterfaces.
+ <braunr> that's enough to completely break compability, yes
+ <braunr> at least binary
+ <tschwinge> Yes.
+ <tschwinge> However, I don't see an urgent need for that, do you?
+ <tschwinge> Why not just recycled a skip that has been unused for a decade?
+ <braunr> i don't think we should care much about that, as the only real
+ issue i can see is when upgrading a system
+ <braunr> i don't say we shouldn't do that
+ <braunr> actually, my current patch does exactly this
+ <tschwinge> OK. :-)
+ <braunr> purging is another topic
+ <braunr> but purging without making numbers larger seems a bit pointless
+ <braunr> as the point is allowing developers to change interfaces without
+ breaking short time compability
+ <braunr> compatibility*
+ <braunr> also, interfaces, even stable, can have more than 100 calls
+ <braunr> (at the same time, i don't think there would ever be many
+ interfaces, so using 16-bits integers for the subsystems and the calls
+ should really be fine, and cleanly aligned in memory)
+ <antrik> tschwinge: you are right, it was a brain fart :-)
+ <antrik> no purge obviously
+ <antrik> but I think we only should start with filling skips once all IDs
+ in the subsystem are exhausted
+ <antrik> braunr: the 100 is not fixed in MIG IIRC; it's a definition we
+ make somewhere
+ <antrik> BTW, using multiple subsystems for "overflowing" interfaces is a
+ bit ugly, but not to bad I'd say... so I wouldn't really consider this a
+ major problem
+ <antrik> err... not too bad
+ <antrik> especially since Hurd subsystem usually are spaced 1000 aways, so
+ there are some "spare" blocks between them anyways
+ <braunr> hm i'm almost sure it's related to mig
+ <braunr> that's how the reply id is computed
+ <antrik> of course it is related to MIG... but I have a vague recollection
+ that this constant is not fixed in the MIG code, but rather supplied
+ somewhere. might be wrong though :-)
+ <pinotree> you mean like the 101-200 skip block in hurd/tioctl.defs?
+ <antrik> pinotree: exactly
+ <antrik> these are reserved for reply message IDs
+ <antrik> at 200 a new request message block begins...
+ <braunr> server.c: fprintf(file, "\tOutP->Head.msgh_id = InP->msgh_id +
+ 100;\n");
+ <braunr> it's not even a define in the mig code :/
+ <pinotree> meaning that in the space of an hurd subsystem there are max 500
+ effective rpc's?
+ <antrik> actually, ioctls are rather special, as the numbers are computed
+ from the ioctl properties...
+ <antrik> braunr: :-(
+ <braunr> pinotree: how do you get this value ?
+ <pinotree> braunr: 1000/2? :)
+ <braunr> ?
+ <braunr> why not 20000/3 ?
+ <antrik> pinotree: yes
+ <braunr> where do they come from ?
+ <braunr> ah ok sorry
+ <pinotree> braunr: 1000 is the space of each subsystem, and each rpc takes
+ an id + its replu
+ <pinotree> *reply
+ <braunr> right
+ <braunr> 500 is fine
+ <braunr> better than 100
+ <braunr> but still, 64k is way better
+ <braunr> and not harder to do
+ <pinotree> (hey, i'm the noob in this :) )
+ <antrik> braunr: it's just how "we" lay out subsystems... nothing fixed
+ about it really; we could just as well define new subsystems with 10000
+ or whatever if we wanted
+ <braunr> yes
+ <braunr> but we still have to consider this mig limit
+ <antrik> there are one or two odd exceptions though, with "related"
+ subsystems starting at ??500...
+ <antrik> braunr: right. it's not pretty -- but I wouldn't consider it
+ enough of a problem to invest major effort in changing this...
+ <braunr> agreed
+ <braunr> at least not while our interfaces don't change often
+ <braunr> which shouldn't happen any time soon
+
+ <tschwinge> Hmm, I also remember seeing some emails about indeed versioning
+ RPCs (by Roland, I think). I can try to look that up if there's
+ interest.
+
+ <braunr> i'm only adding a cached pages count you know :)
+ <braunr> (well actually, this is now a vm_stats call that can replace
+ vm_statistics, and uses flavors similar to task_info)
+ <antrik> braunr: I don't think introducing "flavors" is a good idea
+ <braunr> i just did it the way others calls were done
+ <braunr> other*
+ <braunr> woud you prefer a larger structure with append-only upgrades ?
+ <antrik> I prefer introducing new calls. it avoids an unncessary layer of
+ indirection
+ <antrik> flavors are not exactly RPC-over-RPC, but definitely going down
+ that road...
+ <braunr> right
+ <antrik> as fetching VM statistics is not performance-critical, I would
+ suggest adding a new call with only the extra stats you are
+ introducing. then if someone runs an old kernel not implementing that
+ call, the values are simply left blank in the caller. makes
+ backward-compatibility a no-brainer
+ <antrik> (the alternative is a new call fetching both the traditional and
+ the new stats -- but this is not necessary here, as an extra call
+ shouldn't hurt)
+ <braunr> antrik: all right
+
+
+## IRC, freenode, #hurd, 2012-07-13
+
+ <braunr> so, should i replace old, unused mach.defs RPCs with mine, or add
+ them to e.g. mach4.defs ?
+ <antrik> braunr: hm... actually I wonder whether we shouldn't add a
+ gnumach.defs -- after all, it's neither old mach nor mach4 interfaces...
+ <braunr> true
+ <braunr> good idea
+ <braunr> i'll do just that
+ <braunr> hm, doesn't adding a new interface file requires some handling in
+ glibc ?
+ <youpi> simply rebuild it
+ <braunr> youpi: no i mean
+ <braunr> youpi: glibc knows about mach.defs and mach4.defs, but i guess we
+ should add something so that it knows about gnumach.defs
+ <youpi> ah
+ <youpi> probably, yes
+ <braunr> ok
+ <braunr> i don't understand why these files are part of the glibc headers
+ <pinotree> are they?
+ <braunr> (i mean mach_interface.h and mach4.h)
+ <braunr> for example
+ <braunr> youpi: the interface i'll add is vm_cache_statistics(task,
+ &cached_objects, &cached_pages)
+ <braunr> if it's ok i'll commit directly into the gnumach repository
+ <youpi> shouldn't it rather be a int array, to make it extensible?
+ <youpi> like other stat functions of gnumach
+ <braunr> antrik was against doing that
+ <braunr> well, he was against using flavors
+ <braunr> maybe we could have an extensible array yes, and require additions
+ at the end of the structure
+
+
+## IRC, freenode, #hurd, 2012-07-14
+
+ <antrik> braunr: there are two reasons why the files are part of glibc. one
+ is that glibc itself uses them, so it would be painful to handle
+ otherwise. the other is that libc is traditionally responsible for
+ providing the system interface...
+ <antrik> having said that, I'm not sure we should stick with that :-)
+ <braunr> antrik: what do you think about having a larger structure with
+ reserved fields ? sounds a lot better than flavors, doesn't it ?
+ <youpi> antrik: it's in debian, yes
+ <braunr> grmbl, adding a new interface just for a single call is really
+ tedious
+ <braunr> i'll just add it to mach4
+ <antrik> braunr: well, it's not unlikely there will be other new calls in
+ the future... but I guess using mach4.defs isn't too bad
+ <antrik> braunr: as for reserved fields, I guess that is somewhat better
+ than flavors; but I can't say I exactly like the idea either...
+ <braunr> antrik: there is room in mach4 ;p
+
+
+## IRC, freenode, #hurd, 2012-07-23
+
+ <tschwinge> I'm not sure yet whether I'm happy with adding the RPC to
+ mach4.defs.
+ <braunr> that's the only question yes
+ <braunr> (well, no, not only)
+ <braunr> as i know have a better view of what's involved, it may make sense
+ to create a gnumach.defs file
+ <braunr> tschwinge: all right i'll create a gnumach.defs file
+ <tschwinge> braunr: Well, if there is general agreement that this is the
+ way to go.
+ <tschwinge> braunr: In that case, I guess there's no point in being more
+ fine-grained -- gnumach-vm.defs or similar -- that'd probably be
+ over-engineering. If the glibc bits for libmachuser are not
+ straight-forward, I can help with that of course.
+ <braunr> ok
+
+
+## IRC, freenode, #hurd, 2012-07-27
+
+ <braunr> tschwinge: i've pushed a patch on the gnumach page_cache branch
+ that adds a gnumach.defs interface
+ <braunr> tschwinge: if you think it's ok, i'll rewrite a formal changelog
+ so it can be applied
diff --git a/microkernel/mach/port.mdwn b/microkernel/mach/port.mdwn
index 26b55456..ccc7286f 100644
--- a/microkernel/mach/port.mdwn
+++ b/microkernel/mach/port.mdwn
@@ -1,5 +1,5 @@
-[[!meta copyright="Copyright © 2002, 2003, 2007, 2008, 2010, 2011 Free Software
-Foundation, Inc."]]
+[[!meta copyright="Copyright © 2002, 2003, 2007, 2008, 2010, 2011, 2012 Free
+Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -86,4 +86,4 @@ When a server process' thread receives from a port set, it dequeues exactly one
message from any of the ports that has a message available in its queue.
This concept of port sets is also the facility that makes convenient
-implementation of [[UNIX]]'s `select` [[system_call]] possible.
+implementation of [[UNIX's `select` system call|glibc/select]] possible.
diff --git a/microkernel/mach/rpc.mdwn b/microkernel/mach/rpc.mdwn
index 422e0441..3615fc12 100644
--- a/microkernel/mach/rpc.mdwn
+++ b/microkernel/mach/rpc.mdwn
@@ -1,5 +1,5 @@
-[[!meta copyright="Copyright © 2002, 2003, 2007, 2008, 2010, 2011 Free Software
-Foundation, Inc."]]
+[[!meta copyright="Copyright © 2002, 2003, 2007, 2008, 2010, 2011, 2012 Free
+Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -18,7 +18,7 @@ transparently. This can be implemented with user [[task]]s, but there is an
implementation in the kernel possible, too, which is called *NORMA*, but is not
avilable in [[GNU Mach|gnumach]].
-The RPC stub code generated by [[MIG]].
+The RPC stub code is generated by [[MIG]] to send appropriate [[message]]s.
# Tracing