From 5bd36fdff16871eb7d06fc26cac07e7f2703432b Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Thu, 29 Nov 2012 01:33:22 +0100 Subject: IRC. --- microkernel/mach/deficiencies.mdwn | 262 +++++++++++++++++++++ microkernel/mach/gnumach/memory_management.mdwn | 14 ++ microkernel/mach/gnumach/ports.mdwn | 9 +- .../mach/gnumach/projects/clean_up_the_code.mdwn | 11 +- microkernel/mach/history.mdwn | 20 ++ microkernel/mach/port.mdwn | 6 +- 6 files changed, 316 insertions(+), 6 deletions(-) (limited to 'microkernel/mach') diff --git a/microkernel/mach/deficiencies.mdwn b/microkernel/mach/deficiencies.mdwn index f2f49975..e1f6debc 100644 --- a/microkernel/mach/deficiencies.mdwn +++ b/microkernel/mach/deficiencies.mdwn @@ -258,3 +258,265 @@ License|/fdl]]."]]"""]] working on research around mach braunr: BTW, I have little doubt that making RPC first-class would solve a number of problems... I just wonder how many others it would open + + +# IRC, freenode, #hurd, 2012-09-04 + +X15 + + it was intended as a mach clone, but now that i have better + knowledge of both mach and the hurd, i don't want to retain mach + compatibility + and unlike viengoos, it's not really experimental + it's focused on memory and cpu scalability, and performance, with + techniques likes thread migration and rcu + the design i have in mind is closer to what exists today, with + strong emphasis on scalability and performance, that's all + and the reason the hurd can't be modified first is that my design + relies on some important design changes + so there is a strong dependency on these mechanisms that requires + the kernel to exists first + + +## IRC, freenode, #hurd, 2012-09-06 + +In context of [[open_issues/multithreading]] and later [[open_issues/select]]. + + And you will address the design flaws or implementation faults + with x15? + no + i'll address the implementation details :p + and some design issues like cpu and memory resource accounting + but i won't implement generic resource containers + assuming it's completed, my work should provide a hurd system on + par with modern monolithic systems + (less performant of course, but performant, scalable, and with + about the same kinds of problems) + for example, thread migration should be mandatory + which would make client calls behave exactly like a userspace task + asking a service from the kernel + you have to realize that, on a monolithic kernel, applications are + clients, and the kernel is a server + and when performing a system call, the calling thread actually + services itself by running kernel code + which is exactly what thread migration is for a multiserver system + thread migration also implies sync IPC + and sync IPC is inherently more performant because it only + requires one copy, no in kernel buffering + sync ipc also avoids message floods, since client threads must run + server code + and this is not achievable with evolved gnumach and/or hurd? + well that's not entirely true, because there is still a form of + async ipc, but it's a lot less likely + it probably is + but there are so many things to change i prefer starting from + scratch + scalability itself probably requires a revamp of the hurd core + libraries + and these libraries are like more than half of the hurd code + mach ipc and vm are also very complicated + it's better to get something new and simpler from the start + a major task nevertheless:-D + at least with the vm, netbsd showed it's easier to achieve good + results from new code, as other mach vm based systems like freebsd + struggled to get as good + well yes + but at least it's not experimental + everything i want to implement already exists, and is tested on + production systems + it's just time to assemble those ideas and components together + into something that works + you could see it as a qnx-like system with thread migration, the + global architecture of the hurd, and some improvements from linux like + rcu :) + + +### IRC, freenode, #hurd, 2012-09-07 + + braunr: thread migration is tested on production systems? + BTW, I don't think that generally increasing the priority of + servers is a good idea + in most cases, IPC should actually be sync. slpz looked at it at + some point, and concluded that the implementation actually has a + fast-path for that case. I wonder what happens to scheduling in this case + -- is the receiver sheduled immediately? if not, that's something to + fix... + antrik: qnx does something very close to thread migration, yes + antrik: i agree increasing the priority isn't a good thing, but + it's the best of the quick and dirty ways to reduce message floods + the problem isn't sync ipc in mach + the problem is the notifications (in our cases the dead name + notifications) that are by nature async + and a malicious program could send whatever it wants at the + fastest rate it can + braunr: malicious programs can do any number of DOS attacks on the + Hurd; I don't see how increasing priority of system servers is relevant + in that context + (BTW, I don't think dead name notifications are async by + nature... just like for most other IPC, the *usual* case is that a server + thread is actively waiting for the message when it's generated) + antrik: it's async with respect to the client + antrik: and malicious programs shouldn't be able to do that kind + of dos + but this won't be fixed any time soon + on the other hand, a higher priority helps servers not create too + many threads because of notifications, and that's a good thing + gnu_srs: the "fix" for this will be to rewrite select so that it's + synchronous btw + replacing dead name notifications with something like cancelling a + previously installed select request + no idea what "async with respect to the client" means + it means the client doesn't wait for anything + what is the client? what scenario are you talking about? how does + it affect scheduling? + for notifications, it's usually the kernel + it doesn't directly affect scheduling + it affects the amount of messages a hurd server has to take care + of + and the more messages, the more threads + i'm talking about event loops + and non blocking (or very short) selects + the amount of messages is always the same. the question is whether + they can be handled before more come in. which would be the case if be + default the receiver gets scheduled as soon as a message is sent... + no + scheduling handoff doesn't imply the thread will be ready to + service the next message by the time a client sends a new one + the rate at which a message queue gets filled has nothing to do + with scheduling handoff + I very much doubt rates come into play at all + well they do + in my understanding the problem is that a lot of messages are sent + before the receive ever has a chance to handle them. so no matter how + fast the receiver is, it looses + a lot of non blocking selects means a lot of reply ports + destroyed, a lot of dead name notifications, and what i call message + floods at server side + no + it used to work fine with cthreads + it doesn't any more with pthreads because pthreads are slightly + slower + if the receiver gets a chance to do some work each time a message + arrives, in most cases it would be free to service the next request with + the same thread + no, because that thread won't have finished soon enough + no, it *never* worked fine. it might have been slighly less + terrible. + ok it didn't work fine, it worked ok + it's entirely a matter of rate here + and that's the big problem, because it shouldn't + I'm pretty sure the thread would finish before the time slice ends + in almost all cases + no + too much contention + and in addition locking a contended spin lock depresses priority + so servers really waste a lot of time because of that + I doubt contention would be a problem if the server gets a chance + to handle each request before 100 others come in + i don't see how this is related + handling a request doesn't mean entirely processing it + there is *no* relation between handoff and the rate of incoming + message rate + unless you assume threads can always complete their task in some + fixed and low duration + sure there is. we are talking about a single-processor system + here. + which is definitely not the case + i don't see what it changes + I'm pretty sure notifications can generally be handled in a very + short time + if the server thread is scheduled as soon as it gets a message, it + can also get preempted by the kernel before replying + no, notifications can actually be very long + hurd_thread_cancel calls condition_broadcast + so if there are a lot of threads on that .. + (this is one of the optimizations i have in mind for pthreads, + since it's possible to precisely select the target thread with a doubly + linked list) + but even if that's the case, there is no guarantee + you can't assume it will be "quick enough" + there is no guarantee. but I'm pretty sure it will be "quick + enough" in the vast majority of cases. which is all it needs. + ok + that's also the idea behind raising server priorities + braunr: so you are saying the storms are all caused by select(), + and once this is fixed, the problem should be mostly gone and the + workaround not necessary anymore? + yes + let's hope you are right :-) + :) + (I still think though that making hand-off scheduling default is + the right thing to do, and would improve performance in general...) + sure + well + no it's just a hack ;p + but it's a right one + the right thing to do is a lot more complicated + as roland wrote a long time ago, the hurd doesn't need dead-name + notifications, or any notification other than the no-sender (which can be + replaced by a synchronous close on fd like operation) + well, yes... I still think the viengoos approach is promising. I + meant the right thing to do in the existing context ;-) + better than this priority hack + oh? you happen to have a link? never heard of that... + i didn't want to do it initially, even resorting to priority + depression on trhead creation to work around the problem + hm maybe it wasn't him, i can't manage to find it + antrik: + http://lists.gnu.org/archive/html/l4-hurd/2003-09/msg00009.html + "Long ago, in specifying the constraints of + what the Hurd needs from an underlying IPC system/object model we + made it + very clear that we only need no-senders notifications for object + implementors (servers)" + "We don't in general make use of dead-name notifications, + which are the general kind of object death notification Mach + provides and + what serves as task death notification." + "In the places we do, it's to serve + some particular quirky need (and mostly those are side effects of + Mach's + decouplable RPCs) and not a semantic model we insist on having." + + +### IRC, freenode, #hurd, 2012-09-08 + + The notion that seemed appropriate when we thought about these + issues for + Fluke was that the "alert" facility be a feature of the IPC system + itself + rather than another layer like the Hurd's io_interrupt protocol. + braunr: funny, that's *exactly* what I was thinking when looking + at the io_interrupt mess :-) + (and what ultimately convinced me that the Hurd could be much more + elegant with a custom-tailored kernel rather than building around Mach) + + +## IRC, freenode, #hurd, 2012-09-24 + + my initial attempt was a mach clone + but now i want a mach-like kernel, without compability + which new licence ? + and some very important changes like sync ipc + gplv3 + (or later) + cool 8) + yes it is gplv2+ since i didn't take the time to read gplv3, but + now that i have, i can't use anything else for such a project: ) + what is mach-like ? (how it is different from Pistachio like ?) + l4 doesn't provide capabilities + hmmm.. + you need a userspace for that + +server + and it relies on complete external memory management + how much work is done ? + my kernel will provide capabilities, similar to mach ports, but + simpler (less overhead) + i want the primitives right + like multiprocessor, synchronization, virtual memory, etc.. + + +### IRC, freenode, #hurd, 2012-09-30 + + for those interested, x15 is now a project of its own, with no + gnumach compability goal, and covered by gplv3+ diff --git a/microkernel/mach/gnumach/memory_management.mdwn b/microkernel/mach/gnumach/memory_management.mdwn index c630af05..3e158b7c 100644 --- a/microkernel/mach/gnumach/memory_management.mdwn +++ b/microkernel/mach/gnumach/memory_management.mdwn @@ -48,6 +48,7 @@ License|/fdl]]."]]"""]] and mmu management (but maybe that's what you meant by physical memory) + ## IRC, freenode, #hurd, 2011-02-16 antrik: youpi added it for xen, yes @@ -119,3 +120,16 @@ License|/fdl]]."]]"""]] there is issue with watch ./slabinfo which turned in a infinite loop, but it didn't affect the stability of the system actually with a 64-bits kernel, we could use a 4/x split + + +# IRC, freenode, #hurd, 2012-08-10 + + all modern systems embed the kernel in every address space + which allows reduced overhead when making a system call + sometimes there is no context switch at all + on i386, there are security checks to upgrade the privilege level + (switch to ring 0), and when used, kernel page tables are global, so + they're not flushed + using sysenter/sysexit makes it even faster + +[[open_issues/system_call_mechanism]]. diff --git a/microkernel/mach/gnumach/ports.mdwn b/microkernel/mach/gnumach/ports.mdwn index f114460c..e7fdb446 100644 --- a/microkernel/mach/gnumach/ports.mdwn +++ b/microkernel/mach/gnumach/ports.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2007, 2008, 2009, 2011 Free Software Foundation, -Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2009, 2011, 2012 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -13,6 +13,11 @@ License|/fdl]]."]]"""]] * [[Xen]] + * [[open_issues/64-bit_port]]. There is some preliminary work for a + x86\_64 port. + + * [[open_issues/ARM_port]]. Is not in a usable state. + * [PowerPC](http://www.pjbruin.dds.nl/hurd/). Is not in a usable state. * Alpha: [project I](http://savannah.nongnu.org/projects/hurd-alpha), and diff --git a/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn b/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn index 2a9b4b60..89a27b01 100644 --- a/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn +++ b/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2005, 2006, 2007, 2008, 2010 Free Software +[[!meta copyright="Copyright © 2005, 2006, 2007, 2008, 2010, 2012 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -121,3 +121,12 @@ further files (also exported ones) that serve no real value, but are being # Rewrite ugly code + + +# IRC, freenode, #hurd, 2012-09-06 + + hello. Why size parameter of rpc device_read has type + "mach_msg_type_number_t *"? Why not just "vm_size_t *"? + this parameter has name data_count + that's one of the reasons mach is confusing + i can't really tell you why, it's messy :/ diff --git a/microkernel/mach/history.mdwn b/microkernel/mach/history.mdwn index 5a3608cd..776bb1d7 100644 --- a/microkernel/mach/history.mdwn +++ b/microkernel/mach/history.mdwn @@ -58,3 +58,23 @@ Verbatim copying and distribution of this entire article is permitted in any med Apple's Macintosh OSX (OS 10.x) is based on [Darwin](http://www.apple.com/macosx/technologies/darwin.html). _"Darwin uses a monolithic kernel based on [[TWiki/FreeBSD]] 4.4 and the OSF/mk Mach 3."_ Darwin also has a [Kernel Programming](http://developer.apple.com/techpubs/macosx/Darwin/General/KernelProgramming/About/index.html) Book. -- [[Main/GrantBow]] - 22 Oct 2002 + +IRC, freenode, #hurd, 2012-08-29: + + was moved the page from apple.com about darwin kernel programming + as described on the + https://www.gnu.org/software/hurd/microkernel/mach/history.html + i found the page and it's + https://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/About/About.html + it's not anymore the old page + http://developer.apple.com/techpubs/macosx/Darwin/General/KernelProgramming/About/index.html + and the link about darwin does noit exists anymore ! the new one + could be https://ssl.apple.com/science/profiles/cornell + the old one was + http://www.apple.com/macosx/technologies/darwin.html + the link to Darwin is changed i suppose that the nw one it's + https://ssl.apple.com/science/profiles/cornell + and the link to Kern Programming it's + https://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/About/About.html + can't be anymore + http://developer.apple.com/techpubs/macosx/Darwin/General/KernelProgramming/About/index.html diff --git a/microkernel/mach/port.mdwn b/microkernel/mach/port.mdwn index 26b55456..ccc7286f 100644 --- a/microkernel/mach/port.mdwn +++ b/microkernel/mach/port.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2002, 2003, 2007, 2008, 2010, 2011 Free Software -Foundation, Inc."]] +[[!meta copyright="Copyright © 2002, 2003, 2007, 2008, 2010, 2011, 2012 Free +Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -86,4 +86,4 @@ When a server process' thread receives from a port set, it dequeues exactly one message from any of the ports that has a message available in its queue. This concept of port sets is also the facility that makes convenient -implementation of [[UNIX]]'s `select` [[system_call]] possible. +implementation of [[UNIX's `select` system call|glibc/select]] possible. -- cgit v1.2.3