diff options
Diffstat (limited to 'microkernel')
| -rw-r--r-- | microkernel/mach/deficiencies.mdwn | 128 | ||||
| -rw-r--r-- | microkernel/mach/gnumach/debugging.mdwn | 5 | ||||
| -rw-r--r-- | microkernel/mach/gnumach/interface/syscall/mach_print.mdwn | 29 | ||||
| -rw-r--r-- | microkernel/mach/history.mdwn | 134 | ||||
| -rw-r--r-- | microkernel/mach/mig.mdwn | 6 | ||||
| -rw-r--r-- | microkernel/mach/mig/structured_data.mdwn | 119 |
6 files changed, 417 insertions, 4 deletions
diff --git a/microkernel/mach/deficiencies.mdwn b/microkernel/mach/deficiencies.mdwn index d1cdeb54..03e4a8b0 100644 --- a/microkernel/mach/deficiencies.mdwn +++ b/microkernel/mach/deficiencies.mdwn @@ -2190,3 +2190,131 @@ In context of [[open_issues/multithreading]] and later [[open_issues/select]]. <core-ix> i really need to dig into this code at some point :) <braunr> well you may but you may not see that property from the code itself + + +## IRC, freenode, #hurd, 2013-06-28 + + <teythoon> so tell me about x15 if you are in the mood to talk about that + <braunr> what do you want to know ? + <teythoon> well, the high level stuff first + <teythoon> like what's the big picture + <braunr> the big picture is that x15 is intended to be a "better mach for + the hurd + <braunr> " + <braunr> mach is too general purpose + <braunr> its ipc mechanism too powerful + <braunr> too complicated, error prone, and slow + <braunr> so i intend to build something a lot simpler and faster :p + <teythoon> so your big picture includes actually porting hurd? i thought i + read somewhere that you have a rewrite in mind + <braunr> it's a clone, yes + <braunr> x15 will feature mostly sync ipc, and no high level types inside + messages + <braunr> the ipc system call will look like what qnx does + <braunr> send-recv from the client, recv/reply/reply-recv from the server + <teythoon> but doesn't sync mean that your context switch will have to be + quite fast? + <braunr> how does that differ from the async approach ? + <braunr> (keep in mind that almost all hurd RPCs are synchronous) + <teythoon> yes, I know, and it also affects async mode, but a slow switch + is worse for the sync case, isn't it? + <teythoon> ok so your ipc will be more agnostic wrt to what it transports? + unlike mig I presume? + <braunr> no it's the same + <braunr> yes + <braunr> input will be an array, each entry denoting either memory or port + rights + <braunr> (or directly one entry for fast ipcs) + <teythoon> memory as in pointers? + <braunr> (well fast ipc when there is only one entry to avoid hitting a + table) + <braunr> pointer/size yes + <teythoon> hm, surely you want a way to avoid copying that, right? + <braunr> the only operation will be copy (i.e. unlike mach which allows + sharing) + <braunr> why ? + <braunr> copy doesn't exclude zero copy + <braunr> (zero copy being adjusting page tables with copy on write + techniques) + <teythoon> right + <teythoon> but isn't that too coarse, like in cow a whole page? + <braunr> depends on the message size + <braunr> or options provided by the caller, i don't know yet + <teythoon> oh, you are going to pack the memory anyway? + <braunr> depends on the caller + <braunr> i'm not yet sure about these details + <braunr> ideally, i'd like to avoid serialization altogether + <teythoon> wouldn't that be like cheating b/c it's the first copy? + <braunr> directly pass pointers/sizes from the sender address space, and + either really copy or use zero copy + <teythoon> right, but then you're back at the page size issue + <braunr> yes + <braunr> it's not a real issue + <braunr> the kernel must support both ways + <braunr> the minor issue is determining which way to choose + <braunr> it's not a critical issue + <braunr> my current plan is to always copy, unless the caller has + explicitely set a flag and is passing properly aligned buffers + <teythoon> u sure? I mean the caller is free to arange the stuff he intends + to send anyway he likes, how are you going to cow that then? + <teythoon> ok + <teythoon> right + <braunr> properly aligned buffers :) + <braunr> otherwise the kernel rejects the request + <teythoon> that's reasonable, yes + <braunr> in addition to being synchronous, ipc will also take a special + path in the scheduler to directly use the client scheduling context + <braunr> avoiding the sleep/wakeup overhead, and providing priority + inheritence by side effect + <teythoon> uh, but wouldn't dropping serialization create security and + reliability issues? if the receiver isn't doing a proper job sanitizing + its stuff + <braunr> why would the client not sanitize ? + <braunr> err + <braunr> server + <braunr> it has to anyway + <teythoon> sure, but a proper parser written once might be more robust, + even if it adds overhead + <teythoon> the serialization i mean + <braunr> it's just a layer + <braunr> even with high level types, you still need to sanitize + <braunr> the real downside is loosing cross architecture portability + <braunr> making the potential implementation of a single system image a lot + more restricted or difficult + <braunr> but i don't care about that much + <braunr> mach was built with this in mind though + <teythoon> it's a nice idea, but i don't believe anyone does ssi anymore + <braunr> i don't know + <teythoon> and certainly not across architectures + <braunr> there are few projects + <braunr> anyway it's irrelevant currently + <braunr> and my interface just restricts it, it doesn't prevent it + <braunr> so i consider it an acceptable compromise + <teythoon> so, does it run? what does it do? + <teythoon> it certainly is, yes + <braunr> for now, it manages memory (physical, virtual, kernel, and soon, + anonymous) + <braunr> support multiple processors with the required posix scheduling + policies + <braunr> (it uses a cute proportionally fair time sharing algorithm) + <braunr> there are locks (spin locks, mutexes, condition variables) and + lockless stuff (à la rcu) + <braunr> both x86 and x86_64 are supported + <braunr> (even pae) + <braunr> work queues + <teythoon> sounds impressive :) + <braunr> :) + <braunr> i also added basic debugging + <braunr> stack trace (including getting the symbol table) handling + <braunr> so yes, it's much much better than what i previously did + <braunr> and on the right track + <braunr> it already scales a lot better than mach for what it does + <braunr> there are generic data structures (linked list, red-black tree, + radix tree) + <braunr> the radix tree supports lockless lookups, so looking up both the + page cache and the ipc spaces is lockless) + <teythoon> that's nice :) + <braunr> there are a few things using global locks, but there are TODOs + about them + <braunr> even with that, it should be scalable enough for a start + <braunr> and improving those parts shouldn't be too difficult diff --git a/microkernel/mach/gnumach/debugging.mdwn b/microkernel/mach/gnumach/debugging.mdwn index 71e92459..7e7cfb4e 100644 --- a/microkernel/mach/gnumach/debugging.mdwn +++ b/microkernel/mach/gnumach/debugging.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2007, 2008, 2009, 2011, 2012 Free Software +[[!meta copyright="Copyright © 2007, 2008, 2009, 2011, 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -75,6 +75,9 @@ When you're [[running_a_system_in_QEMU|hurd/running/qemu]] you can directly kernel](http://www.nongnu.org/qemu/qemu-doc.html#SEC48). +## [[open_issues/debugging_gnumach_startup_qemu_gdb]] + + # Code Inside the Kernel Alternatively you can use an approach like this one: add the following code diff --git a/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn b/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn index ca52dca5..a169e92e 100644 --- a/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn +++ b/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn @@ -59,3 +59,32 @@ License|/fdl]]."]]"""]] it [[Makefile]], [[mach_print.S]], [[main.c]]. + + +## IRC, freenode, #hurd, 2013-07-01 + + <youpi> braunr: btw, we are missing the symbol in mach/Versions + <braunr> youpi: what symbol ? + <youpi> so the libc-provided RPC stub is not available + <youpi> mach_printf + <youpi> -f + <braunr> it's a system calll + <braunr> not exported + <youpi> s/RPC/system call/ + <braunr> that's expected + <youpi> libc does provide stubs for system calls too + <braunr> yes but not for this one + <youpi> I don't see why we wouldn't want to include it + <youpi> ?! it does + <braunr> it's temporary + <braunr> oh + <braunr> there must be automatic parsing during build + <youpi> sure + <braunr> nice + + <braunr> youpi: if we're going to make this system call exported by glibc, + i should change its interface first + <braunr> it was meant as a very quick-and-dirty hack and directly accesses + the caller's address space without going through a special copy-from-user + function + <braunr> not very portable diff --git a/microkernel/mach/history.mdwn b/microkernel/mach/history.mdwn index 776bb1d7..c22ea739 100644 --- a/microkernel/mach/history.mdwn +++ b/microkernel/mach/history.mdwn @@ -78,3 +78,137 @@ IRC, freenode, #hurd, 2012-08-29: https://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/About/About.html <pavlx> can't be anymore http://developer.apple.com/techpubs/macosx/Darwin/General/KernelProgramming/About/index.html + +IRC, freenode, #hurd, 2013-07-03: + + *** natsukao (~natsukao@dynamic-adsl-94-37-184-109.clienti.tiscali.it) has + joined channel #hurd + <natsukao> hi + <natsukao> on 2012-08-29: i wrote a part of messages that then were posted + on http://www.gnu.org/software/hurd/microkernel/mach/history.html + <natsukao> i am sorry to inform you that apple computer cuèertino.inc, has + moved the URL: https://ssl.apple.com/science/profiles/cornell + <natsukao> and i have not found nothing on the source code of that page, + <natsukao> i used lftp without any success + <natsukao> and then wget, nothing to do + <natsukao> i have not found a copy cache of + https://ssl.apple.com/science/profiles/cornell + <natsukao> next time we save the documents and we provide to do our + archive/s + <natsukao> so that will be always available the infos + *** natsukao (~natsukao@dynamic-adsl-94-37-184-109.clienti.tiscali.it) is + now known as pavlx + <pavlx> happy hacking !!!! + <pavlx> "paolo del bene" <W3C_Freedom@riseup.net> + <pavlx> p.s: i'll turn back as soon as possible + + <pavlx> i found the page of Darwin History, removed from apple compter + cupertino.inc + <pavlx> "Cached_ http___developer.apple.com_darwin_history.html" + <pavlx> the page http://developer.apple.com/darwin/history.html was moved + and now is available on: + <pavlx> + http://www.google.it/url?q=http://www.macmark.de/files/darwin_evolution.pdf%3FPHPSESSID%3D8b8683a81445f69d510734baa13aabfc&sa=U&ei=wMzTUb-NBIeFOOL4gNgE&ved=0CCQQFjAD&usg=AFQjCNFlLwC24nB5t14VUmedK4EmeE7Ohw + <pavlx> or simply: http://www.macmark.de/files/darwin_evolution.pdf + <pavlx> slides on: "Travel - Computer Science and Software Engineering" + <pavlx> www.csse.uwa.edu.au/~peterj/personal/PDFs/WWDC2004-6UP.pdf + <pavlx> about apple computer cupertino.inc, but there are many interesting + news + <teythoon> pavlx: uh, lot's of marketing noise from apple >,< + <pavlx> i found better material just now: + http://www.pcs.cnu.edu/~mzhang/CPSC450_550/2003CaseStudy/Mach_Presentation_DavidRamsey.ppt + <pavlx> teythoon, sorry, i turn back to sleep, see you later, paolo + W3C_Freedom@riseup.net + <pavlx> i'll charge of that page only things dedicated to GNU/HURD, but + slides are not mine, i found on internet + <teythoon> pavlx: sure, I didn't ment to offend you in any way + +IRC, freenode, #hurd, 2013-07-04: + + <pavlx> there are few problems: + <pavlx> http://www.gnu.org/software/hurd/microkernel/mach/history.html + <pavlx> on the page GrantBow wrote: Apple's Macintosh OSX (OS 10.x) is + based on Darwin. "Darwin uses a monolithic kernel based on ?FreeBSD 4.4 + and the OSF/mk Mach 3." Darwin also has a Kernel Programming Book. + <pavlx> the link to Darwin was moved, is not anymore + http://www.apple.com/macosx/technologies/darwin.html + <pavlx> then it's not FreeBSD 4.4 but BSD + <pavlx> and the link to Kernel Programming was moved is not + http://developer.apple.com/techpubs/macosx/Darwin/General/KernelProgramming/About/index.html + but + https://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/About/About.html + <pavlx> apple has moved the URL: + https://ssl.apple.com/science/profiles/cornell + <pavlx> apple has moved the URL: + http://www.apple.com/macosx/technologies/darwin.html + <pavlx> so on the website you can left few things about my old post: + <pavlx> from IRC, freenode, #hurd, 2012-08-29: needs to remove + <pavlx> http://dpaste.com/1286266/ + <pavlx> the new one will be: http://pastebin.com/yMm8tcTN + IRC, freenode, #hurd, 2013-07-04: + + <pavlx> was moved the page from apple.com about darwin kernel programming + as described on the https://www.gnu.org/software/hurd/microkernel/mach/history.html + + <pavlx> the link to Kernel Programming: + https://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/About/About.html + <pavlx> (anyway i searching with any key the things moved from apple) + <pavlx> about Darwin type http://apple.com/darwin + <pavlx> on the right side, towards the end of the website it says: Darwin + Technologies + <pavlx> click on it, or copy the URL in an other tab of your own browser, + and read: + https://developer.apple.com/library/mac/#documentation/MacOSX/Conceptual/OSX_Technology_Overview/SystemTechnology/SystemTechnology.html + <pavlx> and something is related to Darwin + <pavlx> and again : http://pastebin.com/DHkJDxy8 + # Mac OS X Server + + ... This kernel, known as Darwin, provides a stable, high-performance platform + for developing groundbreaking applications and system technologies. ... + http://www.apple.com/server/docs/MacOSX_Server_TO_300195.pdf + + # Mac OS X Server Command-Line Administration + + Page 1. Mac OS X Server Command-Line Administration For Version 10.3 + http://www.apple.com/server/docs/Command_Line.pdf + + # Press Info - Apple “Open Sources” Rendezvous + + ... Rendezvous is part of a broader open source release today from Apple at + http://developer.apple.com/darwin which includes the Darwin 6.0.1 ... + http://www.apple.com/pr/library/2002/09/25Apple-Open-Sources-Rendezvous.html + + # Press Info - Apple Releases Darwin 1.0 Open Source + + ... Apple Releases Darwin 1.0 Open Source. New ... modules. Darwin 1.0 gives + developers access to essential Mac OS X source code. ... + http://www.apple.com/pr/library/2000/04/05Apple-Releases-Darwin-1-0-Open-Source.html + + # Press Info - Apple's Mac OS X to Ship on March 24 + + ... Mac OS X is built upon an incredibly stable, open source, UNIX based + foundation called Darwin and features true memory protection, preemptive ... + http://www.apple.com/pr/library/2001/01/09Apples-Mac-OS-X-to-Ship-on-March-24.html + + # Press Info - Mac OS X “Gold Master” Released To ... + + ... Mac OS X is built upon an incredibly stable, open source, UNIX + basedfoundation called Darwin and features true memory protection ... + http://www.apple.com/pr/library/2001/03/07Mac-OS-X-Gold-Master-Released-To-Manufacturing.html + + * Press Info - Apple Announces Mac OS X “Jaguar” ... + + ... As an active member of the Open Source community, Apple has distributed + Open Directory technology through the Darwin Open Source Project. ... + http://www.apple.com/pr/library/2002/07/17Apple-Announces-Mac-OS-X-Jaguar-Server-Worlds-Easiest-to-Manage-UNIX-Based-Server-Software.html + <pavlx> and: + http://lists.apple.com/archives/darwinos-users/2005/Apr/msg00021.html + <youpi> pavlx: it's hard to follow the changes you are talking + about. Perhaps you could simply edit these wiki pages? + <pavlx> anyway i am saying to you that i found a mailing list where are + availables the sources codes of darwin ppc-801 and x86 + <pavlx> and as last thing mac os x 10.4 + <braunr> pavlx: what's all this about ? + <pavlx> i am sorry, i did changes on the wiki + <pavlx> but after the preview and after to have saved, it show again the + old chat of 2012 diff --git a/microkernel/mach/mig.mdwn b/microkernel/mach/mig.mdwn index 331b3bf4..d6340574 100644 --- a/microkernel/mach/mig.mdwn +++ b/microkernel/mach/mig.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2001, 2002, 2003, 2006, 2007, 2008, 2010 Free -Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2001, 2002, 2003, 2006, 2007, 2008, 2010, 2013 +Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -22,9 +22,9 @@ wait for a result on a newly created [[reply port|port]], decode return arguments from the reply message (*demarshalling*, or *unmarshalling*) and pass them to the client program. Similar actions are provided in the skeletons that are linked to server programs. - MIG allows very precise semantics to be specified about what the arguments are and how to be passed. +It has its problems with [[structured_data]], however. * [[Documentation]] diff --git a/microkernel/mach/mig/structured_data.mdwn b/microkernel/mach/mig/structured_data.mdwn new file mode 100644 index 00000000..1c8abe08 --- /dev/null +++ b/microkernel/mach/mig/structured_data.mdwn @@ -0,0 +1,119 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_mig]] + +# IRC, freenode, #hurd, 2013-06-25 + + <teythoon> is there a nice way to get structured data through mig that I + haven't found yet? + <teythoon> say an array of string triples + <braunr> no + <teythoon> :/ + <braunr> but you shouldn't need that + <teythoon> my use case is getting info about fs translators from init to + procfs + +[[community/gsoc/project_ideas/mtab]]. + + <teythoon> should I go for an iterator like interface instead? + <braunr> depends + <braunr> how many do you need ? + <braunr> you could go for a variable sized array too + <braunr> have a look at what already exists + <teythoon> records, maybe 10-15, depends on many fs translators are running + <braunr> a variable sized array is ok if the size isn't too big (and when i + say too big, i mean hundreds of MiB) + <braunr> an iterator is ok too if there aren't too many items + <braunr> you may want to combine both (i think that's what proc does) + <braunr> be aware that the maximum size of a message is limited to 512 MiB + <teythoon> yeah I saw the array[] of stuff stuff, but array[] of string_t + does not work, I guess b/c string_t is also an array + <teythoon> how would I send an array of variable length strings? + <braunr> i'm not sure you can + <braunr> or maybe out of line + <teythoon> somehow I expected mig to serialize arbitrary data structures, + maybe it's to old for that? + <teythoon> yeah, I read about uot of line, but that seems overkill + <braunr> it is old yes + <braunr> and not very user friendly in the end + <braunr> let me check + <teythoon> we could stuff json into mig... + <braunr> see proc_getallpids for example + <braunr> we could get rid of low level serialization altogether :p + <teythoon> hah, exactly what I was looking at + <braunr> (which is what i'll do in x15) + <braunr> type pidarray_t = array[] of pid_t; + <teythoon> but that is trivial b/c its array[] of pid_t + <braunr> and always have the server writing guide near you + <teythoon> yes + <braunr> well, make one big string and an array of lengths :p + <teythoon> thought about that and said to myself, there must be a better + way that I haven't found yet + <braunr> or one big string filled with real null-terminated c strings that + you keep parsing until you ate all input bytes + <braunr> i'm almost certain there isn't + <braunr> type string_t = c_string[1024]; /* XXX */ + <teythoon> yes + <braunr> even that isn't really variable sized + <teythoon> you think anyone would object to me putting a json encoder in + /hurd/init? it is probably better than me at serializing stuff... + <braunr> try with mig anyway + <braunr> the less dependencies we have for core stuff, the simpler it is + <braunr> but i agree, mig is painful + <teythoon> would it be too hacky if I abused the argz functions? they do + exactly what I'd need + + +## IRC, freenode, #hurd, 2013-06-26 + + <teythoon> there is https://code.google.com/p/protobuf-c/ and it has a rpc + mechanism and I believe one could plug arbitrary transports easily + <braunr> please don't think about it + <braunr> we really don't want to add another layer of serialization + <braunr> it's better to completely redesign mach ipc anyway + <braunr> and there is a project for that :p + <teythoon> ive seen x15 + <teythoon> just food for thought + <braunr> i've studied google protocol buffers + <braunr> and fyi, no, it wouldn't be easy to plug arbitrary transports on + top of mach + <braunr> there is a lot of knowledge about mach ports in mig + +[[community/gsoc/project_ideas/mtab]]. + + <teythoon> but again I face the challenge of serializing a arbitrary sized + list of arbitrary sized strings + <braunr> yes + <teythoon> list of ports is easier ;) but I think its worthwile + <teythoon> so what about abusing argz* for this? you think it's too bad a + hack? + <braunr> no since it's in glibc + <teythoon> awesome :) + <braunr> but i don't remember the details well and i'm not sure the way you + use it is safe + <teythoon> yeah, I might have got the details wrong, I hadn't had the + chance to test it ;) + + <braunr> about this dynamic size problem + <braunr> a "simple" varying size array should do + <braunr> you can easily put all your strings in there + <teythoon> seperated by 0? + <braunr> yes + <teythoon> that's exactly what the argz stuff does + <braunr> you'll get the size of the array anyway, and consume it until + there is no byte left + <braunr> good + <braunr> but be careful with this too + <braunr> since translators can be run by users, they somtimes can't be + trusted + <braunr> and even a translator running as root may behave badly + <braunr> so careful with parsing + <teythoon> noted |