diff options
Diffstat (limited to 'microkernel')
| -rw-r--r-- | microkernel/mach/deficiencies.mdwn | 844 | ||||
| -rw-r--r-- | microkernel/mach/gnumach/boot_trace.mdwn | 22 | ||||
| -rw-r--r-- | microkernel/mach/gnumach/debugging.mdwn | 23 | ||||
| -rw-r--r-- | microkernel/mach/gnumach/interface.mdwn | 38 | ||||
| -rw-r--r-- | microkernel/mach/gnumach/interface/device/time.mdwn | 88 | ||||
| -rw-r--r-- | microkernel/mach/gnumach/interface/syscall/mach_print.mdwn | 32 | ||||
| -rw-r--r-- | microkernel/mach/gnumach/ports/xen.mdwn | 208 | ||||
| -rw-r--r-- | microkernel/mach/gnumach/projects/clean_up_the_code.mdwn | 111 | ||||
| -rw-r--r-- | microkernel/mach/message/msgh_id.mdwn | 2 | ||||
| -rw-r--r-- | microkernel/mach/mig/documentation.mdwn | 16 | ||||
| -rw-r--r-- | microkernel/x15.mdwn | 24 |
11 files changed, 1368 insertions, 40 deletions
diff --git a/microkernel/mach/deficiencies.mdwn b/microkernel/mach/deficiencies.mdwn index 8f47f61f..4d261627 100644 --- a/microkernel/mach/deficiencies.mdwn +++ b/microkernel/mach/deficiencies.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013, 2014 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -812,6 +813,10 @@ In context of [[open_issues/multithreading]] and later [[open_issues/select]]. <zacts> or both? <braunr> probably netbsd drivers <zacts> and if netbsd, will it utilize rump? + +[[open_issues/user-space_device_drivers]], *External Projects*, *The Anykernel +and Rump Kernels*. + <braunr> i don't know yet <zacts> ok <braunr> device drivers and networking will arrive late @@ -2384,3 +2389,840 @@ In context of [[open_issues/multithreading]] and later [[open_issues/select]]. concurrently <braunr> (which is another contention issue when using mach-like ipc, which often do need to allocate/release virtual memory) + + +## IRC, freenode, #hurd, 2013-09-28 + + <rah> braunr: http://git.sceen.net/rbraun/x15.git/blob/HEAD:/README + <rah> "X15 is a free microkernel." + <rah> braunr: what distinguishes it from existing microkernels? + + +## IRC, freenode, #hurd, 2013-09-29 + + <braunr> rah: the next part maybe ? + <braunr> "Its purpose is to provide a foundation for a Hurd-like operating + system." + <rah> braunr: there are already microkernels that canbe used as the + foundatin for Hurd=like operating systems; why are you creating another + one? + <rah> braunr: what distinguishes your microkernel from existing + microkernels? + <tschwinge> rah: + http://www.gnu.org/software/hurd/microkernel/mach/deficiencies.html + <braunr> rah: it's better :) + <braunr> rah: and please, cite one suitable kernel for the hurd + <rah> tschwinge: those are deficiencies in Mach; I'm asking about x15 + <rah> braunr: in what way is it better exactly? + <braunr> rah: more performant, more scalable + <rah> braunr: how? + <braunr> better algorithms, better interfaces + <braunr> for example, it supports smp + <rah> ah + <rah> it supports SMP + <rah> ok + <rah> that's one thing + <braunr> it implements lockless synchronization à la rcu + <rah> are there any others? + <rah> ok + <rah> lockless sync + <rah> anything else? + <braunr> it can scale from 4MB of physical memory up to several hundreds + GiB + <braunr> ipc is completely different, leading to simpler code, less data + involved, faster context switches + <braunr> (although there is no code for that yet) + <rah> how can it support larger memory while other microkernels can't? + <rah> how is the ipc "different"? + <braunr> others can + <braunr> gnumach doesn't + <rah> how can it support larger memory while gnumach can't? + <azeem_> because it's not the same code base? + <braunr> gnumach doesn't support temporary kernel mapping + <rah> ok, so x15 supports temporary kernel mapping + <braunr> not exactly + <braunr> virtual memory is completely different + <rah> how so? + <braunr> gnumach does the same as linux, physical memory is mapped in + kernel space + <braunr> so you can't have more physical memory than you have kernel space + <braunr> which is why gnumach can't handle more than 1.8G right now + <braunr> it's a 2/2 split + <braunr> in x15, the kernel maps what it needs + <braunr> and can map it from anywhere in physical memory + <tschwinge> rah: I think basically all this has already been discussed + before and captured on that page? + <braunr> it already supports i386/pae/amd64 + <rah> I see + <braunr> the drawback is that it needs to update kernel page tables more + often + <braunr> on linux, a small part of the kernel space is reserved for + temporary mappings, which need page table updates too + <braunr> but most allocations don't use that + <braunr> it's complicated + <braunr> also, i plan to make virtual memory operations completely + concurrent on x15, similar to what is described in radixvm + <rah> ok + <braunr> which means mapping operations on non overlapping regions won't be + serialized + <braunr> a big advantage for microkernels which base their messaging + optimizations on mapping + <braunr> so simply put, better performance because of simpler ipc and data + structures, and better scalability because of improved data structure + algorithms and concurrency + <rah> tschwinge: yes but that page is no use to someone who wants a summary + of what distinguishes x15 + <braunr> x15 is still far from complete, which is why i don't advertise it + other than here + <rah> "release early, release often"? + <braunr> give it a few more years :p + <braunr> release what ? + <braunr> something that doesn't work ? + <rah> software + <rah> yes + <braunr> this release early practice applies to maintenance + <rah> release something that doesn't work so that others can help make it + work + <braunr> not big developments + <braunr> i don't want that for now + <braunr> i have a specific idea of what i want, and both explaining and + defending it would take time, better spent in development itself + <braunr> just wait for a first prototype + <braunr> and then you'll see if you want to help or not + * rah does not count himself as one of the "others" who might help make it + work + <braunr> one big difference with other microkernels is that x15 is + specifically intended to run a unix like system + <braunr> a hurd like system providing a psoix interface more accurately + <braunr> and efficiently + <braunr> so for example, while many microkernels provide only sync ipc, x15 + provides both sync ipc and signals + <braunr> and then, there are a lot of small optimizations, like port names + which will transparently identify as file descriptors + <braunr> light reference counting + <braunr> a restriction on ipc that only allows reliable transfers across + network to machines with same arch and endianness + <braunr> etc.. + + +## Summary + +Created on 2013-09-29 by wiki user *BobHam*, *rah* on IRC. + +> The x15 microkernel is under development by Richard Braun. Overall, x15 is intended to provide better performance because of simpler IPC and data structures and better scalability because of improved data structure algorithms and concurrency. +> +> The following specific features are intended to distinguish x15 from other microkernels. However, it should be noted that the microkernel is under heavy development and so the list may (and almost certainly will) change. +> +> * SMP support +> * Lockless synchronisation à la RCU +> * Support for large amounts of physical memory. GNU Mach does the same as Linux, physical memory is mapped in kernel space so you can't have more physical memory than you have kernel space which is why GNU Mach can't handle more than 1.8G right now, it's a 2/2 split. In x15, the kernel maps what it needs and can map it from anywhere in physical memory the drawback is that it needs to update kernel page tables more often. +> * Virtual memory operations are planned to be completely concurrent on x15, similar to what is described in radixvm +> * Intended to efficiently run a Hurd-like system providing a POSIX interface +> * Providing both synchronisation IPC and signals, as opposed to just synchronisation IPC +> * Port names which will transparently identify as file descriptors +> * Light reference counting +> * A restriction on IPC that only allows reliable transfers across network to machines with same arch and endianness +> * etc. + + +## IRC, freenode, #hurd, 2013-10-12 + + <zacts> braunr: are you still working on x15/propel? + * zacts checks the git logs + <braunr> zacts: taking a break for now, will be back on it when i have a + clearer view of the new vm system + + +## IRC, freenode, #hurd, 2013-10-15 + + <gnufreex> braunr, few questions about x15. I was reading IRC logs on hurd + site, and in the latest part, you say (or I misunderstood) that x15 is + now hybrid kernel. So what made you change design... or did you? + <braunr> gnufreex: i always intended to go for a hybrid + + +## IRC, freenode, #hurd, 2013-10-19 + + <zacts> braunr: when do you plan to start on x15/propel again? + <braunr> zacts: after i'm done with thread destruction on the hurd + +[[open_issues/libpthread/t/fix_have_kernel_resources]]. + + <zacts> and do you plan to actually run hurd on top of x15, or are you + still going to reimplement hurd as propel? + <braunr> and no, i don't intend to run the hurd on top of x15 + + +## IRC, freenode, #hurd, 2013-10-24 + + <neal> braunr: What is your Mach replacement doing? + <braunr> "what" ? :) + <braunr> you mean how i guess + <neal> Sure. + <braunr> well it's not a mach replacement any more + <braunr> and for now it's stalled while i'm working on the hurd + <neal> that could be positive :) + <braunr> it's in good shape + <neal> how did it diverge? + <braunr> sync ipc, with unix-like signals + <braunr> and qnx-like bare data messages + <neal> hmm, like okl5? + <braunr> (with scatter gather) + <neal> okl4 + <braunr> yes + <braunr> btw, if you can find a document that explains this property of + okl4, i'm interested, since i can't find it again on my own :/ + <braunr> basically, x15 has a much lighter ipc interface + <neal> capabilities? + <braunr> mach ports are mostly retained + <braunr> but reference counting will be simplified + <neal> hmm + <neal> I don't like the reference counting part + <braunr> port names will be plain integers, to directly be usable as file + descriptors and avoid a useless translation layer + <braunr> (same as in qnx) + <neal> this sounds like future tense + <braunr> there is no ipc code yet + <neal> so I guess this stuff is not implemented + <neal> ok. + <braunr> next step is virtual memory + <braunr> and i'm taking my time because i want it to be a killer feature + <neal> so if you don't IPC and you don't have VM, what do you have? :) + <braunr> i have multiprocessor multithreading + <neal> I see. + <braunr> mutexes, condition variables, rcu-like lockless synchronization, + work queues + <braunr> basic bsd-like virtual memory + <braunr> which i want to rework + <neal> I ignored all of that in Viengoos :) + <braunr> and since ipc will still depend on virtual memory for zero-copy, i + want the vm system to be right + <braunr> well, i'm more interested in the implementation than the + architecture + <braunr> for example, i have unpublished code that features a lockless + radix tree for vm_object lookups + <braunr> that's quite new for a microkernel based system, but the ipc + interface itself is very similar to what already exists + <braunr> your half-sync ipc are original :) + <neal> I'm considering getting back in the OS game. + <braunr> oh + <neal> But, I'm not going to write a kernel this time. + <braunr> did anyone here consider starting a company for such things, like + genode did ? + <neal> I was considering using genode as a base. + <braunr> neal: why genode ? + <neal> I want to build a secure system. + <neal> I think the best way to do that is using capabilities. + <neal> Genode runs on Fiasco.OC, for instance + <neal> and it provides a lot of infrastructure + <braunr> neal: why not l4re for example ? + <braunr> neal: how important is the ability to revoke capabilities ? + +In the discussion on [[community/gsoc/project_ideas/object_lookups]], *IRC, +freenode, #hurd, 2013-10-24*: + + <teythoon> and, with some effort, getting rid of the hash table lookup by + letting the kernel provide the address of the object (iirc neil knew the + proper term for that) + <braunr> teythoon: that is a big interface change + <teythoon> how so + <braunr> optimizing libihash and libpthread should already be a good start + <braunr> well how do you intend to add this information ? + <braunr> ok, "big" is overstatement, but still, it's a low level interface + change that would probably break a lot of things + <teythoon> store a pointer in the port structure in gnumach, make that + accessible somehow + <braunr> yes but how ? + <teythoon> interesting question indeed + <braunr> my plan for x15 is to make this "label" part of received messages + <braunr> which means you need to change the format of messages + <braunr> that is what i call a big change + + +### IRC, freenode, #hurd, 2013-10-31 + + <antrik> neal: you mentioned you want to use Genode as a base... what + exactly would you want to build on top of it, different than what the + Genode folks are doing? + +[[Genode]]. + + <neal> antrik: I want to build a secure operating system. + <neal> antrik: One focused on user security. + + <neal> braunr: You mean revoke individual send rights? + <neal> braunr: Or, what do you mean? + <neal> Or do you mean the ability to receive anotification on revocation? + <braunr> neal: yes, revoking individual send rights + <neal> I don't think it is needed in practice. + <braunr> neal: ok + <neal> But, you need a membrane object + <neal> Here's the idea: + <braunr> like a peropen ? + <neal> you have say a file server + <neal> and a proxy + <neal> a process only talks to the file server via the proxy + <neal> for the proxy to revoke access to the file object it gave out, it + needs to either use your revoke + <neal> interpose on all ipcs (which is expensive) + <neal> or use a proxy object/membrane + <neal> which basically forwards messages to the underlying object + <braunr> isn't that also interposing ? + <neal> of course + <neal> but if it is done in the kernel, it is fast + <braunr> ah in the kernel + <neal> you just walk a linked list + <braunr> what's the difference with a peropen object ? + <neal> That's another option + <neal> you use a peropen and then provide a call to force the per-open to + be closed + <neal> so the proxy now invokes the server + <neal> the issue here is that the proxy has to trust the server + <braunr> yes + <braunr> how can you not trust servers ? + <neal> that is, if the intent is to prevent further communication between + the server and the process, the server may ignore the request + <neal> in this case, you probably trust the server + <braunr> hum + <neal> but it could be that you have two processes communicating + <braunr> if the intent is to prevent communication, doesn't the client just + need to humm not communicate ? :) + <neal> the point is that the two processes are colluding + <braunr> what are these two processes ? + <neal> I'm not sure this case is of practical relevance + <braunr> ok + <neal> https://www.cs.cornell.edu/courses/cs513/2002sp/L10.html + <braunr> thanks + + +### IRC, freenode, #hurd, 2013-11-14 + + <antrik> neal: hm... I was under the impression that the Genode themselves + are also interested in user security... what is missing from their + version that you want to add? + <antrik> err... the Genode folks + <neal> antrik: I'm missing some context + <antrik> neal: a while back you said that you want to build a secure system + on top of Genode + <neal> yes + <neal> the fact that they are doing what I want is great + <neal> but there is more to a secure system than an operating system + <antrik> ah, so it's about applications+ + <antrik> ? + <neal> yes, that is part of it + <neal> it's also about secure messaging + <neal> and hiding "meta-data" + <braunr> i'm still wondering how you envision the powerbox + <neal> when a program wants the user to select a file, it makes an upcall + to the power box application + <antrik> braunr: you can probably find some paper from Shapiro ;-) + <braunr> well, sure, it looks easy + <braunr> but is there always a power box application ? + <braunr> is there always a guarantee there won't be recursive calls made by + that application ? + <braunr> how does it integrate with the various interfaces a system can + have ? + <neal> there is always a power box application + <neal> I don't know what you mean by recursive calls + <braunr> aer techniques such as remembering for some time like sudo does + applicable to a powerbox application ? + <neal> if you mean many calls, then it is possible to rate limit it + <braunr> well, the powerbox will use messaging itself + <braunr> is it always privileged ? + <braunr> privileged enough + <neal> it is privileged such like the X11 display manager is privileged and + can see all of the video content + <braunr> what else other than accessing a file would it be used for ? + <braunr> one case i think of is accessing the address space of another + application, in debuggers + <braunr> 14:56 < neal> there is always a power box application + <braunr> what would it be when logging on a terminal ? + <antrik> braunr: when running pure command line tools, you can already pass + the authority as part of the command line. however, I'm wondering whether + it really makes sense to apply this to traditional shell tools... + <braunr> that's one of my concerns + <braunr> when does it really make sense ? + <antrik> for interactive use (opening new files from within a running + program), I don't think it can be accomplished in a pure terminal + interaction model... + <braunr> and you say "you pass the authority" + <antrik> braunr: it makes sense for interactive applications + <braunr> i thought the point of the powerbox is precisely not to do that + <antrik> no, it's still possible and often reasonable to pass some initial + authority on startup. the powerbox is only necessary when further access + needs to be provided at runtime + <braunr> ok + <neal> the power box enable dynamic delegation of authority, as antrik said + <braunr> ok + <braunr> but how practical is it ? + <neal> applications whose required authority is known apriori and + max(required authority) is approximately min(required authority) can be + handled with static policies + <braunr> don't application sometimes need a lot of additional authority ? + <braunr> ok + <antrik> actally, thinking about it, a powerbox should also be possible on + a simple terminal, if we make sure the application doesn't get full + control of the terminal, but rather allow the powerbox to temporarily + take over input/output without the application being able to + interpose... so not quite a traditional UNIX terminal, but close enough + I'd say + <braunr> the terminal itself maybe ? + <antrik> hm... that would avoid having to implement a more generic + multiplexing approach -- but it would mix things that are normally quite + orthogonal... + <antrik> BTW, I personally believe terminals need to get smarter anyways + :-) + <braunr> ok + <antrik> the traditional fully linear dialog has some nice properties; but + it is also pretty limited, leading to usability problems soon. I have + some vague ideas for an approach that still looks mostly like a linear + dialog, but is actually more structured + + +## IRC, freenode, #hurd, 2013-11-04 + + <braunr> yes the learning curve [of the Hurd] is too hard + <braunr> that's an entry barrier + <braunr> this is why i use well known posix-like (or other well + established) apis in x15 + <braunr> also why i intend to make port rights blend into file descriptors + <teythoon> right + <braunr> well + <braunr> the real reason is efficiency + <braunr> but matching existing practices is very good too + + +## IRC, freenode, #hurd, 2013-11-08 + + <gnufreex> braunr, how is work on x-15 progressing? Is there some site to + check what is new? + <braunr> gnufreex: stalled for 2 months + <braunr> i'm working on the hurd for now, will get back to it later + <braunr> no site + <braunr> well + <gnufreex> so, you hit some design problem, or what? I mean why stalled + <braunr> http://git.sceen.net/rbraun/x15.git/ :p + <gnufreex> Thanks + <braunr> something like that yes + <braunr> i came across + http://darnassus.sceen.net/~rbraun/radixvm_scalable_address_spaces_for_multithreaded_applications.pdf + <gnufreex> I read that, I think I found it on Hurd site. + <braunr> and since x15 aims at being performant and scalable, it seems like + a major feature to bring in + <braunr> but it's not simple to integrate + <gnufreex> So you want to add that? + <braunr> gnufreex: yes + <gnufreex> branur, but what are the problems? + <braunr> ? + <braunr> ah + <braunr> you really want to know ? :) + <gnufreex> Well... yeah + <braunr> you need to know both x15 and radixvm for that + <braunr> for one, refcache, as described in the radixvm paper, doesn't seem + scalable + <braunr> it is in practice in their experiments, but only because they + didn't push some parameters too high + <braunr> so i need to rethink it + <gnufreex> I don't know x15... but I read radixvm paper + <braunr> next, the bsd-like vm used by x15 uses a red-black tree to store + memory areas, which doesn't need external storage + <braunr> radixvm as implemented in xv6 is only used for user processes, not + the kernel + <braunr> which means the kernel allocator is a separate implementation, as + it's done in linux + <braunr> x15 uses the same implementation for both the kernel and user maps + <braunr> which results in a recursion problem + <braunr> because a radix tree uses external nodes that must be dynamically + allocated + <gnufreex> so you would pretty much need to rewrite x15 + <braunr> no + <braunr> just vm/ + <braunr> and $arch/pmap + <braunr> and yes, pmap needs to handle per-core page tables + <braunr> something i wanted to add already but couldn't because of similar + recursion problems + <gnufreex> Yeah, vm system... but what else did you do with x15... it is at + early stage... + <braunr> multithreading + <gnufreex> That doesn't need to be rewriten? + <braunr> no + <gnufreex> Ok... good. + <braunr> physical memory allocation neither + <braunr> only virtual memory + <gnufreex> is x15 in runable state? I mean in virtual machine? + <braunr> you can start it + <braunr> but you won't go far :) + <gnufreex> What do you use as development platform? + <braunr> it basically detects memory and processors, starts idle, migration + and worker threads, and leaves + <gnufreex> Is it compilable on fedora 19 + <braunr> probably + <braunr> i use debian stable + <braunr> and unstable on the hurd + <gnufreex> ok, I will probably try it in KVM... + <braunr> better do it on real hardware too in case you find a bug + <gnufreex> I cant make new partition now... it seems my hard drive is + dying. When I get a new one I will try on real harware. + <braunr> you don't need a new partition + <braunr> the reason radixvm is important is twofold + <braunr> 1/ ipc will probably make use of the core vm operations used by + mmap and munmap + <braunr> 2/ no other system currently provides scalable + mmap/munmap/mprotect + <gnufreex> Yes, that would make x15 pretty special... + <gnufreex> But I read somewhere that you wanted to implement RCU during + summer + <gnufreex> Did you do that? + + +## IRC, freenode, #hurd, 2013-11-12 + + <braunr> neal: about secure operating systems + <braunr> i assume you consider clients providing their own memory a strong + requirement for that, right ? + <neal> no + <neal> I'm less interested in availability + <neal> or performance guarantees + <braunr> ok + <braunr> but + <braunr> i thought it was a requirement to avoid denial of service + <neal> of course + <braunr> then why don't you consider it required ? + <neal> I want something working in a reasonable amount of time :) + <braunr> agreed + <neal> more seriously: + <neal> my primary requirement is that a program cannot access information + that the user has not authorized it to access + <braunr> ok + <neal> the requirement that you are suggesting is that a program be able to + access information that the user has authorized it to access + <neal> this is availability + <braunr> i'm not following + <braunr> what's the difference ? + <neal> assume we have two programs: A and B + <neal> on Unix, if they run under the same uid, they access access each + other files + <neal> I want to fix this + <braunr> ok, that's not explicit authorization + <braunr> but is that what you mean ? + <neal> Now, assuming that A cannot access B's data and vice versa + <neal> we have an availability problem + <neal> A could prevent B from accessing its data + <neal> via a DoS attach + <neal> I'm not going to try to fix that. + <braunr> ok + <braunr> and how do you intend to allow A to access B's data ? + <braunr> i guess the powerbox mentioned in the critique + <braunr> but do you have a more precise description about something + practical to use ? + + +## IRC, freenode, #hurd, 2013-11-14 + +In context of [[hurd/libports]], *Open Issues*, *IRC, freenode, #hurd, +2013-11-14*. + + <braunr> fyi, x15 will not provide port renaming + <braunr> teythoon: also, i'm considering enforcing port names to be as + close as possible to 0 when being allocated as part of the interface + <braunr> what do you think about that ? + <teythoon> braunr: that's probably wise, yes + <teythoon> you could hand out receive ports close to 0 and send ports close + to ~0 + <braunr> teythoon: what for ? + <teythoon> well, if one stores only one kind in an array, it won't waste as + much space + <braunr> this also means you need to separate receive from send rights in + the interface + <braunr> so that you know where to look for them + <braunr> i'm not sure it's worth the effort + <braunr> using the same code for them both looks more efficient + <braunr> the right lookup code is probably one of the hottest path in the + system + <teythoon> right + <neal> one of the nice things about not reusing port names is that it helps + catch bugs + <neal> you don't want to accidently send a message to the wrong recipient + <braunr> how could you, if the same name at different times denotes + different rights ? + <neal> you forget to clean up something + <braunr> if you don't clean, how could you get the same name for a right + you didn't release ? + <neal> that's not hard to do :) + <neal> ah, you cleaned up the port right but not the name + <braunr> ah ok + <neal> destroy the port and forget that a thread is still working on a + response + <neal> the data structure says use the port at index X + <neal> X is reallocated in the mean time + <teythoon> excuse my ignorance, but gnumach *is* reusing port names, isn't + it? + <braunr> that policy is why i'm not sure i want to enforce allocation + policy in the interface :/ + <neal> This is not about a security property of the system + <neal> this is about failing fast + <neal> you want to fail as close to the source of the problem as possible + <braunr> we could make the kernel use different allocation policies for + names, to catch bugs, yes + <neal> make the index X valid again and you've potentially masked the bug + <teythoon> braunr: if you were to merge your radix tree implementation into + gnumach and replace the splay tree with it, would that make using renamed + ports fast enough so we can just rename all receive ports doing away with + the extra lookup like mach-defpager does ? + <braunr> i don't think so + <braunr> the radix tree code is able to compress its size when keys are + close to 0 + <braunr> using addresses would add 1, 2, maybe 3 levels of internal nodes + <braunr> for every right + <braunr> we could use a true integer hash table for that though + <braunr> hm no, hurd packages crash ... :/ + <teythoon> but malloc allocates stuff in a contigious space, so the + pointers should be similar in the most significant bits + <braunr> if you use malloc, yes + <teythoon> sure + <teythoon> but that'd make the radix tree representation compact, no? + <braunr> it could + <braunr> the current code only compresses near 0 + <teythoon> oh + <braunr> better compression could be implemented though + + +## IRC, freenode, #hurd, 2013-11-21 + + <teythoon> have you seen liburcu ? + <braunr> a bit, yes + <teythoon> it might be worth investigating to use it in some servers + <braunr> it is + <teythoon> the proc server comes to mind + <braunr> personally, i think all hurd servers should use rcu + <braunr> libports should use rcu + <teythoon> yes + <braunr> lockless synchronization should be a major feature of x15/propel + <braunr> present even during message passing + + +## IRC, freenode, #hurd, 2013-12-09 + + <braunr> improving our page cache with arc would be great + <braunr> it's on the todo list for x15 :> + <braunr> not sure you referred to virtual memory management though + <braunr> (actually, it's CAR, not ARC that is planned for x15) + + +## IRC, freenode, #hurd, 2013-12-30 + + <braunr> zacts: http://darnassus.sceen.net/~rbraun/x15/qemu_x15.sh + + +## IRC, freenode, #hurd, 2014-01-03 + + <braunr> oh, btw, i've started working on x15 again :> + <teythoon> saw that :) + <braunr> first item on the list: per-cpu page tables + <braunr> the magic that will make ipc extremely scalable :) + <teythoon> i'm worried about your approach tbh + <braunr> too much overhead ? + <teythoon> not on any technical level + <teythoon> but haven + <braunr> ? + <teythoon> 't there been enough reimplementation efforts that got nowhere ? + <braunr> oh that + <teythoon> ^^ + <braunr> well, i have personal constraints and frustrations with the + existing code, and my goal isn't to actually produce anything serious + until it actually gets there + <braunr> which, yes, it might not + <braunr> really, i'm doing it for fun + <teythoon> well sure + <teythoon> that's a damn good reason ;) + <braunr> and if it ever reaches a state where it can actually be used to + run stuff, i would be very happy + <braunr> and considering how it's done, i'm pretty sure things could be + built a lot faster on such a system + <teythoon> but you need to reimplement all the userspace servers as well, + and the libc stuff + <braunr> yes + <teythoon> do you plan to reimplement this from scratch or do you have + plans to 'bootstrap' propel from hurd ? + <braunr> from scratch + <teythoon> well... i'm not sure that this is feasible or even a good + idea. that's what i meant in a nutshell i guess. + <braunr> i'm familiar with that criticism + <braunr> and you may be right + <braunr> this is also why i keep working on the hurd at the same time + <teythoon> we could also talk about making hurd more easily portable + <braunr> portable with regard to what ? + <teythoon> evolving hurd and mach to the point where it might be feasible + to port hurd to another ukernel + <braunr> not so easy + <teythoon> i know + <braunr> i'm not even sure i would want that + <braunr> well, since the hurd isn't optimized at all, why not + <teythoon> why would it neccessarily hinder optimization ? + <braunr> because in practice, it's rare for a microkernel to provide all + the features the hurd would require to run really well + <braunr> the most severe issue being that they either provide asynchronous + ipc, used for signals, or only synchronous ipc, making signal and other + event-driven code hard to emulate (usually requiring separate threads) + + +## IRC, freenode, #hurd, 2014-01-20 + +[[open_issues/translate_fd_or_port_to_file_name]]: + + <teythoon> i wonder if it would not be best to add a description to mach + tasks + <braunr> i think it would + <teythoon> to aid fixing these kind of issues + <braunr> in x15, i actually add descriptions (names) to all kernel objects + <teythoon> that's probably a good idea, yes + <braunr> well, not all, but many + + <braunr> i'd like to push x15 this year + <braunr> it currently is the only design of a truely scalable microkernel + that i know of + <azeem_> push how? + <braunr> spend time on it + <azeem_> k + <azeem_> do you think it will make sense to solicit outside contributions + at one point? + <braunr> yes + <braunr> the roadmap is vm system -> ipc system -> userspace (including RPC + handling) + <braunr> once we can actually do things in userspace, the priority will be + getting a shell with glibc + <braunr> people will be able to help "easily" at that point + <azeem_> just wondering, apart from scalability, did you write it for + performance, for hackability, or something else? + <braunr> it's basically the hurd architecture, including improvements from + the critique, with performance and scalability in mind + <azeem_> ok + <braunr> the main improvements i think of currently are resource + containers, lexical .. resolution, and lists of trusted users with which + to communicate + <braunr> it's strongly oriented for posix compatibility though + <teythoon> sounds nice, i like it already ;) + <azeem_> is it compatible with Mach to some degree? + <braunr> so things like running without an identity will be forbidden in + the default system personality + <braunr> no, not compatible with mach at all + <azeem_> this sounds like it is doing more than Mach did + <azeem_> braunr: ah, ok + <braunr> it's not "x15mach" any more :) + <azeem_> right, I missed out on that + + +### IRC, freenode, #hurd, 2014-01-21 + + <braunr> i also don't write anything that would prevent real-time + <teythoon> b/c that's a potential market for such an operating system ? + <braunr> yes + <teythoon> well, i can't say i don't like the sound of that ;) + <braunr> the ipc interface should be close to that of qnx + + +## IRC, freenode, #hurd, 2014-02-23 + + <cluck> braunr: have you looked at genode? + <cluck> braunr: i sometimes wonder how hard it'd be to port hurd atop it + because i find some similarities with what l4/fiasco/viengos provided + <braunr> cluck: i have, but genode seems a bit too far from posix for our + tastes + <cluck> (and yes, i realize we'd be getting farther from the hw) + <braunr> ah you really mean running the hurd on top of it + <braunr> i personally don't like the idea + <cluck> braunr: well, true, but their noux implementation proves it's not a + dealbreaker + <cluck> braunr: at least initially that'd be the best implementation + approach, no? as time went on integrating hurd servers more tightly at a + lower level makes sense but doing so from the get go would be foolhardy + <cluck> braunr: or am i missing something obvious? + <braunr> cluck: why would it be ? + <cluck> braunr: going by what happened with l4 it's too much code to port + and optimize at once + <braunr> cluck: i don't think it is + <braunr> cluck: problems with l4 didn't have much to do with "too much + code" + <cluck> braunr: i won't debate that, you have more experience than me with + hurd code. anyway that's how i'd go about it, first get it all running + then get it running fast. breakage is bad + <braunr> and you think moving from something like linux or genode to an + implementation closer to hardware won't break things ? + <cluck> braunr: yes, i read the paper, obvious unexpected shortcomings but + even had them not been there the paradigms are too different and creating + proper mappings from one model to the other would at least be time + consuming + <braunr> ye + <braunr> yes + <braunr> i'm convinved the simple approach of a small microkernel with the + proper interfacen along with the corresponding sysdeps layer in glibc + would be enough to get a small hurd like system quickly + <braunr> experience with other systems shows how to directly optimize a lot + of things from the start, without much effort + + <cluck> braunr: sorry. back to our talk, i mentioned genode because of the + nice features it has that'd be useful on hurd + <braunr> cluck: which ones do you refer to ? + <cluck> braunr: the security model is the biggest one + <braunr> how does it differ from the hurd, except for revocation ? + <cluck> braunr: then there's the ease of portability + <braunr> ? + <cluck> braunr: it's more strict + <braunr> how would that help us ? + <cluck> braunr: if hurd was running atop it we'd get extra platforms + supported almost for free whenever they did (since we'd be using the same + primitives) + <braunr> why not choose the underlying microkernel directly ? + <cluck> call me crazy but i believe code reuse is a good thing, i see + little point in duplicating existing code just because you can + <braunr> what part of genode should be reused then ? + <cluck> that's what got me thinking about genode in the first place, + ideologically they share a lot (if not most) of hurd's goals and code + wise they feel close enough to make a merge of sorts not seem crazy talk, + thus my asking if i'm missing something obvious + <braunr> i think the design is incompatible with our goals of posix + compatibility + <cluck> braunr: oh, ok. + <cluck> braunr: i was assuming that wasn't an issue, as i mentioned before + they have noux already and if hurd's servers got ported they'd provide + whatever else that was missing + <braunr> noux looks like a unix server for binary compatibility + <braunr> i'm not sure it is but that's what the description makes me think + <braunr> and if it really, then it's no different than running linux on top + of an hypervisor + <braunr> ok it's not for binary compatibility but it definitely is a + (partial) unix server + <braunr> i much prefer the way the hurd is posix compliant without any + additional layer for compatibility or virtualization + <cluck> braunr: noux is a runtime, as i understand it there's no binary + compatibility just source (ie library/api calls) + <braunr> yes i corrected that just now + <cluck> sorry, i'm having lag issues + <braunr> no worries + <cluck> braunr: anyway, how's x15 coming along? still far from being a + practical replacement? + <braunr> yes .. :( + <braunr> and it's not a replacement + <cluck> (for mach) + <braunr> no + <cluck> huh? + <braunr> it's not a replacement for the hurd + <braunr> err, for mach + <cluck> braunr: i thought you were writing it to be compatible with mach's + interfaces + <braunr> no + <braunr> it used to be that way + <braunr> but no + <cluck> braunr: what changed? + <braunr> mach ipc is too ccmplicated + <braunr> complicated* + <braunr> its supposed benefit (of allowing the creation of computer + clusters for single system images) are outdated and not very interesting + <braunr> it's error prone + <braunr> and it incurrs more overhead than it should + <cluck> no arguing there + <cluck> braunr: are you still targeting being able to run hurd atop x15 or + is it just your pet project now? + <braunr> i don't intend the hurd to run on top of it + <braunr> the reason it's a rewrite is to fix a whole bunch of major issues + in one go diff --git a/microkernel/mach/gnumach/boot_trace.mdwn b/microkernel/mach/gnumach/boot_trace.mdwn index 7b729c23..ea999a9b 100644 --- a/microkernel/mach/gnumach/boot_trace.mdwn +++ b/microkernel/mach/gnumach/boot_trace.mdwn @@ -227,3 +227,25 @@ License|/fdl]]."]]"""]] >> vm\_pageout >> Does not return. + + +# IRC, freenode, #hurd, 2013-10-07 + + <cureOS> look, where should i dig or where from should i start from, if i + have desire to know how the kernel was written from baremetal? Can it be + ever done nowadays? + <youpi> cureOS: the boot entry of the kernel is i386/i386at/boothdr.S , + boot_entry + <youpi> that's what grub jumps to + <youpi> then that jumps to c_boot_entry + <youpi> and everything else is C + <cureOS> grub loads it somehow. how does it prepare cpu and memoty, cpu + cache control if any... segments for stack.. + <youpi> see the grub documentation + <youpi> basically it's all flat linear space + <cureOS> does kernel transform it after that? + <youpi> see the ldt/gdt initialization + <youpi> from i386at_init and children + <youpi> nothing much fancy, a kernel cs/ds, and user cs/ds + <braunr> and paging, naturally + <youpi> sure diff --git a/microkernel/mach/gnumach/debugging.mdwn b/microkernel/mach/gnumach/debugging.mdwn index 7e7cfb4e..cf02471d 100644 --- a/microkernel/mach/gnumach/debugging.mdwn +++ b/microkernel/mach/gnumach/debugging.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2007, 2008, 2009, 2011, 2012, 2013 Free Software -Foundation, Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2009, 2011, 2012, 2013, 2014 Free +Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -135,7 +135,7 @@ update its display buffer anymore under certain conditions. # Halting the CPU and Examining Registers -IRC, freenode, #hurd, 2011-07-14: +## IRC, freenode, #hurd, 2011-07-14 <braunr> one ugly trick i use when printf isn't available is to halt the cpu @@ -146,7 +146,22 @@ IRC, freenode, #hurd, 2011-07-14: # Serial Console -IRC, freenode, #hurd, 2011-11-13: +## IRC, freenode, #hurd, 2011-11-13 <youpi> use console=com0 <youpi> to activate the console on the first serial port + + +# `ud2` instruction + +## IRC, freenode, #hurd, 2013-10-31 + + [master-x86_64] + <phcoder> GNU Mach 1.3.99 + <phcoder> Running on xen-3.0-x86_64. + <phcoder> AT386 boot: physical memory from 0x0 to 0x40000000 + <youpi> \o/ + <phcoder> well when loaded through pvgrub2 i hangs without any message + <phcoder> any pointers on debugging? + <youpi> I usually put the ud2 instruction along the path to see where it + crashes diff --git a/microkernel/mach/gnumach/interface.mdwn b/microkernel/mach/gnumach/interface.mdwn index b2451887..bfcb9ea6 100644 --- a/microkernel/mach/gnumach/interface.mdwn +++ b/microkernel/mach/gnumach/interface.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2013, 2014 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -14,3 +14,39 @@ License|/fdl]]."]]"""]] [[!map pages="microkernel/mach/gnumach/interface/*" show=title]] + + +# Proposed + +## Futex + +### Patches on bug-hurd + +### IRC, freenode, #hurd, 2014-01-20 + + <teythoon> braunr: i meant to ask about the futex stuff + <teythoon> what do you mean when you talk about the interface ? + <teythoon> i imagined something like this + <braunr> more precisely ? + <teythoon> for shared futexes, do not use a pointer, but a right + referencing a memory object, and the offset ? + <braunr> why not a pointer ? + <braunr> well, an address + <teythoon> that's how it's currently done, right ? + <teythoon> in marins prototype i mean + <braunr> yes + <braunr> it's a task/address pair + <teythoon> so the only concern is the access in the kernel ? + <braunr> hum not only no + <braunr> or yes, but there are multiple ways to access + <teythoon> i meant to say this, for me, it would be very helpful if you + were more explicit when you describe how you picture something + <braunr> i thought i did it clearly + <braunr> i even agree with what roland describes + <braunr> which looks even clearer + <braunr> also, i said i didn't picture it completely + <braunr> :) + <braunr> i didn't want to work on that, only review + <teythoon> sure, but what might be clear to you is most likely not that + clear to me + <braunr> ok diff --git a/microkernel/mach/gnumach/interface/device/time.mdwn b/microkernel/mach/gnumach/interface/device/time.mdwn index d1e9a488..18681f21 100644 --- a/microkernel/mach/gnumach/interface/device/time.mdwn +++ b/microkernel/mach/gnumach/interface/device/time.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2013, 2014 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -16,3 +16,89 @@ Typically available as `/dev/time`, [[hurd/translator/storeio]]. Using that, [[hurd/libshouldbeinlibc]]'s `<maptime.h>` provides `maptime_map` and `maptime_read`, see the [[hurd/reference_manual]]. Candidate for replacement with [[open_issues/vDSO]] code? + + +# IRC, freenode, #hurd, 2013-11-20 + + <teythoon> braunr: about the mach device interface, if i open a device, and + then create a memory mapping using device_map, does that increment the + open count of the device ? + <teythoon> can i call device_close w/o destroying the mapping directly + after mapping it ? + <antrik> teythoon: I have a vague recollection that the mapping (or more + precisely, the memory object) is not bound to the open once + established... but don't take my word on it -- it's been some years since + I played with that stuff :-) + <teythoon> antrik: yes, that would actually match my expectation + <braunr> hum + <braunr> normally, mapping increments the usage count of the resource + mapped, but not the open count + <braunr> i don't know if that's the case for mach devices + <braunr> teythoon: which mach device btw ? + <teythoon> time + <teythoon> libshouldbeinlibc/maptime.c line ~53 + <teythoon> the device is opened but never closed + <braunr> is that a problem ? + <teythoon> not sure, but I'd think so, yes + <braunr> why ? + <teythoon> the open count is incremented each time + <braunr> at map time ? + <braunr> ah no, since that's your question + <braunr> the open count is normally decremented when the send right for the + device is destroyed, which occurs when tasks exit + <teythoon> hm + <teythoon> but wouldn't only important long running servers use the mach + device ? + <braunr> all tasks do + <braunr> a simple call to gettimeofday will use it + <teythoon> well, but only privileged processes may get teh device master + port + <braunr> the device is probably accessible through some other method + <teythoon> yes. /dev/time + <teythoon> err, have you looked at the function ? ;) + <braunr> no + <braunr> which one ? + <teythoon> maptime_map + <braunr> i did once but quickly + <teythoon> if use_mach_dev, the mach device is used, /dev/time otherwise + <braunr> gettimeofday apparently uses __host_get_time + <braunr> mhmm + <braunr> ok so i was wrong + <braunr> the time device, whether it is the mach or the hurd one, seems to + be mapped only by translators + <braunr> 14:10 < teythoon> but wouldn't only important long running servers + use the mach device ? + <braunr> so yes :) + <teythoon> so we should close the device + <braunr> why ? + <teythoon> to prevent an overflow in the open count + <braunr> when is it open multiple times ? + <teythoon> isn't it ? maybe /me lacks some context ;) + <braunr> it's called once at init time + <teythoon> well, ok then + <braunr> gettimeofday-like functions then only read the mapped memory + <braunr> at least, that's how it's done in the servers i've looked at such + as pfinet + <teythoon> makes sense, yes + <braunr> something i learnt from experience and failures: check the problem + actually exists before fixing it :p + <teythoon> well, if the memory mapping is independent of the device, then + there is a problem + <teythoon> the device is kept open for no reason + <braunr> teythoon: if you can determine that the device doesn't need to + stay open for the mapping to remain, then you can close it + <braunr> otherwise, it's such a minor leak that we don't care at all + <braunr> i wouldn't even consider it a leak more than a small static + variable used at init time only + <teythoon> looks like, yes + <teythoon> also, it's only in the rootfs translator + <braunr> ? + <teythoon> only the root filesystem uses the mach device directly + <braunr> ok + <braunr> well, /dev/time too right ? + <teythoon> yes, but that is a storeio translator that does not use this + code + <braunr> yes + <braunr> hm + <teythoon> only the root filesystem uses the mach device directly *using + this function* diff --git a/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn b/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn index a169e92e..d03c9674 100644 --- a/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn +++ b/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2013, 2014 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -88,3 +88,33 @@ License|/fdl]]."]]"""]] the caller's address space without going through a special copy-from-user function <braunr> not very portable + + +## IRC, freenode, #hurd, 2013-11-25 + + <teythoon> can we get the mach_printf function somewhere so that it's + easier to use please ? + <braunr> well + <braunr> i'm not comfortable with that + <teythoon> y not ? + <braunr> i don't consider mach_print being part of the interface yet + <teythoon> ... + <braunr> it's really a debugging call + <braunr> not always available + <teythoon> so what, let it fail if it is not + <braunr> ok + <braunr> i should change it first + <braunr> pass a size argument + <braunr> and do a proper copyin + <teythoon> cool, thanks :) + <braunr> then we could add that printf function in libshouldbeinlibc i + guess + <teythoon> that'd be nice, yes + <braunr> hm + <braunr> that would make only hurd servers able to use it though + <braunr> unless we preload it + <teythoon> well, i thought it's a staging area for libc ? + <braunr> in theory yes, in practice some functions have been stuck there + for ages + <braunr> we'll discuss that with youpi and tschwinge + <braunr> and pinotree diff --git a/microkernel/mach/gnumach/ports/xen.mdwn b/microkernel/mach/gnumach/ports/xen.mdwn index c6023786..a8a1fcbf 100644 --- a/microkernel/mach/gnumach/ports/xen.mdwn +++ b/microkernel/mach/gnumach/ports/xen.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2007, 2008, 2009, 2011 Free Software Foundation, -Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2009, 2011, 2013, 2014 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -96,6 +96,176 @@ Download http://youpibouh.thefreecat.org/hurd-xen/pv-grub.gz into /boot, and use extra is now the path to the grub config file. + +## IRC, freenode, #hurd, 2013-11-09 + + <phcoder> youpi: would a limitation of 32 modules to hurd in pvgrub2 be a + problem? + <phcoder> *31 + <youpi> phcoder: probably not + <phcoder> youpi: ok + + <phcoder> youpi: gnumach goes into infinite loop with "warning: nsec + 0x000096dae65d2697 < lastnsec 0x000096db11dee20d". Second value stays + constant, first value loops from 0x000096da14968a59 to + 0x000096db08bf359e. Not sure if the problem is on GRUB or gnumach ide + <youpi> loops?! + <youpi> that's the time coming from the hypervisor + <youpi> not a problem from GRUB anyway + <phcoder> Yes, loops in steps of around 0x40 and comes back regularly. + <youpi> Mmm, maybe it could be grub not properly setting up + hyp_shared_info.vcpu_info[], actually + <youpi> i.e. the mfn in boot_info.shared_info + <phcoder> I don't think we write to shared page at all + <phcoder> could gnumach suffer from overflow on fast CPU? + <phcoder> next_start.shared_info = grub_xen_start_page_addr->shared_info; + <phcoder> And shared_info is machine address, so no need to adjust it + <phcoder> tsc_shift can be negative. Does gnumach handle this? + <youpi> yes + <youpi> here it's the base which doesn't change, actually + <phcoder> Do you mean this: system_time = + time->system_time; ? + <phcoder> But wait: ((delta * (unsigned64_t) mul) >> 32) + <phcoder> this overflows after 2^32 nanoseconds + <phcoder> which is about 4 seconds + <phcoder> I think this is the mistake + <phcoder> which is more or less what I see + <phcoder> Let me make a patch + <youpi> does xen have some tickless feature now? + <youpi> I'd expect the clock to get updated at least sometimes during 4 + seconds :) + <phcoder> Hm, can't compile master: + <phcoder> ./include/mach/xen.h:52:18: error: ‘MACH2PHYS_VIRT_START_PAE’ + undeclared (first use in this function) + <phcoder> #define PFN_LIST MACH2PHYS_VIRT_START_PAE + <phcoder> Here is the patch: http://paste.debian.net/64857/ + <youpi> it's defined in xen/public/arch-x86/xen-x86_32.h + <phcoder> yes it is. Let's see why it's not included + <phcoder> Hm, for some reason it pulls 64-bit headers in + <youpi> how do you cross-compile? + <youpi> I use + <youpi> ./configure --host=i686-gnu CC='gcc -m32' LD='ld -melf_i386' + <phcoder> Yes. GRUB adds those itself + <phcoder> youpi: confirmed: my patch solves the problem + <phcoder> any yes: I tried with unpatched master and it fails + <youpi> ok + <youpi> phcoder: thanks! + + <phcoder> Now I get plenty of "getcwd: cannot access parent directories: + Inappropriate file type or format". But I don't think it's grub-related + <youpi> what do you get before that? + <youpi> perhaps ext2fs doesn't get properly initialized + <youpi> which module commande line do you get in the boot log? + <youpi> perhaps it's simply a typo in there + <phcoder> http://paste.debian.net/64865/ + <youpi> $(task-create) $(task-resume) is missing at the end of the ext2fs + line indeed + <youpi> in your paste it stops at $( + <phcoder> this is at the end of my console. I believe it to be + cosmetic. Let me reset console to some sane state + <youpi> ok + <youpi> the spurious event at the start is probably worth checking up + <youpi> it looks like events that pvgrub2 should have eaten + <youpi> (in its own drivers, before finishing shutting them down) + <phcoder> when redirecting console to file: http://paste.debian.net/64868/ + <phcoder> could swapon have sth to do with it? + <youpi> I'd be surprised + <phcoder> my guess it's because I use older userland (debian about May) and + new kernel (fresh from master) + <youpi> the kernel hasn't really changed + <youpi> you could rebuild the may-debian kernel with your patch to make + sure + <youpi> but probably better trying to fix swapon first, at least + <youpi> (even if that'd surprise me) + <youpi> "trying fixiing* swapon", actually + <youpi> it makes a difference :) + <phcoder> We actually never eat event on evtchn, we look into buffers to + check for response + <youpi> ah, that's why + <youpi> you should really eat the events too + <youpi> in principle it wouldn't hurt not to, but you'd probably get + surprises + + <phcoder> youpi: would doing EVTCHNOP_reset at the end be enough? + <youpi> possibly, I don't know that one + <youpi> looks like a good thing to do before handing control, indeed + <youpi> /* Clear pending event to avoid unexpected behavior on + re-bind. */ + <youpi> evtchn_port_clear_pending(d1, chn1); + <youpi> yes, it does clear the pending events + <phcoder> http://paste.debian.net/64870/ + <phcoder> I did this: http://paste.debian.net/64871/ + <youpi> well, closing the event channels would be a good idea too + <youpi> (reset does not only clear pending events, it also closes the event + channels) + <phcoder> well we can't close console one. So it leave to close disk ones + (the ones we allocated) + <phcoder> http://paste.debian.net/64875/ + <phcoder> New log: http://paste.debian.net/64876/ (swapon fixed, given 1G + of memory) + <youpi> ok, so it really is something else + <phcoder> looks like there is a space after $(task-resume) but can't tell + if it's real or comes from message + <phcoder> tottally artefact + + <phcoder> youpi: this happens when booted in qemu with old kernel now. Now + my bet is on weird fs corruption or because I accessed it with Linux in + rw. In any case I feel like it's time to call it a port and commit + <youpi> I'd say so, yes + <phcoder> Let's look what's remaining: vfb, vkbd and vif: don't need them + for first port commit. Hm, there is an issue of default configfile. What + is pvgrub default behaviour? + <youpi> iirc it just enters the shell + <youpi> I had implemented vfb and vkbd to get the graphical support, but + that's optional indeed + <youpi> vif is useful for netboot only + <youpi> ah, no, by default it runs dhcp --with-configfile + <phcoder> youpi: port committed to trunk + <youpi> \o/ + <youpi> I was lamenting for 5 years that that wasn't happening :) + <youpi> Citrix could have asked one of his engineers to work on it, really + <phcoder> documentation on using the port is still missing though + <youpi> amazon EC2 users will be happy to upgrade from pv-grub to pv-grub2 + :) + <youpi> I asked some amazon guy at SuperComputing whether he knew how many + people were using pv-grub, but he told me that was customer private + information + <phcoder> Another interesting idea would be to switch between 64-bit and + 32-bit domains somehow + <youpi> yes, we were discussing about it at XenSource when I implemented + pv-grub + <youpi> that's not really an easy thing + <youpi> pvh would probably help there, again + <youpi> in the end, we considered that it was usually not hard to select a + 32bit or 64bit pv-grub depending on the userland bitness + <youpi> we considered adding a hypercall to change the bitness of a domU, + but that's really involved + <phcoder> Well when you discussed i386 domains were still around + <phcoder> now it's only PAE and amd64 and they are very similar. Only few + gdt differ + <youpi> well, switching from 32-PAE to 64 is not *so* hard + <youpi> since a 32bit-loaded OS can fit in 64bit + <youpi> the converse is more questionable of course + <phcoder> yes + <youpi> still, it's really not easy for the hypervisor + <youpi> it'd mean converting stuff here and there + <youpi> most probably missing things here and there :) + <phcoder> Ok, not that important anyway + <youpi> we felt it was too dangerous to promise the feature as working :) + <youpi> heh, 5000 lines patch, just like my patch adding support to Mach :) + <phcoder> BTW do you know how to check if kernel supports dom0 ? Apparently + there is feature "privilegied" and dom0 kernels are supposed to have it + in notes but my linux one doesn't even though I'm in xen now + <youpi> it's XENFEAT_dom0 + <youpi> called dom0 in the notes + <phcoder> http://paste.debian.net/64894/ + <youpi> well, maybe the hypervisor doesn't actually check it's there + <youpi> phcoder: what does grub-mkstandalone? + <phcoder> puts all modules in memdisk which is embed into core + <youpi> ah, ok + <youpi> we didn't have to care about that in grub1 indeed :) + + # Partitions You will need the following notation for the gnumach root= parameter: @@ -108,6 +278,23 @@ You will also need to use the parted storeio module for the /dev entries, for in settrans -fgap /dev/hd0s1 /hurd/storeio -T typed part:1:device:hd0 + +## IRC, freenode, #hurd, 2013-11-09 + + <phcoder> youpi: now I get "hd0: dom0's VBD 768 + (/home/phcoder/diskimg/debian-hurd-20130504.img,w) 3001MB" + <phcoder> but "start ext2fs: ext2fs: device:hd0s1: No such device or + address" + <phcoder> disk = [ + 'file:/home/phcoder/diskimg/debian-hurd-20130504.img,hda,w' ] + <phcoder> Hm, using "disk = [ 'phy:loop0,hda1,w' ]" instead worked (loop0 + is an offset loop) + <youpi> yes, xen disks don't support partitioning + <youpi> and we haven't migrated userland to userland partitioning yet + +[[hurd/libstore/part]]. + + # Miscellaneous [[Internals]]. @@ -125,3 +312,20 @@ IRC, freenode, #hurd, 2011-06-08 <braunr> youpi: does xen provide disk caching options ? <youpi> through a blktap, probably <braunr> ok + + +# IRC, freenode, #hurd, 2013-11-09 + + <phcoder> youpi: debian-hurd-20130504.img apparently has a kernel without + xen note. Do I have to do sth special to get xen kernel? + <youpi> phcoder: there's the -xen package for that + <youpi> I haven't made the kernel hybrid + <phcoder> youpi: easiest way is probably to have different entry + points. You could even just link both of them at different addresses and + then glue together though it's not very efficient + <youpi> it's also about all the privileged operations that have to be + replaced with PV operations + <youpi> PVH will help with that regard + + <phcoder> youpi: btw, I recommend compiling xen kernel for 686 and drop + non-pae diff --git a/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn b/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn index 89a27b01..0ca6d91e 100644 --- a/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn +++ b/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn @@ -1,13 +1,13 @@ -[[!meta copyright="Copyright © 2005, 2006, 2007, 2008, 2010, 2012 Free Software -Foundation, Inc."]] +[[!meta copyright="Copyright © 2005, 2006, 2007, 2008, 2010, 2012, 2013, 2014 +Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!tag open_issue_gnumach]] @@ -130,3 +130,106 @@ further files (also exported ones) that serve no real value, but are being <mcsim> this parameter has name data_count <braunr> that's one of the reasons mach is confusing <braunr> i can't really tell you why, it's messy :/ + + +# [[open_issues/Code_Analysis]], *Coverity* + +## IRC, freenode, #hurd, 2014-02-03 + + <teythoon> braunr: at the end of kmem_cache_error the comment says 'never + reached'. i do not believe that to be true... + <braunr> teythoon: yes, this is a mistake + <braunr> those kmem_error calls were panics in the original version + <braunr> the comment was left over + <braunr> or rather, the panic equivalent not added + + +### IRC, freenode, #hurd, 2014-02-04 + + <teythoon> braunr: but the code relies uppon the fact that kmem_cache_error + does not return + <teythoon> the fact that it doesn't upsets static analysis tools + <teythoon> + http://darnassus.sceen.net/~teythoon/qa/gnumach/scan-build/2014-01-17/report-DfjC43.html#Path9 + <braunr> 21:20 < braunr> or rather, the panic equivalent not added + <braunr> teythoon: you're right + <braunr> it needs to be fixed + <teythoon> i setup a coverity project for gnu mach + <teythoon> it's interesting + <teythoon> very enterprisy >,< + <braunr> teythoon: heh + <teythoon> braunr: i assume kmem_error is supposed to panic + <teythoon> if so, i'd propose to use panic instead of printf in that macro + definition + <braunr> yes + <braunr> teythoon: again, that's how it's done in the original + implementation + <teythoon> and fix kmem_cache_error to use kmem_warn near the top of the + function + <braunr> teythoon: i suggest you use the x15 variant as a reference + (http://git.sceen.net/rbraun/x15.git/blob/HEAD:/kern/kmem.c) + <teythoon> right + + +### IRC, OFTC, #debian-hurd, 2014-02-05 + + <pere> ah, cool. <URL: https://scan.coverity.com/projects/1307 > + is now in place. :) + <teythoon> pere: it was you who made me create the coverity project + <teythoon> and it was most fruitful already + <pere> glad to hear that. :) + <pere> while the mach thread made me discover scan-build, which + seem promising but have way to few rules. + <pere> teythoon: btw, could I get access to the mach coverity + status? + <teythoon> pere: sure + <pere> I've used up all my 'join project' slots, so I suspect you + have to invite me. try pere@hungry.com + <teythoon> pere: ok + <teythoon> pere: i'm fuzzy about the roles + <teythoon> what kind of invite shall i send you ? + <pere> not sure either. + <teythoon> ok + <pere> contributor/* would be fine for me. want to check the + details, not just the overview. + <teythoon> oh + * pere looked up the role description + <teythoon> do you have a link ? + <teythoon> i can always change the role afaics + <pere> go to the member list, found it there. + <pere> (the link is project specific, it seem. + <teythoon> right, found it + <pere> gah, it gave me a new user. :( + <pere> I assumed I would be allowed to accept the invite using my + normal user. + <pere> can you try again with pere-github@hungry.com, which + hopefully will hook the invite to my github authenticated user? + <teythoon> sure + <pere> were you allowed to invite that address? + <pere> ah, I got it in my dashboard without having to do anything + more. + <pere> heh, the email I got claimed the user would not be created + unless I provided a password, but given the member list for mach, + I fail to see how that could be true. + + +### IRC, OFTC, #debian-hurd, 2014-02-06 + + <teythoon> pere: your coverity invitations are still saying 'not + accepted', did neither work ? + <pere> teythoon: one worked just fine. I got access. the other I + did not accept. + <teythoon> pere: good :) + + +### IRC, OFTC, #debian-hurd, 2014-02-25 + + <pere> I am quite impressed with the defect density of Mach, <URL: + https://scan.coverity.com/projects/1307 >. :) + <pere> 12 outstanding issues (but 54 dismissed was surprisingly + high :). + <youpi> pere: 54 dismissed because that's a pattern generated by + mig + <youpi> repeated dozens of times + +[[!tag open_issue_mig]] diff --git a/microkernel/mach/message/msgh_id.mdwn b/microkernel/mach/message/msgh_id.mdwn index 799ed5cc..b20f1fe0 100644 --- a/microkernel/mach/message/msgh_id.mdwn +++ b/microkernel/mach/message/msgh_id.mdwn @@ -15,6 +15,8 @@ files. [[!toc]] +See also [[open_issues/versioning]]. + # IRC, freenode, #hurd, 2012-07-12 diff --git a/microkernel/mach/mig/documentation.mdwn b/microkernel/mach/mig/documentation.mdwn index e6bd1bb9..96731a5d 100644 --- a/microkernel/mach/mig/documentation.mdwn +++ b/microkernel/mach/mig/documentation.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2002, 2003, 2005, 2007, 2008, 2009, 2010, 2013 -Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2002, 2003, 2005, 2007, 2008, 2009, 2010, 2013, +2014 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -99,3 +99,15 @@ pp. 67--77." <braunr> simple (but optimized) scatter-gather <braunr> it makes sense for mig since mach messages do include serialization metadata such as types + + +## IRC, freenode, #hurd, 2013-11-01 + + <gnu_srs> Is it possible to call server functions from user space? What + about functions returning MIG_NO_REPLY? + <neal> gnu_srs: server functions are only called from user space... + <neal> gnu_srs: Normally, servers use a server loop. When an RPC comes it, + it is processed and the processing returns some values that the server + loop returns to the client. + <neal> If the server wants to return the thread to the thread pool without + responding to the RPC, it uses MIG_NO_REPLY. diff --git a/microkernel/x15.mdwn b/microkernel/x15.mdwn deleted file mode 100644 index b56c97fc..00000000 --- a/microkernel/x15.mdwn +++ /dev/null @@ -1,24 +0,0 @@ -[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -The x15 microkernel is under development by Richard Braun. Overall, x15 is intended to provide better performance because of simpler IPC and data structures and better scalability because of improved data structure algorithms and concurrency. - -The following specific features are intended to distinguish x15 from other microkernels. However, it should be noted that the microkernel is under heavy development and so the list may (and almost certainly will) change. - -* SMP support -* Lockless synchronisation à la RCU -* Support for large amounts of physical memory. GNU Mach does the same as Linux, physical memory is mapped in kernel space so you can't have more physical memory than you have kernel space which is why GNU Mach can't handle more than 1.8G right now, it's a 2/2 split. In x15, the kernel maps what it needs and can map it from anywhere in physical memory the drawback is that it needs to update kernel page tables more often. -* Virtual memory operations are planned to be completely concurrent on x15, similar to what is described in radixvm -* Intended to efficiently run a Hurd-like system providing a POSIX interface -* Providing both synchronisation IPC and signals, as opposed to just synchronisation IPC -* Port names which will transparently identify as file descriptors -* Light reference counting -* A restriction on IPC that only allows reliable transfers across network to machines with same arch and endianness -* etc. |