From c4ad3f73033c7e0511c3e7df961e1232cc503478 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Wed, 26 Feb 2014 12:32:06 +0100 Subject: IRC. --- microkernel/mach/deficiencies.mdwn | 596 ++++++++++++++++++++- microkernel/mach/gnumach/debugging.mdwn | 23 +- microkernel/mach/gnumach/interface.mdwn | 38 +- .../mach/gnumach/interface/device/time.mdwn | 88 ++- .../mach/gnumach/interface/syscall/mach_print.mdwn | 32 +- microkernel/mach/gnumach/ports/xen.mdwn | 208 ++++++- .../mach/gnumach/projects/clean_up_the_code.mdwn | 111 +++- microkernel/mach/mig/documentation.mdwn | 16 +- 8 files changed, 1096 insertions(+), 16 deletions(-) (limited to 'microkernel/mach') diff --git a/microkernel/mach/deficiencies.mdwn b/microkernel/mach/deficiencies.mdwn index 2e205a9a..d79fa337 100644 --- a/microkernel/mach/deficiencies.mdwn +++ b/microkernel/mach/deficiencies.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013, 2014 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -812,6 +813,10 @@ In context of [[open_issues/multithreading]] and later [[open_issues/select]]. or both? probably netbsd drivers and if netbsd, will it utilize rump? + +[[open_issues/user-space_device_drivers]], *External Projects*, *The Anykernel +and Rump Kernels*. + i don't know yet ok device drivers and networking will arrive late @@ -2691,3 +2696,592 @@ freenode, #hurd, 2013-10-24*: my plan for x15 is to make this "label" part of received messages which means you need to change the format of messages that is what i call a big change + + +### IRC, freenode, #hurd, 2013-10-31 + + neal: you mentioned you want to use Genode as a base... what + exactly would you want to build on top of it, different than what the + Genode folks are doing? + +[[Genode]]. + + antrik: I want to build a secure operating system. + antrik: One focused on user security. + + braunr: You mean revoke individual send rights? + braunr: Or, what do you mean? + Or do you mean the ability to receive anotification on revocation? + neal: yes, revoking individual send rights + I don't think it is needed in practice. + neal: ok + But, you need a membrane object + Here's the idea: + like a peropen ? + you have say a file server + and a proxy + a process only talks to the file server via the proxy + for the proxy to revoke access to the file object it gave out, it + needs to either use your revoke + interpose on all ipcs (which is expensive) + or use a proxy object/membrane + which basically forwards messages to the underlying object + isn't that also interposing ? + of course + but if it is done in the kernel, it is fast + ah in the kernel + you just walk a linked list + what's the difference with a peropen object ? + That's another option + you use a peropen and then provide a call to force the per-open to + be closed + so the proxy now invokes the server + the issue here is that the proxy has to trust the server + yes + how can you not trust servers ? + that is, if the intent is to prevent further communication between + the server and the process, the server may ignore the request + in this case, you probably trust the server + hum + but it could be that you have two processes communicating + if the intent is to prevent communication, doesn't the client just + need to humm not communicate ? :) + the point is that the two processes are colluding + what are these two processes ? + I'm not sure this case is of practical relevance + ok + https://www.cs.cornell.edu/courses/cs513/2002sp/L10.html + thanks + + +### IRC, freenode, #hurd, 2013-11-14 + + neal: hm... I was under the impression that the Genode themselves + are also interested in user security... what is missing from their + version that you want to add? + err... the Genode folks + antrik: I'm missing some context + neal: a while back you said that you want to build a secure system + on top of Genode + yes + the fact that they are doing what I want is great + but there is more to a secure system than an operating system + ah, so it's about applications+ + ? + yes, that is part of it + it's also about secure messaging + and hiding "meta-data" + i'm still wondering how you envision the powerbox + when a program wants the user to select a file, it makes an upcall + to the power box application + braunr: you can probably find some paper from Shapiro ;-) + well, sure, it looks easy + but is there always a power box application ? + is there always a guarantee there won't be recursive calls made by + that application ? + how does it integrate with the various interfaces a system can + have ? + there is always a power box application + I don't know what you mean by recursive calls + aer techniques such as remembering for some time like sudo does + applicable to a powerbox application ? + if you mean many calls, then it is possible to rate limit it + well, the powerbox will use messaging itself + is it always privileged ? + privileged enough + it is privileged such like the X11 display manager is privileged and + can see all of the video content + what else other than accessing a file would it be used for ? + one case i think of is accessing the address space of another + application, in debuggers + 14:56 < neal> there is always a power box application + what would it be when logging on a terminal ? + braunr: when running pure command line tools, you can already pass + the authority as part of the command line. however, I'm wondering whether + it really makes sense to apply this to traditional shell tools... + that's one of my concerns + when does it really make sense ? + for interactive use (opening new files from within a running + program), I don't think it can be accomplished in a pure terminal + interaction model... + and you say "you pass the authority" + braunr: it makes sense for interactive applications + i thought the point of the powerbox is precisely not to do that + no, it's still possible and often reasonable to pass some initial + authority on startup. the powerbox is only necessary when further access + needs to be provided at runtime + ok + the power box enable dynamic delegation of authority, as antrik said + ok + but how practical is it ? + applications whose required authority is known apriori and + max(required authority) is approximately min(required authority) can be + handled with static policies + don't application sometimes need a lot of additional authority ? + ok + actally, thinking about it, a powerbox should also be possible on + a simple terminal, if we make sure the application doesn't get full + control of the terminal, but rather allow the powerbox to temporarily + take over input/output without the application being able to + interpose... so not quite a traditional UNIX terminal, but close enough + I'd say + the terminal itself maybe ? + hm... that would avoid having to implement a more generic + multiplexing approach -- but it would mix things that are normally quite + orthogonal... + BTW, I personally believe terminals need to get smarter anyways + :-) + ok + the traditional fully linear dialog has some nice properties; but + it is also pretty limited, leading to usability problems soon. I have + some vague ideas for an approach that still looks mostly like a linear + dialog, but is actually more structured + + +## IRC, freenode, #hurd, 2013-11-04 + + yes the learning curve [of the Hurd] is too hard + that's an entry barrier + this is why i use well known posix-like (or other well + established) apis in x15 + also why i intend to make port rights blend into file descriptors + right + well + the real reason is efficiency + but matching existing practices is very good too + + +## IRC, freenode, #hurd, 2013-11-08 + + braunr, how is work on x-15 progressing? Is there some site to + check what is new? + gnufreex: stalled for 2 months + i'm working on the hurd for now, will get back to it later + no site + well + so, you hit some design problem, or what? I mean why stalled + http://git.sceen.net/rbraun/x15.git/ :p + Thanks + something like that yes + i came across + http://darnassus.sceen.net/~rbraun/radixvm_scalable_address_spaces_for_multithreaded_applications.pdf + I read that, I think I found it on Hurd site. + and since x15 aims at being performant and scalable, it seems like + a major feature to bring in + but it's not simple to integrate + So you want to add that? + gnufreex: yes + branur, but what are the problems? + ? + ah + you really want to know ? :) + Well... yeah + you need to know both x15 and radixvm for that + for one, refcache, as described in the radixvm paper, doesn't seem + scalable + it is in practice in their experiments, but only because they + didn't push some parameters too high + so i need to rethink it + I don't know x15... but I read radixvm paper + next, the bsd-like vm used by x15 uses a red-black tree to store + memory areas, which doesn't need external storage + radixvm as implemented in xv6 is only used for user processes, not + the kernel + which means the kernel allocator is a separate implementation, as + it's done in linux + x15 uses the same implementation for both the kernel and user maps + which results in a recursion problem + because a radix tree uses external nodes that must be dynamically + allocated + so you would pretty much need to rewrite x15 + no + just vm/ + and $arch/pmap + and yes, pmap needs to handle per-core page tables + something i wanted to add already but couldn't because of similar + recursion problems + Yeah, vm system... but what else did you do with x15... it is at + early stage... + multithreading + That doesn't need to be rewriten? + no + Ok... good. + physical memory allocation neither + only virtual memory + is x15 in runable state? I mean in virtual machine? + you can start it + but you won't go far :) + What do you use as development platform? + it basically detects memory and processors, starts idle, migration + and worker threads, and leaves + Is it compilable on fedora 19 + probably + i use debian stable + and unstable on the hurd + ok, I will probably try it in KVM... + better do it on real hardware too in case you find a bug + I cant make new partition now... it seems my hard drive is + dying. When I get a new one I will try on real harware. + you don't need a new partition + the reason radixvm is important is twofold + 1/ ipc will probably make use of the core vm operations used by + mmap and munmap + 2/ no other system currently provides scalable + mmap/munmap/mprotect + Yes, that would make x15 pretty special... + But I read somewhere that you wanted to implement RCU during + summer + Did you do that? + + +## IRC, freenode, #hurd, 2013-11-12 + + neal: about secure operating systems + i assume you consider clients providing their own memory a strong + requirement for that, right ? + no + I'm less interested in availability + or performance guarantees + ok + but + i thought it was a requirement to avoid denial of service + of course + then why don't you consider it required ? + I want something working in a reasonable amount of time :) + agreed + more seriously: + my primary requirement is that a program cannot access information + that the user has not authorized it to access + ok + the requirement that you are suggesting is that a program be able to + access information that the user has authorized it to access + this is availability + i'm not following + what's the difference ? + assume we have two programs: A and B + on Unix, if they run under the same uid, they access access each + other files + I want to fix this + ok, that's not explicit authorization + but is that what you mean ? + Now, assuming that A cannot access B's data and vice versa + we have an availability problem + A could prevent B from accessing its data + via a DoS attach + I'm not going to try to fix that. + ok + and how do you intend to allow A to access B's data ? + i guess the powerbox mentioned in the critique + but do you have a more precise description about something + practical to use ? + + +## IRC, freenode, #hurd, 2013-11-14 + +In context of [[hurd/libports]], *Open Issues*, *IRC, freenode, #hurd, +2013-11-14*. + + fyi, x15 will not provide port renaming + teythoon: also, i'm considering enforcing port names to be as + close as possible to 0 when being allocated as part of the interface + what do you think about that ? + braunr: that's probably wise, yes + you could hand out receive ports close to 0 and send ports close + to ~0 + teythoon: what for ? + well, if one stores only one kind in an array, it won't waste as + much space + this also means you need to separate receive from send rights in + the interface + so that you know where to look for them + i'm not sure it's worth the effort + using the same code for them both looks more efficient + the right lookup code is probably one of the hottest path in the + system + right + one of the nice things about not reusing port names is that it helps + catch bugs + you don't want to accidently send a message to the wrong recipient + how could you, if the same name at different times denotes + different rights ? + you forget to clean up something + if you don't clean, how could you get the same name for a right + you didn't release ? + that's not hard to do :) + ah, you cleaned up the port right but not the name + ah ok + destroy the port and forget that a thread is still working on a + response + the data structure says use the port at index X + X is reallocated in the mean time + excuse my ignorance, but gnumach *is* reusing port names, isn't + it? + that policy is why i'm not sure i want to enforce allocation + policy in the interface :/ + This is not about a security property of the system + this is about failing fast + you want to fail as close to the source of the problem as possible + we could make the kernel use different allocation policies for + names, to catch bugs, yes + make the index X valid again and you've potentially masked the bug + braunr: if you were to merge your radix tree implementation into + gnumach and replace the splay tree with it, would that make using renamed + ports fast enough so we can just rename all receive ports doing away with + the extra lookup like mach-defpager does ? + i don't think so + the radix tree code is able to compress its size when keys are + close to 0 + using addresses would add 1, 2, maybe 3 levels of internal nodes + for every right + we could use a true integer hash table for that though + hm no, hurd packages crash ... :/ + but malloc allocates stuff in a contigious space, so the + pointers should be similar in the most significant bits + if you use malloc, yes + sure + but that'd make the radix tree representation compact, no? + it could + the current code only compresses near 0 + oh + better compression could be implemented though + + +## IRC, freenode, #hurd, 2013-11-21 + + have you seen liburcu ? + a bit, yes + it might be worth investigating to use it in some servers + it is + the proc server comes to mind + personally, i think all hurd servers should use rcu + libports should use rcu + yes + lockless synchronization should be a major feature of x15/propel + present even during message passing + + +## IRC, freenode, #hurd, 2013-12-09 + + improving our page cache with arc would be great + it's on the todo list for x15 :> + not sure you referred to virtual memory management though + (actually, it's CAR, not ARC that is planned for x15) + + +## IRC, freenode, #hurd, 2013-12-30 + + zacts: http://darnassus.sceen.net/~rbraun/x15/qemu_x15.sh + + +## IRC, freenode, #hurd, 2014-01-03 + + oh, btw, i've started working on x15 again :> + saw that :) + first item on the list: per-cpu page tables + the magic that will make ipc extremely scalable :) + i'm worried about your approach tbh + too much overhead ? + not on any technical level + but haven + ? + 't there been enough reimplementation efforts that got nowhere ? + oh that + ^^ + well, i have personal constraints and frustrations with the + existing code, and my goal isn't to actually produce anything serious + until it actually gets there + which, yes, it might not + really, i'm doing it for fun + well sure + that's a damn good reason ;) + and if it ever reaches a state where it can actually be used to + run stuff, i would be very happy + and considering how it's done, i'm pretty sure things could be + built a lot faster on such a system + but you need to reimplement all the userspace servers as well, + and the libc stuff + yes + do you plan to reimplement this from scratch or do you have + plans to 'bootstrap' propel from hurd ? + from scratch + well... i'm not sure that this is feasible or even a good + idea. that's what i meant in a nutshell i guess. + i'm familiar with that criticism + and you may be right + this is also why i keep working on the hurd at the same time + we could also talk about making hurd more easily portable + portable with regard to what ? + evolving hurd and mach to the point where it might be feasible + to port hurd to another ukernel + not so easy + i know + i'm not even sure i would want that + well, since the hurd isn't optimized at all, why not + why would it neccessarily hinder optimization ? + because in practice, it's rare for a microkernel to provide all + the features the hurd would require to run really well + the most severe issue being that they either provide asynchronous + ipc, used for signals, or only synchronous ipc, making signal and other + event-driven code hard to emulate (usually requiring separate threads) + + +## IRC, freenode, #hurd, 2014-01-20 + +[[translate_fd_or_port_to_file_name]]: + + i wonder if it would not be best to add a description to mach + tasks + i think it would + to aid fixing these kind of issues + in x15, i actually add descriptions (names) to all kernel objects + that's probably a good idea, yes + well, not all, but many + + i'd like to push x15 this year + it currently is the only design of a truely scalable microkernel + that i know of + push how? + spend time on it + k + do you think it will make sense to solicit outside contributions + at one point? + yes + the roadmap is vm system -> ipc system -> userspace (including RPC + handling) + once we can actually do things in userspace, the priority will be + getting a shell with glibc + people will be able to help "easily" at that point + just wondering, apart from scalability, did you write it for + performance, for hackability, or something else? + it's basically the hurd architecture, including improvements from + the critique, with performance and scalability in mind + ok + the main improvements i think of currently are resource + containers, lexical .. resolution, and lists of trusted users with which + to communicate + it's strongly oriented for posix compatibility though + sounds nice, i like it already ;) + is it compatible with Mach to some degree? + so things like running without an identity will be forbidden in + the default system personality + no, not compatible with mach at all + this sounds like it is doing more than Mach did + braunr: ah, ok + it's not "x15mach" any more :) + right, I missed out on that + + +### IRC, freenode, #hurd, 2014-01-21 + + i also don't write anything that would prevent real-time + b/c that's a potential market for such an operating system ? + yes + well, i can't say i don't like the sound of that ;) + the ipc interface should be close to that of qnx + + +## IRC, freenode, #hurd, 2014-02-23 + + braunr: have you looked at genode? + braunr: i sometimes wonder how hard it'd be to port hurd atop it + because i find some similarities with what l4/fiasco/viengos provided + cluck: i have, but genode seems a bit too far from posix for our + tastes + (and yes, i realize we'd be getting farther from the hw) + ah you really mean running the hurd on top of it + i personally don't like the idea + braunr: well, true, but their noux implementation proves it's not a + dealbreaker + braunr: at least initially that'd be the best implementation + approach, no? as time went on integrating hurd servers more tightly at a + lower level makes sense but doing so from the get go would be foolhardy + braunr: or am i missing something obvious? + cluck: why would it be ? + braunr: going by what happened with l4 it's too much code to port + and optimize at once + cluck: i don't think it is + cluck: problems with l4 didn't have much to do with "too much + code" + braunr: i won't debate that, you have more experience than me with + hurd code. anyway that's how i'd go about it, first get it all running + then get it running fast. breakage is bad + and you think moving from something like linux or genode to an + implementation closer to hardware won't break things ? + braunr: yes, i read the paper, obvious unexpected shortcomings but + even had them not been there the paradigms are too different and creating + proper mappings from one model to the other would at least be time + consuming + ye + yes + i'm convinved the simple approach of a small microkernel with the + proper interfacen along with the corresponding sysdeps layer in glibc + would be enough to get a small hurd like system quickly + experience with other systems shows how to directly optimize a lot + of things from the start, without much effort + + braunr: sorry. back to our talk, i mentioned genode because of the + nice features it has that'd be useful on hurd + cluck: which ones do you refer to ? + braunr: the security model is the biggest one + how does it differ from the hurd, except for revocation ? + braunr: then there's the ease of portability + ? + braunr: it's more strict + how would that help us ? + braunr: if hurd was running atop it we'd get extra platforms + supported almost for free whenever they did (since we'd be using the same + primitives) + why not choose the underlying microkernel directly ? + call me crazy but i believe code reuse is a good thing, i see + little point in duplicating existing code just because you can + what part of genode should be reused then ? + that's what got me thinking about genode in the first place, + ideologically they share a lot (if not most) of hurd's goals and code + wise they feel close enough to make a merge of sorts not seem crazy talk, + thus my asking if i'm missing something obvious + i think the design is incompatible with our goals of posix + compatibility + braunr: oh, ok. + braunr: i was assuming that wasn't an issue, as i mentioned before + they have noux already and if hurd's servers got ported they'd provide + whatever else that was missing + noux looks like a unix server for binary compatibility + i'm not sure it is but that's what the description makes me think + and if it really, then it's no different than running linux on top + of an hypervisor + ok it's not for binary compatibility but it definitely is a + (partial) unix server + i much prefer the way the hurd is posix compliant without any + additional layer for compatibility or virtualization + braunr: noux is a runtime, as i understand it there's no binary + compatibility just source (ie library/api calls) + yes i corrected that just now + sorry, i'm having lag issues + no worries + braunr: anyway, how's x15 coming along? still far from being a + practical replacement? + yes .. :( + and it's not a replacement + (for mach) + no + huh? + it's not a replacement for the hurd + err, for mach + braunr: i thought you were writing it to be compatible with mach's + interfaces + no + it used to be that way + but no + braunr: what changed? + mach ipc is too ccmplicated + complicated* + its supposed benefit (of allowing the creation of computer + clusters for single system images) are outdated and not very interesting + it's error prone + and it incurrs more overhead than it should + no arguing there + braunr: are you still targeting being able to run hurd atop x15 or + is it just your pet project now? + i don't intend the hurd to run on top of it + the reason it's a rewrite is to fix a whole bunch of major issues + in one go diff --git a/microkernel/mach/gnumach/debugging.mdwn b/microkernel/mach/gnumach/debugging.mdwn index 7e7cfb4e..cf02471d 100644 --- a/microkernel/mach/gnumach/debugging.mdwn +++ b/microkernel/mach/gnumach/debugging.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2007, 2008, 2009, 2011, 2012, 2013 Free Software -Foundation, Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2009, 2011, 2012, 2013, 2014 Free +Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -135,7 +135,7 @@ update its display buffer anymore under certain conditions. # Halting the CPU and Examining Registers -IRC, freenode, #hurd, 2011-07-14: +## IRC, freenode, #hurd, 2011-07-14 one ugly trick i use when printf isn't available is to halt the cpu @@ -146,7 +146,22 @@ IRC, freenode, #hurd, 2011-07-14: # Serial Console -IRC, freenode, #hurd, 2011-11-13: +## IRC, freenode, #hurd, 2011-11-13 use console=com0 to activate the console on the first serial port + + +# `ud2` instruction + +## IRC, freenode, #hurd, 2013-10-31 + + [master-x86_64] + GNU Mach 1.3.99 + Running on xen-3.0-x86_64. + AT386 boot: physical memory from 0x0 to 0x40000000 + \o/ + well when loaded through pvgrub2 i hangs without any message + any pointers on debugging? + I usually put the ud2 instruction along the path to see where it + crashes diff --git a/microkernel/mach/gnumach/interface.mdwn b/microkernel/mach/gnumach/interface.mdwn index b2451887..bfcb9ea6 100644 --- a/microkernel/mach/gnumach/interface.mdwn +++ b/microkernel/mach/gnumach/interface.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2013, 2014 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -14,3 +14,39 @@ License|/fdl]]."]]"""]] [[!map pages="microkernel/mach/gnumach/interface/*" show=title]] + + +# Proposed + +## Futex + +### Patches on bug-hurd + +### IRC, freenode, #hurd, 2014-01-20 + + braunr: i meant to ask about the futex stuff + what do you mean when you talk about the interface ? + i imagined something like this + more precisely ? + for shared futexes, do not use a pointer, but a right + referencing a memory object, and the offset ? + why not a pointer ? + well, an address + that's how it's currently done, right ? + in marins prototype i mean + yes + it's a task/address pair + so the only concern is the access in the kernel ? + hum not only no + or yes, but there are multiple ways to access + i meant to say this, for me, it would be very helpful if you + were more explicit when you describe how you picture something + i thought i did it clearly + i even agree with what roland describes + which looks even clearer + also, i said i didn't picture it completely + :) + i didn't want to work on that, only review + sure, but what might be clear to you is most likely not that + clear to me + ok diff --git a/microkernel/mach/gnumach/interface/device/time.mdwn b/microkernel/mach/gnumach/interface/device/time.mdwn index d1e9a488..18681f21 100644 --- a/microkernel/mach/gnumach/interface/device/time.mdwn +++ b/microkernel/mach/gnumach/interface/device/time.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2013, 2014 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -16,3 +16,89 @@ Typically available as `/dev/time`, [[hurd/translator/storeio]]. Using that, [[hurd/libshouldbeinlibc]]'s `` provides `maptime_map` and `maptime_read`, see the [[hurd/reference_manual]]. Candidate for replacement with [[open_issues/vDSO]] code? + + +# IRC, freenode, #hurd, 2013-11-20 + + braunr: about the mach device interface, if i open a device, and + then create a memory mapping using device_map, does that increment the + open count of the device ? + can i call device_close w/o destroying the mapping directly + after mapping it ? + teythoon: I have a vague recollection that the mapping (or more + precisely, the memory object) is not bound to the open once + established... but don't take my word on it -- it's been some years since + I played with that stuff :-) + antrik: yes, that would actually match my expectation + hum + normally, mapping increments the usage count of the resource + mapped, but not the open count + i don't know if that's the case for mach devices + teythoon: which mach device btw ? + time + libshouldbeinlibc/maptime.c line ~53 + the device is opened but never closed + is that a problem ? + not sure, but I'd think so, yes + why ? + the open count is incremented each time + at map time ? + ah no, since that's your question + the open count is normally decremented when the send right for the + device is destroyed, which occurs when tasks exit + hm + but wouldn't only important long running servers use the mach + device ? + all tasks do + a simple call to gettimeofday will use it + well, but only privileged processes may get teh device master + port + the device is probably accessible through some other method + yes. /dev/time + err, have you looked at the function ? ;) + no + which one ? + maptime_map + i did once but quickly + if use_mach_dev, the mach device is used, /dev/time otherwise + gettimeofday apparently uses __host_get_time + mhmm + ok so i was wrong + the time device, whether it is the mach or the hurd one, seems to + be mapped only by translators + 14:10 < teythoon> but wouldn't only important long running servers + use the mach device ? + so yes :) + so we should close the device + why ? + to prevent an overflow in the open count + when is it open multiple times ? + isn't it ? maybe /me lacks some context ;) + it's called once at init time + well, ok then + gettimeofday-like functions then only read the mapped memory + at least, that's how it's done in the servers i've looked at such + as pfinet + makes sense, yes + something i learnt from experience and failures: check the problem + actually exists before fixing it :p + well, if the memory mapping is independent of the device, then + there is a problem + the device is kept open for no reason + teythoon: if you can determine that the device doesn't need to + stay open for the mapping to remain, then you can close it + otherwise, it's such a minor leak that we don't care at all + i wouldn't even consider it a leak more than a small static + variable used at init time only + looks like, yes + also, it's only in the rootfs translator + ? + only the root filesystem uses the mach device directly + ok + well, /dev/time too right ? + yes, but that is a storeio translator that does not use this + code + yes + hm + only the root filesystem uses the mach device directly *using + this function* diff --git a/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn b/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn index a169e92e..d03c9674 100644 --- a/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn +++ b/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2013, 2014 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -88,3 +88,33 @@ License|/fdl]]."]]"""]] the caller's address space without going through a special copy-from-user function not very portable + + +## IRC, freenode, #hurd, 2013-11-25 + + can we get the mach_printf function somewhere so that it's + easier to use please ? + well + i'm not comfortable with that + y not ? + i don't consider mach_print being part of the interface yet + ... + it's really a debugging call + not always available + so what, let it fail if it is not + ok + i should change it first + pass a size argument + and do a proper copyin + cool, thanks :) + then we could add that printf function in libshouldbeinlibc i + guess + that'd be nice, yes + hm + that would make only hurd servers able to use it though + unless we preload it + well, i thought it's a staging area for libc ? + in theory yes, in practice some functions have been stuck there + for ages + we'll discuss that with youpi and tschwinge + and pinotree diff --git a/microkernel/mach/gnumach/ports/xen.mdwn b/microkernel/mach/gnumach/ports/xen.mdwn index c6023786..a8a1fcbf 100644 --- a/microkernel/mach/gnumach/ports/xen.mdwn +++ b/microkernel/mach/gnumach/ports/xen.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2007, 2008, 2009, 2011 Free Software Foundation, -Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2009, 2011, 2013, 2014 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -96,6 +96,176 @@ Download http://youpibouh.thefreecat.org/hurd-xen/pv-grub.gz into /boot, and use extra is now the path to the grub config file. + +## IRC, freenode, #hurd, 2013-11-09 + + youpi: would a limitation of 32 modules to hurd in pvgrub2 be a + problem? + *31 + phcoder: probably not + youpi: ok + + youpi: gnumach goes into infinite loop with "warning: nsec + 0x000096dae65d2697 < lastnsec 0x000096db11dee20d". Second value stays + constant, first value loops from 0x000096da14968a59 to + 0x000096db08bf359e. Not sure if the problem is on GRUB or gnumach ide + loops?! + that's the time coming from the hypervisor + not a problem from GRUB anyway + Yes, loops in steps of around 0x40 and comes back regularly. + Mmm, maybe it could be grub not properly setting up + hyp_shared_info.vcpu_info[], actually + i.e. the mfn in boot_info.shared_info + I don't think we write to shared page at all + could gnumach suffer from overflow on fast CPU? + next_start.shared_info = grub_xen_start_page_addr->shared_info; + And shared_info is machine address, so no need to adjust it + tsc_shift can be negative. Does gnumach handle this? + yes + here it's the base which doesn't change, actually + Do you mean this: system_time = + time->system_time; ? + But wait: ((delta * (unsigned64_t) mul) >> 32) + this overflows after 2^32 nanoseconds + which is about 4 seconds + I think this is the mistake + which is more or less what I see + Let me make a patch + does xen have some tickless feature now? + I'd expect the clock to get updated at least sometimes during 4 + seconds :) + Hm, can't compile master: + ./include/mach/xen.h:52:18: error: ‘MACH2PHYS_VIRT_START_PAE’ + undeclared (first use in this function) + #define PFN_LIST MACH2PHYS_VIRT_START_PAE + Here is the patch: http://paste.debian.net/64857/ + it's defined in xen/public/arch-x86/xen-x86_32.h + yes it is. Let's see why it's not included + Hm, for some reason it pulls 64-bit headers in + how do you cross-compile? + I use + ./configure --host=i686-gnu CC='gcc -m32' LD='ld -melf_i386' + Yes. GRUB adds those itself + youpi: confirmed: my patch solves the problem + any yes: I tried with unpatched master and it fails + ok + phcoder: thanks! + + Now I get plenty of "getcwd: cannot access parent directories: + Inappropriate file type or format". But I don't think it's grub-related + what do you get before that? + perhaps ext2fs doesn't get properly initialized + which module commande line do you get in the boot log? + perhaps it's simply a typo in there + http://paste.debian.net/64865/ + $(task-create) $(task-resume) is missing at the end of the ext2fs + line indeed + in your paste it stops at $( + this is at the end of my console. I believe it to be + cosmetic. Let me reset console to some sane state + ok + the spurious event at the start is probably worth checking up + it looks like events that pvgrub2 should have eaten + (in its own drivers, before finishing shutting them down) + when redirecting console to file: http://paste.debian.net/64868/ + could swapon have sth to do with it? + I'd be surprised + my guess it's because I use older userland (debian about May) and + new kernel (fresh from master) + the kernel hasn't really changed + you could rebuild the may-debian kernel with your patch to make + sure + but probably better trying to fix swapon first, at least + (even if that'd surprise me) + "trying fixiing* swapon", actually + it makes a difference :) + We actually never eat event on evtchn, we look into buffers to + check for response + ah, that's why + you should really eat the events too + in principle it wouldn't hurt not to, but you'd probably get + surprises + + youpi: would doing EVTCHNOP_reset at the end be enough? + possibly, I don't know that one + looks like a good thing to do before handing control, indeed + /* Clear pending event to avoid unexpected behavior on + re-bind. */ + evtchn_port_clear_pending(d1, chn1); + yes, it does clear the pending events + http://paste.debian.net/64870/ + I did this: http://paste.debian.net/64871/ + well, closing the event channels would be a good idea too + (reset does not only clear pending events, it also closes the event + channels) + well we can't close console one. So it leave to close disk ones + (the ones we allocated) + http://paste.debian.net/64875/ + New log: http://paste.debian.net/64876/ (swapon fixed, given 1G + of memory) + ok, so it really is something else + looks like there is a space after $(task-resume) but can't tell + if it's real or comes from message + tottally artefact + + youpi: this happens when booted in qemu with old kernel now. Now + my bet is on weird fs corruption or because I accessed it with Linux in + rw. In any case I feel like it's time to call it a port and commit + I'd say so, yes + Let's look what's remaining: vfb, vkbd and vif: don't need them + for first port commit. Hm, there is an issue of default configfile. What + is pvgrub default behaviour? + iirc it just enters the shell + I had implemented vfb and vkbd to get the graphical support, but + that's optional indeed + vif is useful for netboot only + ah, no, by default it runs dhcp --with-configfile + youpi: port committed to trunk + \o/ + I was lamenting for 5 years that that wasn't happening :) + Citrix could have asked one of his engineers to work on it, really + documentation on using the port is still missing though + amazon EC2 users will be happy to upgrade from pv-grub to pv-grub2 + :) + I asked some amazon guy at SuperComputing whether he knew how many + people were using pv-grub, but he told me that was customer private + information + Another interesting idea would be to switch between 64-bit and + 32-bit domains somehow + yes, we were discussing about it at XenSource when I implemented + pv-grub + that's not really an easy thing + pvh would probably help there, again + in the end, we considered that it was usually not hard to select a + 32bit or 64bit pv-grub depending on the userland bitness + we considered adding a hypercall to change the bitness of a domU, + but that's really involved + Well when you discussed i386 domains were still around + now it's only PAE and amd64 and they are very similar. Only few + gdt differ + well, switching from 32-PAE to 64 is not *so* hard + since a 32bit-loaded OS can fit in 64bit + the converse is more questionable of course + yes + still, it's really not easy for the hypervisor + it'd mean converting stuff here and there + most probably missing things here and there :) + Ok, not that important anyway + we felt it was too dangerous to promise the feature as working :) + heh, 5000 lines patch, just like my patch adding support to Mach :) + BTW do you know how to check if kernel supports dom0 ? Apparently + there is feature "privilegied" and dom0 kernels are supposed to have it + in notes but my linux one doesn't even though I'm in xen now + it's XENFEAT_dom0 + called dom0 in the notes + http://paste.debian.net/64894/ + well, maybe the hypervisor doesn't actually check it's there + phcoder: what does grub-mkstandalone? + puts all modules in memdisk which is embed into core + ah, ok + we didn't have to care about that in grub1 indeed :) + + # Partitions You will need the following notation for the gnumach root= parameter: @@ -108,6 +278,23 @@ You will also need to use the parted storeio module for the /dev entries, for in settrans -fgap /dev/hd0s1 /hurd/storeio -T typed part:1:device:hd0 + +## IRC, freenode, #hurd, 2013-11-09 + + youpi: now I get "hd0: dom0's VBD 768 + (/home/phcoder/diskimg/debian-hurd-20130504.img,w) 3001MB" + but "start ext2fs: ext2fs: device:hd0s1: No such device or + address" + disk = [ + 'file:/home/phcoder/diskimg/debian-hurd-20130504.img,hda,w' ] + Hm, using "disk = [ 'phy:loop0,hda1,w' ]" instead worked (loop0 + is an offset loop) + yes, xen disks don't support partitioning + and we haven't migrated userland to userland partitioning yet + +[[hurd/libstore/part]]. + + # Miscellaneous [[Internals]]. @@ -125,3 +312,20 @@ IRC, freenode, #hurd, 2011-06-08 youpi: does xen provide disk caching options ? through a blktap, probably ok + + +# IRC, freenode, #hurd, 2013-11-09 + + youpi: debian-hurd-20130504.img apparently has a kernel without + xen note. Do I have to do sth special to get xen kernel? + phcoder: there's the -xen package for that + I haven't made the kernel hybrid + youpi: easiest way is probably to have different entry + points. You could even just link both of them at different addresses and + then glue together though it's not very efficient + it's also about all the privileged operations that have to be + replaced with PV operations + PVH will help with that regard + + youpi: btw, I recommend compiling xen kernel for 686 and drop + non-pae diff --git a/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn b/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn index 89a27b01..0ca6d91e 100644 --- a/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn +++ b/microkernel/mach/gnumach/projects/clean_up_the_code.mdwn @@ -1,13 +1,13 @@ -[[!meta copyright="Copyright © 2005, 2006, 2007, 2008, 2010, 2012 Free Software -Foundation, Inc."]] +[[!meta copyright="Copyright © 2005, 2006, 2007, 2008, 2010, 2012, 2013, 2014 +Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!tag open_issue_gnumach]] @@ -130,3 +130,106 @@ further files (also exported ones) that serve no real value, but are being this parameter has name data_count that's one of the reasons mach is confusing i can't really tell you why, it's messy :/ + + +# [[open_issues/Code_Analysis]], *Coverity* + +## IRC, freenode, #hurd, 2014-02-03 + + braunr: at the end of kmem_cache_error the comment says 'never + reached'. i do not believe that to be true... + teythoon: yes, this is a mistake + those kmem_error calls were panics in the original version + the comment was left over + or rather, the panic equivalent not added + + +### IRC, freenode, #hurd, 2014-02-04 + + braunr: but the code relies uppon the fact that kmem_cache_error + does not return + the fact that it doesn't upsets static analysis tools + + http://darnassus.sceen.net/~teythoon/qa/gnumach/scan-build/2014-01-17/report-DfjC43.html#Path9 + 21:20 < braunr> or rather, the panic equivalent not added + teythoon: you're right + it needs to be fixed + i setup a coverity project for gnu mach + it's interesting + very enterprisy >,< + teythoon: heh + braunr: i assume kmem_error is supposed to panic + if so, i'd propose to use panic instead of printf in that macro + definition + yes + teythoon: again, that's how it's done in the original + implementation + and fix kmem_cache_error to use kmem_warn near the top of the + function + teythoon: i suggest you use the x15 variant as a reference + (http://git.sceen.net/rbraun/x15.git/blob/HEAD:/kern/kmem.c) + right + + +### IRC, OFTC, #debian-hurd, 2014-02-05 + + ah, cool. + is now in place. :) + pere: it was you who made me create the coverity project + and it was most fruitful already + glad to hear that. :) + while the mach thread made me discover scan-build, which + seem promising but have way to few rules. + teythoon: btw, could I get access to the mach coverity + status? + pere: sure + I've used up all my 'join project' slots, so I suspect you + have to invite me. try pere@hungry.com + pere: ok + pere: i'm fuzzy about the roles + what kind of invite shall i send you ? + not sure either. + ok + contributor/* would be fine for me. want to check the + details, not just the overview. + oh + * pere looked up the role description + do you have a link ? + i can always change the role afaics + go to the member list, found it there. + (the link is project specific, it seem. + right, found it + gah, it gave me a new user. :( + I assumed I would be allowed to accept the invite using my + normal user. + can you try again with pere-github@hungry.com, which + hopefully will hook the invite to my github authenticated user? + sure + were you allowed to invite that address? + ah, I got it in my dashboard without having to do anything + more. + heh, the email I got claimed the user would not be created + unless I provided a password, but given the member list for mach, + I fail to see how that could be true. + + +### IRC, OFTC, #debian-hurd, 2014-02-06 + + pere: your coverity invitations are still saying 'not + accepted', did neither work ? + teythoon: one worked just fine. I got access. the other I + did not accept. + pere: good :) + + +### IRC, OFTC, #debian-hurd, 2014-02-25 + + I am quite impressed with the defect density of Mach, . :) + 12 outstanding issues (but 54 dismissed was surprisingly + high :). + pere: 54 dismissed because that's a pattern generated by + mig + repeated dozens of times + +[[!tag open_issue_mig]] diff --git a/microkernel/mach/mig/documentation.mdwn b/microkernel/mach/mig/documentation.mdwn index e6bd1bb9..96731a5d 100644 --- a/microkernel/mach/mig/documentation.mdwn +++ b/microkernel/mach/mig/documentation.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2002, 2003, 2005, 2007, 2008, 2009, 2010, 2013 -Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2002, 2003, 2005, 2007, 2008, 2009, 2010, 2013, +2014 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -99,3 +99,15 @@ pp. 67--77." simple (but optimized) scatter-gather it makes sense for mig since mach messages do include serialization metadata such as types + + +## IRC, freenode, #hurd, 2013-11-01 + + Is it possible to call server functions from user space? What + about functions returning MIG_NO_REPLY? + gnu_srs: server functions are only called from user space... + gnu_srs: Normally, servers use a server loop. When an RPC comes it, + it is processed and the processing returns some values that the server + loop returns to the client. + If the server wants to return the thread to the thread pool without + responding to the RPC, it uses MIG_NO_REPLY. -- cgit v1.2.3