From 49a086299e047b18280457b654790ef4a2e5abfa Mon Sep 17 00:00:00 2001 From: Samuel Thibault Date: Wed, 18 Feb 2015 00:58:35 +0100 Subject: Revert "rename open_issues.mdwn to service_solahart_jakarta_selatan__082122541663.mdwn" This reverts commit 95878586ec7611791f4001a4ee17abf943fae3c1. --- open_issues/hurd_101.mdwn | 360 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 360 insertions(+) create mode 100644 open_issues/hurd_101.mdwn (limited to 'open_issues/hurd_101.mdwn') diff --git a/open_issues/hurd_101.mdwn b/open_issues/hurd_101.mdwn new file mode 100644 index 00000000..e55b0e8e --- /dev/null +++ b/open_issues/hurd_101.mdwn @@ -0,0 +1,360 @@ +[[!meta copyright="Copyright © 2011, 2013, 2014 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +(See Wikipedia page for the meaning of [[!wikipedia "101_(term)"]].) + +Not the first time that something like this is proposed... + + +# IRC, freenode, #hurd, 2011-07-25 + + [failed GNU/Hurd project] + < antrik> gnu_srs1: I wouldn't say he was on track. just one of the many + many people who insist on picking a hard task; realizing that indeed it's + hard; and going into hiding + < antrik> we see that happen every couple of months + < cluck> maybe we need a "hurd 101" + < cluck> getting a teacher and setting up a regularly held "class" for hurd + noobs + < Tekk_> cluck: what would that include? + < cluck> explaining core concepts, giving out "homework" (small tasks), etc + +[[Anatomy_of_a_Hurd_system]]. + + < cluck> that way "the big guys" could focus on the hard stuff and have an + army of code monkeys at their disposal to write speced stuff + < cluck> (then again this idea would heavily depend on available "teachers" + and "students", which, going by gsoc numbers, may not be all that + helpful) + < Tekk_> cluck: gsoc isn't an accurate indicator + < Tekk_> cluck: I'm not allowed to participate in gsoc but I'd join :P + < antrik> cluck: we don't need code monkeys... we need hackers + < Tekk_`> antrik: code monkeys involve into hackers + < Tekk_`> under the right conditions + < cluck> antrik: jokes aside some sort of triage system/training ground for + newcomers could be helpful + + +# IRC, freenode, #hurd, 2013-01-20 + + so once I have written my first translators, and really understand + that, what kinds of projects would you recommend to an operating + systems/hurd newbie. + I am reading the minix book now as I have it, but I'm waiting on + getting the modern operating systems book by the same author. + I was initially going to start working on minix, but their focus + seems to be on embedded, and I want to work on a system that is more + general purpose, and I like the philosophy of freedom surrounding the + hurd. + I like how the hurd design allows more freedom for users of the + operating system, but I would also like to incorporate ideas from minix + on the hurd. mainly, rebootless updates of servers/translators. + then you should study how translators work + how ipc works + and understand exactly what state is stored where + ok + + +# IRC, freenode, #hurd, 2013-10-12 + + Hi all, can anyone expand on + https://www.gnu.org/software/hurd/contributing.html - if I proceed with + the quick start and have the system running in a virtual image, how do I + go from there to being able to start tweaking the source (and recompiling + ) in a meaningful way? + Would I modify the source, compile within the VM and then what + would be the next step to actually test my new changes? + ahungry: we use debian + i suggest formatting your changes into patches, importing them + into debian packages, rebuilding those packages, and installing them over + the upstream ones + what about modifications to mach itself? or say I wanted to try + to work on the wifi drives - I would build the translator or module or + whatever and just add to the running instance of hurd? + s/drives/drivers + same thing + although + during development, it's obviously a bit too expensive to rebuild + complete packages each time + you can use the hurd on top of a gnumach kernel built completely + from upstream sources + you need a few debian patches for the hurd itself + a lot of them for glibc + i usually create a temporary local branch with the debian patches + i need to make my code run + and then create the true development branch itself from that one + drivers are a a dark corner of the hurd + i wouldn't recommend starting there + but if you did, yes, you'd write a server to run drivers, and + start it + you'd probably write a translator (which is a special kind of + server), yes + braunr: thanks for all the info, hittin the sack now but ill have + to set up a box and try to contribute + + +# Documentation + +## IRC, freenode, #hurd, 2013-11-04 + + i think the problem my hurd have not more developers or + contubutors is the project idears and management , eg, the most problem + is the mach kernel and documatation and the missing subsystem goals + (driver, etc) + no i think you and other have a clue but this is not + tranzparent when i read the webpage + well, fwiw I agree, the documentation is lacking + about what ? + something that doesn't exist ? + like smp or a generic device driver framework ? + no, high level concepts, design stuff + what ? + how come ? + not even the gnumach documentation is complete + for example ? + see http://www.sceen.net/~rbraun/doc/mach/ + which is my personal collection of docs on mach/hurd + and it's lacking at least one paper + well two, since i can't find the original article about the hurd + in pdf format + project ideas are clearly listed in the project ideas page + braunr: do you think the mach kernel decumatation a compleat? + and you think its good documentatition about "how write a drive for mach" + and you think a answare is found why dont work smp and why is have no + arm, x64 support ? + stargater: + http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/ + the page is even named "project ideas" + the mach kernel is probably the most documented in the world + even today + and if there is no documentation about "how to write drivers for + mach", that's because we don't want in kernel drivers any more + and the state of our driver framework is practically non existent + it's basically netdde + partial support for network drivers from linux + that's all + we need to improve that + someone needs to do the job + noone has for now + that's all + why would we document something that doesn't exist ? + only stupid project managers with no clue about the real world do + that + (or great ones who already know everything there is to know before + writing code, but that's rare) + stargater: the answer about smp, architectures etc.. is the same + spirit and magic are nice ;-) braunr sorry, that is only my + meanig and i will help, so i ask and say what i think. when you say, hurd + and mach are good and we on the right way, then its ok for me . i wonder + why not more developer help hurd. and i can read and see the project page + fro side a first time user/developer + i didn't say they're good + they're not, they need to be improved + clearly + ok, then sorry + i wondered about that too, and my conclusion is that people aren't + interested that much in system architectures + and those who are considered the hurd too old to be interesting, + and don't learn about it + consider* + stargater: why are you interested in the hurd ? + that's a question everyone intending to work on it should ask + the spirit of free software and new and other operation system, + with focus to make good stuff with less code and working code for ever + and everone can it used + well, if the focus was really to produce good stuff, the hurd + wouldn't be so crappy + it is now, but it wasn't in the past + a good point whas more documentation in now and in the future, + eg, i like the small project http://wiki.osdev.org/ and i like to see + more how understanding mach and hurd + I love osdev much, it taught me a lot ;-D + osdev is a great source for beginners + teythoon: what else did you find lacking ? + braunr: in my opinion the learning curve of Hurd development is + quite steep at the beginning + yes, documentation exists, but it is distributed all over the + internets + teythoon: hm ok + yes the learning curve is too hard + that's an entry barrier + + +# IRC, freenode, #hurd, 2014-02-04 + +[[!tag open_issue_documentation]] + + Does the GNU Mach kernel have concepts of capabilities? + yes + see ports, port rights and port names + Does it follow the take grant approch + approach* + probably + Can for example I take an endpoint that I retype from untyped + memory and mint it such that it only has read access and pass that to the + cspace of another task over ipc. + Where that read minted cap enforces it may onnly wait on that ep. + ep ? + ah + Endpoint. + probably + Alright cool. + it's a bit too abstract for me to answer reliably + ports are message queues + port rights are capabilities to ports + Not sure exactly how it would be implemented but essentially you + would have a guarded page table with 2 levels, 2^pow slots. + port names are integers referring to port rights + we don't care about the implementation of page tables + Each slot contains a kernel object, which in itself may be more + page tabels that store more caps. + it's not l4 :p + mach is more of a hybrid + It isn't a page table for memory. + it manages virtual memory + Ah ok. + whatever, we don't care about the implementation + So if I want to say port an ethernet driver over. + whether memory or capabilities, mach manages them + Can I forward the interrupts through to my new process? + yes + it has been implemented for netdde + these are debian specific patches for the time being though + Great, and shared memory set ups are all nice and dandy. + yes, the mach vm takes care of that + Can I forward page faults? + Or does mach actually handle the faults? + (Sorry for so many questions just comparing what I know from my + microkernel knowledge to mach and gnu mach) + mach handles them but translates them to requests to userspace + pagers + (Still have a mach paper to read) + Alright that sounds sane. + Does GNU mach have benchmarks on its IPC times? + no but expect them to suck :) + Isn't it fixable though? + mach ipc is known to be extremely heavy in comparison with modern + l4-like kernels + not easily + Yeah so I know that IPC is an issue but never dug into why it is + bad on Mach. + So what design decision really screwed up IPC speed? + for one because they're completely async, and also because they + were designed for network clusters, meaning data is typed inside messages + Oh weird + So how is type marshalled in the message? + in its own field + messages have their own header + and each data field inside has its own header + Oh ok, so I can see this being heavy. + So the big advantage is for RPC + It would make things nice in that case. + Is it possible to send an IPC without the guff though? + Or would this break the model mach is trying to achieve? + I am assuming Mach wanted something where you couldn't tell if a + process was local or not. + So I am assuming then that IPC is costly for system calls from a + user process. + You have some sort of blocking wait on the call to the service + that dispatches the syscall. + I am assuming the current variants of GNU/Hurd run on glibc. + It would be interesting to possibly replace that with UlibC or do + a full port of the FlexSC exceptionless system calls. + Could get rid of some of the bottlenecks in hurd assuming it is + very IPC heavy. + And that won't break the async model. + Actually should be simpler if it is already designed for that. + But would break the "distributed" vibe unless you had the faults + to those shared pages hit a page faulter that sent them over the network + on write. + + bwright: a lot of POSIX compatibility is handled by the glibc, + "porting" another libc to the Hurd will be a titanic task + In theory exceptionless system calls work fine on glibc, it is + just harder to get them working. + has not been done or was not explored in the paper. + Something about it having a few too many annoying assumptions. + Would be interesting to run some benchmarks on hurd and figure + out where the bottlenecks really are. + At least for an exercise in writing good benchmarks :P + I have a paper on the design of hurd I should read actually. + After I get through this l4 ref man. + the main bottleneck is scalability + there are a lot of global locks + and servers are prone to spawning lots of threads + because, despite the fact mach provides async ipc, the hurd mostly + uses sync ipc + so the way to handle async notifications is to receive messages + and spawn threads as needed + Lets take a senario + beyond that, core algorithms such as scanning pages in pagers, are + suboptimal + I want to get a file and send it across the network. + How many copies of the data occur? + define send + ouch :) + disk drivers are currently in the kernel + I read a block from disk, I pass this to my file system it passes + it to the app and it sends to the lwip or whatever interface then out the + ethernet card. + and "block device drivers" in userspace (storeio) are able to + redirect file system servers directly to those in kernel drivers + so + kernel -> fs -> client -> pfinet -> netdde (user space network + drivers on debian hurd) + Alright. Hopefully each arrow is not a copy :p + it is + My currently multiserver does this same thing with zero copy. + because buffers are usually small + yes but zero copy requires some care + Which is possible. + and usually, posix clients don't care about that + Yes it requires a lot of care. + POSIX ruins this + Absolutely. + they assume read/write copy data, or that the kernel is directly + able to access data + But there are some things you can take care with + And not break posix and still have this work. + pfinet handles ethernet packets one at a time, and 1500 isn't + worth zero copying + This depends though right? + i'm not saying it's not possible + i'm saying most often, there are copies + So if I have high throughput I can load up lots of packets and + the data section can then be sectioned with scatter gather + again, the current interface doesn't provide that + Alright yeah that is what I expected which is fine. + It will be POSIX compliant which is the main goal. + not really scatter gather here but rather segment offloading for + example + ah you're working on something like that too :) + Yeah I am an intern :) + Have it mostly working, just lots of pain. + Have you read the netmap paper? + Really interesting. + not sure i have + unless it has another full name + 14.86 million packets per second out of the ethernet card :p + SMOKES everything else. + Implemented in Linux and FreeBSD now. + Packets are UDP 1 byte MTU I think + 1 byte data * + To be correct :p + right, i see + Break posix again + "More Extend" + i've actually worked on a proprietary implementation of such a + thing where i'm currently working + Bloody useful for high frequency trading etc. + Final year as an undergraduate this year doing my thesis which + should be fun, going to be something OS hopefully. + Very fun field lots of weird and crazy problems. -- cgit v1.2.3