diff options
Diffstat (limited to 'community')
84 files changed, 4606 insertions, 484 deletions
diff --git a/community/gsoc.mdwn b/community/gsoc.mdwn index ce3b26fb..efd29841 100644 --- a/community/gsoc.mdwn +++ b/community/gsoc.mdwn @@ -1,53 +1,110 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011, 2012 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="Google Summer of Code"]] -# 2009 +We're in! The GNU Hurd project is again participating in the [Google Summer of +Code](http://www.google-melange.com/) under the [GNU +umbrella](http://www.gnu.org/software/soc-projects/). -The GNU Hurd project this year unfortunately is not in the list of accepted -mentoring organizations. +<!-- -However, don't despair! This doesn't mean that you'd not be allowed to work on -your favorite GNU Hurd project. For one, there's the possibility that a slot -for your project can be allocated under the [auspices of the GNU -project](http://www.gnu.org/software/soc-projects/). Second, we're of course -always open to introducing newcomers to the world of the GNU Hurd outside of -any Google Summer of whatever project. Just pick a task from the list pointed -to below on this page and get in touch with us! +This year's *student application period* is over. Thanks for sending in your +applications! We're now reviewing and discussing these, so please pay +attention to any questions posted on your proposal's page. The Google site's +notification system should be sending out emails, too. -<!-- The GSoC 2009 student application time has come to an end -- we -are now evaluating your applications. --> +As we only have finite resources (meaning that we won't be able to accept all +GNU Hurd applications even if we wanted to), we will eventually need to make a +choice about whom to select. For this, it is a very good idea to be in contact +with us, be it by answering the evaluators' questions on your proposal's page, +or by talking to us on the [[mailing_lists]] or on [[IRC]]. At this time, it +is important for us to get a good impression about the seriousness you're +showing with your application. -The applications have been evaluated and the following student has -been accepted: +It is a good idea to get familiar with the GNU Hurd, by reading some of our +[[documentation]], and by using a GNU/Hurd system. It is also a good idea to +send in some basic patches (this has already been mentioned in our +[[student_application_form]]), or discuss with us the principal steps you're +planning on doing in your intended work area. Of course, we don't expect you +to already start working seriously on your project, but any input you're giving +us will make it easier for us to justify selectiong your specific proposal. At +this time, it is not quantity that matters, and it also is not *the perfect +patch* we're waiting for, but it is rather that we see how you're generally +able to work with the code. - * [[Sergiu Ivanov|scolobb]] +If you have any questions, don't be shy: please ask! Nobody expects you to +know everything. Even for the long-term Hurd contributors it is common to +openly post messages to [[mailing_lists/bug-hurd]] saying: *Hey, I don't know +how to do `X`, can someone please help me?* And, as we're not working next to +each other in a conventional office or university setup, we'll need to +establish and get used to different communication channels. +[Timeline](http://www.google-melange.com/gsoc/events/google/gsoc2011). As +boring as it is, but the next step is waiting: we will have to wait for Google +to announce the number of slots that the whole GNU project gets, and we'll be +discussing with our GNU peers about how to split these up among all the GNU +subprojects. -# History +--> + + +Applications for 2012 are closed. + +# Accepted projects + +## Disk I/O Performance Tuning + +by Maksym Planeta + +See the project's +[public page](http://www.google-melange.com/gsoc/project/google/gsoc2012/mcsim/46002). -2006 and 2007 we participated in GSoC under the umbrella of the GNU project, -getting one slot each year. +## Virtualization Using Hurd Mechanisms -<!-- TODO. Extend. --> +by Pierre Thierry -In 2008 we successfully participated on our own, no longer within the GNU -project. Read about our five students' success on the [[2008]] page. +See the project's +[public page](http://www.google-melange.com/gsoc/project/google/gsoc2012/nowhereman/36001) +and [[complete proposal|gsoc/2012/virt/proposal]]. -# Joining in +# Possible projects -If these successes got you interested in contributing some larger part yourself - -in your free time or maybe in next years Google Summer of Code - -please have a look at our [[project_ideas]] and read up about [[contributing]]. +We have a list of [[project_ideas]], and students are likewise encouraged to +submit their own project proposals. Please follow our +[[student_application_form]]. + +Then, don't forget to visit <http://www.google-melange.com/>, and press the +button for submitting your proposal. + +Please read up about [[contributing]] in general, and please ask any questions +you might have, on the [[mailing_lists]], or on [[IRC]], for example at one of +our [[regular_IRC_meetings|IRC#regular_meetings]]. Generally it's a good idea +to [[get in contact with us|contact_us]] as soon as you're beginning to spend +time on a project. + + +## Outside of the GSoC Scope + +Working on one of these projects is generally a good opportunity to get started +with Hurd development, even outside of the GSoC context. Please don't hesitate +to contact us regarding mentoring even if it's not GSoC time at the moment, or +if you aren't a student anyway. + +# History -Also, feel free to ask your questions at one of our -[[regular_IRC_meetings|IRC#regular_meetings]]. +In 2006 and [[2007]], we participated in GSoC under the umbrella of the GNU +project, getting one slot each year. In the following year, we successfully +participated on our own, instead of as a suborganization of the GNU project. +Read about our five students' success on the [[2008]] page. The next two year, +we participated under the GNU umbrella with one slot in [[2009]], three in +[[2010]], and one again in [[2011]]. diff --git a/community/gsoc/2007.mdwn b/community/gsoc/2007.mdwn new file mode 100644 index 00000000..fa22f785 --- /dev/null +++ b/community/gsoc/2007.mdwn @@ -0,0 +1,16 @@ +[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +Participated under the umbrella of the GNU project, getting one slot: + + * Carl Fredrik Hammar has been working on [[hurd/libchannel]]; [Google's + project + page](http://code.google.com/soc/2007/gnu/appinfo.html?csaid=44FB3FD69C665A08). diff --git a/community/gsoc/2008.mdwn b/community/gsoc/2008.mdwn index d7b467bb..d994f2b0 100644 --- a/community/gsoc/2008.mdwn +++ b/community/gsoc/2008.mdwn @@ -23,7 +23,8 @@ did a great job! vacation). The project however was hampered by various misunderstandings, wrong assumptions, and several major redesigns during the course of the work -- which is probably more our fault than the student's. In the end, though, he - completed nsmux (the main namespace proxy handling the magic filename + completed [[hurd/translator/nsmux]] (the main namespace proxy handling the + magic filename lookups, running dynamic translators on demand); he still works on finishing the translator stack filtering necessary to implement some of the desired functionality (accessing files while skipping existing translators). diff --git a/community/gsoc/2009.mdwn b/community/gsoc/2009.mdwn new file mode 100644 index 00000000..6efeb839 --- /dev/null +++ b/community/gsoc/2009.mdwn @@ -0,0 +1,23 @@ +[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled +[[GNU Free Documentation License|/fdl]]."]]"""]] + +[[!meta title="Google Summer of Code 2009"]] + +The GNU Hurd project unfortunately was not in the list of accepted +mentoring organizations. + +We did get one slot from the GNU project's. + +The applications have been evaluated and the following student has +been accepted: + + * [[Sergiu Ivanov|scolobb]] -- working on a *[[VFS-style union + mount|hurd/translator/unionmount]]* functionality; [Google's project + page](http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2009/karlberry/t124022551214). diff --git a/community/gsoc/2010.mdwn b/community/gsoc/2010.mdwn new file mode 100644 index 00000000..d09e26b6 --- /dev/null +++ b/community/gsoc/2010.mdwn @@ -0,0 +1,35 @@ +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011, 2012 Free Software +Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="Google Summer of Code 2010"]] + +In 2010 we have again been participating in the Google Summer of Code +under the GNU umbrella, with three slots: + + * *[[Jérémie Koenig|jkoenig]]*, mentored by *[[Samuel + Thibault|samuelthibault]]*, has been working on adapting the Debian Installer to + [produce working Debian GNU/Hurd installation + images](http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2010/debian/t127230758239) + so we can easily offer up to date disc-sets. + ([Details](http://wiki.debian.org/SummerOfCode2010/HurdDebianInstaller/JeremieKoenig).) + + * *[[Emilio Pozuelo Monfort|pochu]]*, mentored by *Carl Fredrik Hammar* (who + was a GSoC student in [[2007]]), has been working on a task that may be perceived as + less exciting from the outside, but yet is extremely valuable: [fixing + compatibility problems exposed by projects' + testsuites](http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2010/gnuproject/t127230759396). + ([[Details|community/gsoc/project_ideas/testsuites]].) + + * *[[Karim Allah Ahmed|kam]]*, mentored by *Sergio López*, has been working on + [tuning the VM Subsystem in + GNU/Hurd](http://socghop.appspot.com/gsoc/student_project/show/google/gsoc2010/gnuproject/t127230759587) + to bring the virtual memory management in Hurd/Mach up to date. + ([[Details|community/gsoc/project_ideas/vm_tuning]].) diff --git a/community/gsoc/2011.mdwn b/community/gsoc/2011.mdwn new file mode 100644 index 00000000..ba10a031 --- /dev/null +++ b/community/gsoc/2011.mdwn @@ -0,0 +1,19 @@ +[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="Google Summer of Code 2011"]] + +In 2011 we have again been participating in the Google Summer of Code +under the GNU umbrella, with one slot: + + * [[Jérémie Koenig|jkoenig]], mentored by [[Thomas Schwinge|tschwinge]] and + Richard Braun, has been working on the project [*Java on Hurd (and vice + versa)*](http://www.google-melange.com/gsoc/project/google/gsoc2011/jkoenig/27001). + ([[Details|user/jkoenig/java]].) diff --git a/community/gsoc/2012/virt/discussion.mdwn b/community/gsoc/2012/virt/discussion.mdwn new file mode 100644 index 00000000..e0085322 --- /dev/null +++ b/community/gsoc/2012/virt/discussion.mdwn @@ -0,0 +1,389 @@ +[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + + +# IRC, freenode, #hurd, 2012-07-19 + + <nowhere_man> well, I really actively started last week, so I'm ironing my + various use cases and above all I'm taking my barings in Hurd's code + <nowhere_man> I'm currently reading boot/ and pfinet/ + <braunr> sorry for asking but + <braunr> can you describe brielfy what you mean to achieve + <braunr> i know it sounds weird but the project description is a bit vague + for me + <nowhere_man> OK + <nowhere_man> the main goal is to be able to easily spawn a subhurd that's + connected in some way to its host + <braunr> ok + <nowhere_man> mainly connected by network, possibly sharing resources like + FS + <braunr> is it similar in spirit with something like linux containers ? + <nowhere_man> IIRC about them, yes + <braunr> ok + <braunr> that will do for me then + <tschwinge> Yes, so not complete virtualization, but instaed limitied to + several components. + <braunr> lxc with more runtime features to increase/decrease the level of + isolation + <nowhere_man> at first it would be static, at creation time only + <braunr> ok, i clearly understand the proposal now :) + <braunr> what kind of help could you need in the near future ? + <braunr> (except permanent access to youpi's brain?) + <tschwinge> Yes, that's my question, too -- what can we do to "get this + thing going". + <nowhere_man> by monday or tuesday I should be clear on what I understand + or not in the code + <nowhere_man> I'm still a bit up to my elbows in it + <nowhere_man> at that point I'll be happy to be able to pop a lot of + questions about it + <braunr> so you'll be ready for the next meeting + <nowhere_man> yeah + <tschwinge> Please do as soon as there are questions that you cannot + resolve in a reasonably short amount of time. + <tschwinge> So often a quick hint from someone else already helps to ge + un-stuck. + <nowhere_man> OK + <tschwinge> There is no problem with asking for help given this huge and + convoluted code-base, where often design decisions are not obvious, too. + <nowhere_man> I will + <tschwinge> Good. :-) + <antrik> nowhere_man: hm... what you said so far doesn't sound any + different than the work zhengda already did on boot years ago... + <antrik> (although none of it ever got upstream IIRC :-( ) + <nowhere_man> antrik: wasn't aware of it, is there some code published? + <tschwinge> There are bits and pieces, but certainly there is enough work + left to be done, to put it all together. + <antrik> yes, his git repository should be up somewhere. it's quite + convoluted though, as he worked on several things, and also wasn't very + experienced with revision control in the beginning + <tschwinge> nowhere_man: + http://www.gnu.org/software/hurd/community/gsoc/2008.html + <tschwinge> nowhere_man: http://www.gnu.org/software/hurd/user/zhengda.html + <tschwinge> Second section of the latter one. + <antrik> well, my understanding of the proposal (and more or less what I + was driving at in the project idea, which is rather vague admittedly) is + something lighter than a real subhurd... rather some kind of thin + subenvironment that doesn't actually boot a complete system instance with + various daemons etc. + <tschwinge> nowhere_man: It is certainly valid for you to use pre-existing + code/patches, by the way. + <antrik> BTW, regarding the "full subhurd" thing, the missing pieces are + mostly virtual device implementations + <antrik> (that and some tough bug(s) remaining in zhengda's modified + boot...) + <nowhere_man> cool, I'll take a look + <antrik> in any case, getting a picture of the work zhengda did is, is + definitely the first thing to do :-) + <tschwinge> nowhere_man: I'll also try to locate some bits and stuff from + his verious repositories (I just fond a Subverision one; will convert to + Git). + <antrik> tschwinge: I'm pretty sure zhengda's git repository was converted + from the SVN one... + <tschwinge> antrik: Thanks for reminding us about this -- I failed to + remember all that. + <antrik> (which was in turn converted from CVS...) + <tschwinge> antrik: OK, will have a lot. + <tschwinge> Yeah, found a CVS tree, too. ;-) + <antrik> BTW, zhengda's work more exactly was about subhurd without root + privileges. but that lays a lot of the groundwork for all kinds of more + flexible subhurd usage + <antrik> (but it's still quite a different thing that thing + subenvironments, so don't get confused...) + <antrik> err... thin subenvironments + + +# IRC, freenode, #hurd, 2012-07-27 + + <nowhere_man> bddebian: I'm actually not progressing much while reading the + source, I'm jumping all over the place to grasp the various types and + functions used where I start + <nowhere_man> would there be a few starting points that could help me? + <tschwinge> nowhere_man: So what exactly is your status; what are you + doing, what do you need help with? We surely can provide help, but need + to know where. + <nowhere_man> I'm starting from the source of boot/ and pfinet/ and as soon + as I encounter something that I don't understand, I find its definition + <nowhere_man> I'm kind of doing a depth-first search of what I need to + understand in the source code + <nowhere_man> I'm wondering if there are a few places in the source code + that I should start reading before anything else + <nowhere_man> well, I'll have to go in a few minutes + <nowhere_man> I'll continue my DFS ;-) + + +# IRC, freenode, #hurd, 2012-08-02 + + <nowhereman> well, I made a leap forward in understanding the code, when I + stopped my DFS + <nowhereman> in hindsight, I'd say my way of approaching the code was + probably one of the worst possible + <braunr> oh + <tschwinge> OK, so at least you learned something, which is good. + <tschwinge> So, what's the new approach? And what are you working on at + the moment + <tschwinge> ? + <nowhereman> I just remembered SICP, the idea of wishful thinking when you + code, and didn't bother with the fine details behind what I'm reading + <nowhereman> like, I don't really get what happens when a Mach port is + allocated, but I know approximately what a Mach port is + <tschwinge> So originally you worked on investigating all that, every line + of code? + <nowhereman> almost, yeah + <braunr> nowhereman: again, feel free to ask + <tschwinge> Yes indeed -- that's too complex for a single person to tackle + at one time. + <braunr> and quickly + <braunr> don't loose time + <tschwinge> Not even braunr and I have looked up all these things. + (Speaking for Richard here, but I'm quite sure he'll agree. Perhaps he + has in fact looked up all the Mach things, though.) + <tschwinge> nowhereman: ufc? + <nowhereman> BTW, last week I wanted to push my description of how the tool + could be used, the use cases + <nowhereman> ufs + <nowhereman> but flubber is not online + <tschwinge> nowhereman: Oh, why ufs specifically? + <braunr> don't waste time on ufs + <braunr> really + <tschwinge> nowhereman: Yes, flubber is down. But you can push directly to + the Savannah repository. + <tschwinge> nowhereman: Please immediatelly tell us if you're stuck on + something, like flubber not being available. + <tschwinge> We may not be able to help immediatelly, but we're the at least + aware of issues. + <braunr> and we may be able to help immediately :) + <tschwinge> As we're not sitting in a lab next to each other, we can't tell + otherwise what's going on. + <tschwinge> We may in fact even be able to tell you immediatelly to use + Savannah instead of flubber, indeed. + <tschwinge> nowhereman: So, back to ufs -- which you don't specifically + need to look at, I think -- ext2fs is what everyone uses. But even there + you shouldn't really need to know many details/internals. + <nowhereman> OK, I was looking into it has it appears in hurd.boot + <tschwinge> Ah, OK. Yeah, that's just an example/template, and should use + ext2fs nowadays. + <nowhereman> in fact, as far as FS are concerned, I suppose I will merely + need to know how to pass a port to the host's FS to some proxy FS in the + subhurd + <nowhereman> mmmh, Savannah only mentions a hurd.git + <tschwinge> Exactly that is the abstraction level you need, yes. + <nowhereman> I'm looking at http://savannah.gnu.org/git/?group=hurd + <tschwinge> Yeah, that's a known shortcoming -- look here instead: + http://git.savannah.gnu.org/cgit/hurd + <tschwinge> Here is some more up-to-date stuff on subhurds: + http://www.gnu.org/software/hurd/hurd/subhurd.html + <tschwinge> nowhereman: You know how to tell git to add a new remote to + your web pages checkout and such stuff? + <nowhereman> yeah, no problem with that + <braunr> have you prepared any question to ask us ? + <nowhereman> the only I have now is if you can tell me where to look in the + code about passing Mach ports + <braunr> you don't pass ports, you pass rights + <braunr> http://www.gnu.org/software/hurd/gnumach-doc/index.html is the + best location to have a look at + <braunr> + http://www.gnu.org/software/hurd/gnumach-doc/Exchanging-Port-Rights.html#Exchanging-Port-Rights + <braunr> i suppose the mig doc will help too, as you may be using a higher + level interface to exchange rights + <braunr> be careful about user references on port rights + <braunr> deallocate releases a reference, it doesn't immediately destroy a + resource + <braunr> portinfo -v can help monitoring a task's rights + <braunr> nowhereman: so what are you planning to do now ? + <braunr> during the next week + <nowhereman> documenting what I understand from the boot process and where + things can be changed to fit my various use cases + <braunr> do you expect that to take the whole week ? + <nowhereman> and doing some first modifications to servers for the simplest + cases + <braunr> ok + <braunr> well i hope you're able to really start working on it soon, and + won't face weird issues in the meantime + <braunr> i'm a bit disappointed that you don't have more questions + <braunr> my feeling is you either did understand everything (except passing + port rights), or you didn't attempt to seriously understand the code + <braunr> or you don't dare ask questions + <braunr> this is something that must change + <braunr> or these meetings won't be as useful as they could be + <tschwinge> Yes. But also please don't wait for the meetings, but ask + questions throughout the week, too. + + +# IRC, freenode, #hurd, 2012-08-09 + + <nowhere_man> hey, does anyone knows the network device interface well? + <nowhere_man> I don't get it by reading net_io.c/h in gnumach + <braunr> nowhere_man: ask your question + <braunr> nowhere_man: http://www.sceen.net/~rbraun/pcap-hurd.c <- this may + help + <nowhere_man> I don't see what the entry point is + <nowhere_man> I finally understood that I actually don't need to touch + pfinet for gsoc project + <nowhere_man> but I should do a replacement network device instead + <nowhere_man> is the net_io_init function called at start? + <braunr> what entry point ? + <braunr> and you should perhaps have a look at the eth-multiplexer by + zhengda + <braunr> yes net_io_init is called at startup + <braunr> nowhere_man: did you find your answers about networking ? + <nowhere_man> no, I'm still digging in mach's code + <braunr> nowhere_man: well keep asking :/ + <braunr> you left conversation without notice :/ + <braunr> nowhere_man: and why mach ? + <nowhere_man> I thought hardware devices are there + <tschwinge> nowhere_man: You wanted to push your documentation one/two + weeks ago. Why has that not yet happened? + <youpi> nowhere_man: they used to be there, they are now in netdde, but in + both case it's just a matter of the same RPC interface + <nowhere_man> tschwinge: I spent very few time this week on gsoc, and + completely forgot about the push on savannah + <braunr> nowhere_man: i told you to look at the work by zhengda concerning + eth-multiplexer, did you do that ? + <tschwinge> nowhere_man: You realize GSoC is meant to be a full-time job? + <tschwinge> Or, next to full-time? + <braunr> it's full-time normally + <braunr> the payment is justified by that + <youpi> nowhere_man: most RPC operations you need to know about network can + be seen at work in pfinet/ethernet.c, wherever "ether_port" appears + <youpi> i.e. device_open, set_filter, write, set/get_status + <braunr> again, http://www.sceen.net/~rbraun/pcap-hurd.c should guide you + pretty well + <braunr> since it's the very least necessary to use that interface + <tschwinge> nowhere_man: How, roughly but realistically, are your plans to + continue this task? + <tschwinge> nowhere_man: What has been blocking you this week so you + couldn't work on your task? + <nowhere_man> tschwinge: mostly a previous work that was supposed to end at + the beginning of the summer and only went online now, for which I'm + basically sysadmin + <braunr> 21:25 < tschwinge> nowhere_man: How, roughly but realistically, + are your plans to continue this task? + <braunr> this question is really more interesting actually + <nowhere_man> right now, I want to write a netword device that just sends + its frames by IPC + <braunr> why ? + <nowhere_man> as I never wrote any program using Mach's IPC, that seems the + easiest to get them right + <braunr> you won't have time + <braunr> 21:22 < braunr> nowhere_man: i told you to look at the work by + zhengda concerning eth-multiplexer, did you do that ? + <nowhere_man> braunr: not yet, no + <braunr> well that's your best chance to make some progress + <nowhere_man> braunr: is writing the virtal network device that hard? + <braunr> basically, it allows "bridgind" the pfinet instances of various + subhurds + <braunr> the virtual network device you want *is* eth-multiplexer + <tschwinge> nowhere_man: GSoC is nearly over. That's why I'm asking how + this task is going to continue. I'm sorry but I reckon you have not + spend anywhere near the amount of hours that are meant to be spent on it. + <braunr> and from what antrik told me, yes it's hard, and moreover, why + rewrite it if it already exists and you're late + <braunr> i agree + <nowhere_man> tschwinge: I know, I've started way too late because of my + second round of exams + <tschwinge> nowhere_man: OK, that's how you started. But how is it going + to continue... + <nowhere_man> tschwinge: in short, I write a prototype that just starts a + subhurd, and when that works correctly I add the network + <tschwinge> nowhere_man: I mean from an organizational point of view. + <nowhere_man> well, between now and the beginning of september, I'll work + full-time on this + <nowhere_man> up until september 8th + + +# IRC, freenode, #hurd, 2012-08-09 + + <antrik> nowhere_man: you do *not* have to do a replacement network + device. zhengda did that years ago. + <antrik> nowhere_man: also note that zhengda also implemented the support + for *using* the virtual network device (in fact any replacement devices + -- except that no others actually exist yet) in boot + <youpi> which is already in, actually, isn't it? + <antrik> youpi: hm, yes... it was the patch that zhengda posted on the list + once, but later updated, and at some later point you merged the outdated + variant from the list... + <youpi> outdated? + <youpi> ah, but he never posted the updated one, and it got lost in git + repos, right? + <youpi> (what was updated actually?) + <antrik> he changed the option name and description later for more + clarity. don't remember whether there were other changes + <antrik> -f, --device=device_name=device_file + <antrik> Specify a device file used by subhurd + and its + <antrik> virtual name. + <antrik> that's the one from the Debian package + <antrik> -m, --device-map=DEBICENAME=DEVICEFILE + <antrik> Map the device in subhurd to the + device in the + <antrik> main Hurd. + <antrik> that's the one I have locally built from his tree + <youpi> so you actually have access to his tree? + <antrik> uhm... I used to... it was on flubber + + +# IRC, freenode, #hurd, 2012-08-18 + + <nowhere_man> so, this week I discovered how fun it is to work on a + non-mainstream OS + <nowhere_man> I hoped to start coding the tool itself, put together the + skeleton, but every Lisp implementation I tried had problems + <braunr> ah you want to write it in lisp ? + <nowhere_man> ECL, that I had ported a few years ago, actually FTBFS since + <nowhere_man> I hoped to be able, it would be easier for me + <nowhere_man> and when I tried Scheme, I started with Guile (it's GNU's own + Scheme implementation, after all) + <nowhere_man> and when I execute the FFI functions, to access functions in + libmachdec + <nowhere_man> I get SIGILL + <braunr> i can't advise you about anything lisp related + <braunr> the most reliable thing you'll find on the hurd is C + <nowhere_man> I tried to debug that, but running Guile in GDB gets me a + SIGSEV + <nowhere_man> I'll try to make ECL to build again + <braunr> this seems like a waste of time to me + <braunr> avoid spending time on anything that isn't directly related to + your goal if you still hope to finish it + <nowhere_man> I'm ten times more comfortable coding in Lisp + <braunr> it doesn't matter, you're late + <nowhere_man> yeah, I know, so taking the time to correct that problem + won't change the fact that I won't finish in time + <nowhere_man> so I'll finish anyway, and in Lisp + <braunr> and if you lack something else, like some mach/hurd specific lisp + bindings, you'll have to spend more time on that + <braunr> ok + <nowhere_man> do you know if someone had a SIGILL situation on Hurd in the + past? + <nowhere_man> I'm wondering if that's a known kind of issue + <braunr> there are lots of issues + <braunr> especially when it comes to other languages and runtime + environments + <nowhere_man> but is it like MAX_PATH_LEN, something that is known to + happen when porting something on Hurd? + <braunr> i'm not sure how comparable it is + <braunr> i'd say it's often before of the conformance issues of the hurd + <braunr> because* + <nowhere_man> like missing bits of POSIX ? + <braunr> or simple wrong for some corner cases + <braunr> simply* + <bubu^> nowhere_man, I was able to run guile on my hurd image through qemu + <bubu^> but I didn't make any complexe programms to check if everything + works fine + <nowhere_man> yeah, it runs fine + <nowhere_man> FFI functions get you a SIGILL + <nowhere_man> + http://www.gnu.org/software/guile/manual/html_node/Dynamic-FFI.html + <nowhere_man> the define-module form at the beginning triggers the signal + <antrik> nowhere_man: what do you want to implement in Lisp? + <antrik> BTW, the guy working on Lisp bindings a couple of years ago used + Clisp + <antrik> it was working back then + <nowhere_man> antrik: the program that sets up a subhurd + <nowhere_man> I always forget about clisp, I'll try it right away diff --git a/community/gsoc/2012/virt/proposal.mdwn b/community/gsoc/2012/virt/proposal.mdwn new file mode 100644 index 00000000..d89f45d5 --- /dev/null +++ b/community/gsoc/2012/virt/proposal.mdwn @@ -0,0 +1,95 @@ +[[!meta title="Original proposal"]] + +*This is the proposal as it has been submitted to Google Summer of +Code.* + +# The name of the project + +Virtualization Using Hurd Mechanisms + +# Summary + +The goal is to create tools that let a user create a set of servers +that implement a Hurd environment and the necessary resources, with +the possibility of relying on existing servers in the parent Hurd for +some of them, instead of creating them. + +# Benefits + +This project will permit to create isolated systems but with far more +flexibility than traditional virtualization tools, because the degree +of isolation can be changed and possibly not only at creation time, +and communication and sharing of subsystems can be arranged between +isolated systems. + +# Deliverables + +D1 — User stories for the toolset, that will later serve as examples +for the documentation + +D2 — Exhaustive but concise documentation of the set of needed servers +making a working Hurd system (as much for me as for future users of +the tool, building and linking to existing Hurd documentation) + +D3 — Low-level tool to create a working Hurd environment (possibly +with strong limitations on the shape of the resources used by the +environment, most probably on the underlying filesystem) + +D4 — Fake or noop servers for the documented set of needed servers, to +be provided instead of working ones, where a feature is to be denied +to a Hurd environnement + +D5 — Proxy servers, where desirable, to provide access to servers +outside the environment (in ocaps terminology, caretakers) + +D6 — Extension of the low-level tool from D3 to remove its +unreasonable limitations + +D7 — High-level tools to easily create environments and run programs +in them (akin respectively to debootstrap and schroot) + +D8 — If possible, extensions to the D5 and D7 tools to enable dynamic +modifications of the features and authority granted to environments +and creation of multiple interconnected environments + +# Plan + +I intend to develop using the Scrum method, with sprints of two weeks, +which mean that each two weeks, I will present at least one new +working feature, working incrementally towards the full deliverable. I +will also push my code at least once a day to a public Git hosting, +including topic branches, so my progress can be followed easily. + +I intend to start from crosshurd and see how I can hook in its process +of creation to allow being provided alternatives. Depending on how +crosshurd is malleable to those changes, a modified crosshurd will +either be a learning-stage prototype or the base of the +implementation. + +To reuse Git terminology, once plumbing tools (i.e. tools that take +detailed invocation information for each server) are working fine, +I'll move on to porcelain tools, the final UI (i.e. tools that provide +sensible default options, aliases mechanisms, etc.). + +# Communication + +I'm usually easy to reach through both email and jabber, so those and +IRC will be my main way to inform my mentor and ask questions. I'll +setup an ikiwiki to have a summary of the exchanges and the temporary +documentation of the project (i.e. documentation that doesn't fit with +the code yet). + +# Qualification + +Thansk to or because of my participation to the Hurd mailing lists, +I've been utterly contaminated by the concept of POLA a few years +ago. Since then, I've been longing, almost in a painful way, for a +object-capability flavour of Debian. Having to deal in my previous day +jobs with virtualization tools like Xen and VMWare when I knew there +would be no need for paravirtualization or emulation to isolate +systems in an object-capability OS only made it worst. + +Now most of the code I produce naturally becomes capability oriented, +even if my underlying platform, programming language or OS, doesn't +provide true capabilities. And creating true POLA systems and making +it possible for others to benefit from POLA is now one of my dreams. diff --git a/community/gsoc/organization_application.mdwn b/community/gsoc/organization_application.mdwn index 9fe3a420..8e672af1 100644 --- a/community/gsoc/organization_application.mdwn +++ b/community/gsoc/organization_application.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -8,166 +9,147 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -* Link ID: - -hurd - -* Group Name: +* Organization Name: GNU Hurd -* Home Page URL: - -http://hurd.gnu.org - -* Public Email: - -bug-hurd@gnu.org - * Description: -The Hurd project is a loose community of people sharing a common interest in -developing the Hurd kernel, which is the official kernel of the [GNU operating -system](http://gnu.org). - -When the Hurd was originally started in 1990, it was the last missing major -component for a complete GNU system. Today Linux and other free kernels are -available to fill this gap, and the combination of GNU and Linux (often -[incorrectly](http://www.gnu.org/gnu/why-gnu-linux.html) called just "Linux") -is in wide use. However, the Hurd is still interesting due to its unique -design, better fitting the GNU philosophy than traditional monolithic kernels -like Linux. - -The GNU GPL guarantees that all users of software published under this license -get the legal permission to adapt the software they are using according to -their wishes, and also get the source code and other tools necessary to put -this permission to use. However, in traditional operating systems, the kernel -and related low-level system software are protected from normal users, and -cannot be easily modified; only the system administrator has power over these. - -The Hurd offers special mechanisms that allow any user to change almost all of -the system functionality he uses, without affecting the rest of the system, and -thus easily (at runtime) and without any special permissions. - -This ability to run subenvironments more or less independant from the rest of -the system, can be classified as a very sophisticated [lightweight -virtualization](http://tri-ceps.blogspot.com/2007/10/advanced-lightweight-virtualization.html) -approach. - -To offer these possibilities, the Hurd uses a true multiserver microkernel -architecture. That makes it quite unique: The Hurd is the only general-purpose -multiserver microkernel system in development today that is nearly ready for -everyday use, and offering almost perfect UNIX compatibility. (More than half -of the packages in the Debian repository are available for the Hurd.) All other -existing true microkernel systems are either research projects not nearly -complete enough for actual use, or limited to embedded systems and other -special purposes, or both. - -Marcus Brinkmann and Neal Walfield from the Hurd project are working at the -bleeding edge of microkernel operating system research. They have been in -contact with the most distinguished researchers in that field from the +The mission of the Hurd project is to create a general-purpose kernel suitable +for the [GNU operating system](http://gnu.org), which is viable for everyday +use, and gives users and programs as much control over their computing +environment as possible. + +In traditional operating systems, most system functionality is provided by the +kernel, and thus cannot be easily modified. The Hurd on the other hand -- +following the GNU spirit of giving users more control over the software they +use -- implements a unique design, which makes it feasible to change almost +everything, down to the core features of the system. + +While on other systems, such changes would require a lot of effort and special +privileges to rebuild the system core, with the Hurd this is not necessary: the +extensible architecture enables users (or applications) to simply modify their +local system environment at any time, while leaving the rest of the system in +place. + +The most obvious example is the completely decentralized VFS mechanism: it can +be extended in almost any imaginable way, simply by setting up suitable server +processes (translators). Not only does this empower users, but also it helps +application development: desktop environments such as GNOME for example, when +making use of these possibilities, wouldn't need to create their own VFS +mechanism -- they simply could extend the system VFS to suit their needs. + +One major element of the design which enables this extensibility, is the use of +a true multiserver microkernel architecture. The Hurd is quite unique in being +the only general-purpose multiserver microkernel system in development today, +that is nearly ready for everyday use, and offering almost perfect UNIX +compatibility. (About 65% of all packages in the Debian repository are +available for the Hurd.) The "general-purpose" and "everyday use" bits are +decisive here: all other existing true microkernel systems are either research +projects not nearly complete enough for actual use; or limited to embedded +systems and other special purposes; or both. + +Marcus Brinkmann and Neal Walfield, while working on improvements to the Hurd +design, pushed at the forefront of microkernel operating system research. They +worked with the most distinguished researchers in this field from the [L4](http://l4hq.org/) and [EROS](http://www.eros-os.org/eros.html)/[Coyotos](http://www.coyotos.org/) -microkernel operating system groups, and have written a couple of [research -papers](http://walfield.org/). - -* Why is your group applying to participate? What do you hope to gain by participating? - -The primary goal of course is to find and introduce new long-term contributors -to the project. - -Aside from that, it is a way to make progress with tasks that require an amount of -focused work, that is hard to do for volunteers working in their spare time -only. +microkernel operating system groups, and published a couple of [research +papers](http://walfield.org/) as well in this process. -Also it is a good possibility to get valuable input from new people, as well as -spreading technical and other knowledge about the Hurd among actual and -potential contributors. More generally, participation should help raising -awareness among people who might know about the existence of the Hurd, but -otherwise having very little idea what the project is all about, and how its -progress is. - -Last but not least, we hope the participation will have a positive effect on -our community -- new impulses, increased communication etc. +* Home Page: -* What is the main public mailing list for your group? +http://hurd.gnu.org -bug-hurd@gnu.org, see http://lists.gnu.org/mailman/listinfo/bug-hurd +* Main Organization License: -* Where is the main IRC channel for your group? +GNU General Public License (GPL) -\#hurd on freenode.net +* Why is your organization applying to participate in GSoC 2010? What do you hope to gain by participating? -* What criteria do you use to select the members of your group? Please be as specific as possible. +The primary goal of our participation is of course to find and introduce new long-term contributors +to the Hurd. We are trying to optimise for this in our student selection +process, our mentoring approach, and our choice of project ideas. -The most important criterium is that the person is involved in the project for -some time, knowing the ways; so he can actually instruct the student; and if -there are tough technical questions he can't answer himself, he knows whom to -ask. +The mentor-student setup, together with the period of focused work during the +summer session, also offer a unique opportunity for kick-starting innovative +new projects apart from mainline development, which are hard to fit in among +the normal day-to-day development work. This is particularily important for the +Hurd, as innovative uses are crucial to show the benefits of the unique +architecture. Several such projects came into being through the GSoC program +over the past years. -It's also important that the mentors are reliable and helpful, so the students -won't be left on their own with any problems they face. +Last but not least, GSoC participation always yields a lot of valuable input from new people, and helps +spreading technical and other knowledge about the Hurd among actual and +potential contributors. It has a very positive effect on +our community -- new impulses, increased communication, etc. -* Has your group participated previously? If so, please summarize your involvement and any past successes and failures. +* Did your organization participate in past GSoCs? If so, please summarize your involvement and the successes and challenges of your participation. In 2006 and 2007, we participated under the umbrella of the GNU project, getting one slot each year. -The 2006 participation was mostly a failure. After some intitial work -(available in CVS), the student disappeared -- moving to another country and -other personal issues from what we heard. +In 2008 we participated as an organisation on our own for the first time. This +turned out extremely beneficial: with the better visibility, we got a lot +more applications (more than 20), mostly of good or excellent quality. -The 2007 participation was a considerable success. The student was very bright -and dedicated. We got some code, as well as a lot of ideas, which we continued -discussing after the end of GSoC, and he intends to put into code as well in -the future. +In 2009, we were rejected as an organisation, so we participated under the GNU +umbrella again. -In 2008 we participated as an organisation on our own for the first time. This -turned out extremely beneficial: Not only did it give us much better -possibilities to find and select good students, as we hoped. We also get a lot -more applications, mostly of good or excellent quality. +While the 2006 student disappeared midway, in all the later years all of our +students were successful -- even including one who worked on his project in +spite of not getting an official slot. Half of them are regular Hurd contributors now. -We ended up with four slots. (We didn't request more, because we were not sure -whether we would be able to mentor them properly, and generally didn't want to -overdo it on our first "full" participation.) There was also a fifth student, -who worked on his project in spite of not getting a slot. +Selecting the most promising students, as well as suitable mentors, turned out +to be the most tricky part of GSoC participation -- but we learned our lesson +after the first failure: we didn't have any students that didn't meet our +expectations since then, and we also believe our mentoring is exceptionally +good now -- one project that was in serious trouble, turned out well after all, +due to effective mentor intervention. -All five students were pretty successful, most of them completing or almost -completing the original goals -- some even exceeding them. Even our weakest -student, after serious struggling in the beginning, did quite well in the end. +* If your organization participated in past GSoCs, please let us know the ratio of students passing to students allocated, e.g. 2006: 3/6 for 3 out of 6 students passed in 2006. -Two students are still regularily working on the Hurd -- not as much as we -hoped of course, but probably as much as can be realistically expected... +2008: 4/4 -All in all, the participation was a considerable amount of work, but it was -definitely worth it :-) +(+1 inofficial in 2008) +(under GNU umbrella: 2006: 0/1; 2007: 1/1; 2009: 1/1) -* If your group has not previously participated, have you applied in the past? If so, for what sort of participation? +* If your organization has not previously participated in GSoC, have you applied in the past? If so, for what year(s)? -- -* What license does your organization use? +* What is the URL for your ideas page? -GNU General Public License (GPL) +http://www.gnu.org/software/hurd/community/gsoc/project_ideas.html -* What is the URL to the ideas list of your organization? +* What is the main development mailing list for your organization? This question will be shown to students who would like to get more information about applying to your organization for GSoC 2010. If your organization uses more than one list, please make sure to include a description of the list so students know which to use. -http://www.gnu.org/software/hurd/community/gsoc/project_ideas.html +bug-hurd@gnu.org ( http://lists.gnu.org/mailman/listinfo/bug-hurd ) -* What is the main development mailing list for your group? +* What is the main IRC channel for your organization? -bug-hurd@gnu.org, see http://lists.gnu.org/mailman/listinfo/bug-hurd +\#hurd on freenode.net -* What is the application template you would like contributors to your organization to use. +* Does your organization have an application template you would like to see students use? If so, please provide it now. Please note that it is a very good idea to ask students to provide you with their contact information as part of your template. Their contact details will not be shared with you automatically via the GSoC 2010 site. [[student_application_form]] -* What is your plan for dealing with disappearing contributors? +* What criteria did you use to select the individuals who will act as mentors for your organization? Please be as specific as possible: -The plan is mostly to avoid that happening in the first place. For that, we -will be particularily careful with the selection of the students: Making sure -that they have no other obligations during that time; that they are motivated +The most important criterium is that the person is involved in the project for +some time, knowing the ways; so he can actually instruct the student; and if +there are tough technical questions he can't answer himself, he knows whom to +ask. + +It's also important that the mentors are reliable and helpful, so the students +won't be left on their own with any problems they face. + +* What is your plan for dealing with disappearing students? + +The plan is mostly to avoid that happening in the first place. To this end, we +are particularily careful with selection of students: Making sure +that they have no other major obligations during that time; that they are motivated enough; that they actually have the necessary skills to complete the task; that they fit in our community. @@ -175,50 +157,55 @@ Also, we will make sure that we are constantly in contact with the students -- asking about progress, discussing technical issues, etc. -- so we can act in time if things go wrong. -If a student disappears in spite of that, there is little we can do. Of course +If a student disappears in spite of all this, there is little we can do. Of course we will try to contact him and find out what the problem is; whether the -project can perhaps be scaled down, or at least wrapped up to bring it in a -state where it is useful even if not finished. +project can perhaps be scaled down, or otherwise salvaged, so that the effort +already invested in the student and the project is not wasted. We also try to +make sure that all important design discussions are archieved, and that all +code produced is suitable for upstream inclusion from the beginning -- to allow +others to pick up the project if necessary, without having to start from zero. -We will also try to limit damage by insisting that students regularily check in -their work, so that we get partial results at least if someone disappears. - -* What is your plan for dealing with disappearing members? +* What is your plan for dealing with disappearing mentors? As our mentors all have been with the project for some time, the risk of them disappearing is not too big. If one of them disappears nevertheless, it's not a -problem for us: We have enough mentors, and someone else will take over. +problem for us: we have enough mentors, and someone else will take over. We will encourage the students to keep discussions public as much as possible, keeping private conversations with the mentors to a minimum, so the transition should go smoothly. -* What steps will you take to encourage contributors to interact with your community before, during, and after the program? +* What steps will you take to encourage students to interact with your project's community before, during and after the program? We try to make it very clear that we expect the students to get into regular -contact with us before the end of the student selection process, and won't +contact with us early during the student selection process already, and won't consider their applications otherwise. This way we know that the students are able and willing to communicate with us in the first place. -After the selection, the regular contact will be kept up: We require the -students to participate in weekly IRC meetings, where we ask the students -actively about the work they do, problems they face, decisions they take etc. -Furthermore, we will ask them to hang around on IRC most of the time while -working on their projects, so we keep in close contact. +After selection, the regular contact is kept up: we require the +students to participate in IRC meetings up to twice a week, where we ask the students +actively about the work they do, problems they face, decisions they take, etc. +Furthermore, we ask them to be available on IRC while working on their +projects, so we can communicate easily. We also require the students to join our main development mailing list, so any -design questions etc. can be discussed there. We will encourage them to take +design questions, etc. can be discussed there. We encourage them to take part in other conversations, not directly related to their projects, as well. -After the program we continue the regular meetings, still discussing the -projects: The application of the code created, future directions etc. +After the program we continue the regular meetings, discussing the further +development of their original projects; as well as new projects, after the +original ones are done. -* What will you do to ensure that your accepted contributors stick with the project after the program concludes? +* What will you do to ensure that your accepted students stick with the project after GSoC concludes? -We will try to invite all participating students to a conference afterwards, -where we will discuss the projects, as well as other Hurd-related topics. We -hope this will motivate them to follow up on the work they have done during the +In addition to keeping up the regular IRC meetings, +we try to invite all participating students to meet us at conferences afterwards, +where we discuss the projects, as well as other Hurd-related topics. This should +keep them motivated to follow up on the work they have done during the program, and generally help keeping them involved. -* Please select your backup group administrator. +* Is there anything else you would like to tell the Google Summer of Code program administration team? : + +* Backup Admin (Link ID): +tschwinge diff --git a/community/gsoc/project_ideas.mdwn b/community/gsoc/project_ideas.mdwn index 32c8aed8..e3d2700d 100644 --- a/community/gsoc/project_ideas.mdwn +++ b/community/gsoc/project_ideas.mdwn @@ -1,12 +1,13 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011, 2012 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] We offer a wide range of possible projects to choose from. If you have an idea not listed here, we'd love to hear about it! @@ -39,7 +40,7 @@ Try to find something to improve in the relevant code, by looking at known issues in the [Savannah bug tracker](http://savannah.gnu.org/bugs/?group=hurd); by running the code and testing stuff; and by looking through the code. If you don't find anything, try with some related code -- if you task involves -translator programming, make some improvement to an existing translor; if it +translator programming, make some improvement to an existing translator; if it involves glibc hacking, make an improvement to glibc; if it involves driver hacking, make an improvement to the driver framework; and so on... Makes sense, doesn't it? :-) @@ -73,17 +74,20 @@ After all, the purpose of GSoC is to introduce you to free software development before the end of the application process, with our help -- contact us, and we will assist you as well as we can. -See also the list of [Hurd-related X.org project ideas](http://wiki.x.org/wiki/Hurd_Porting). +See also the list of [Hurd-related X.Org project +ideas](http://xorg.freedesktop.org/wiki/Hurd_Porting). + +<!-- Olaf, wouldn't it make sense to put the following tasks next to each +other: language_bindings, gnat, gccgo, perl_python. --> [[!inline pages="community/gsoc/project_ideas/language_bindings" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/virtualization" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/file_locking" show=0 feeds=no actions=yes]] -[[!inline pages="community/gsoc/project_ideas/driver_glue_code" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/server_overriding" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/tcp_ip_stack" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/nfs" show=0 feeds=no actions=yes]] -[[!inline pages="community/gsoc/project_ideas/libdiskfs_locking" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/pthreads" show=0 feeds=no actions=yes]] +[[!inline pages="community/gsoc/project_ideas/smp" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/sound" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/disk_io_performance" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/vm_tuning" show=0 feeds=no actions=yes]] @@ -91,17 +95,22 @@ See also the list of [Hurd-related X.org project ideas](http://wiki.x.org/wiki/H [[!inline pages="community/gsoc/project_ideas/gnumach_cleanup" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/xmlfs" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/unionfs_boot" show=0 feeds=no actions=yes]] -[[!inline pages="community/gsoc/project_ideas/tmpfs" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/lexical_dot-dot" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/secure_chroot" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/package_manager" show=0 feeds=no actions=yes]] -[[!inline pages="community/gsoc/project_ideas/debian_installer" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/download_backends" show=0 feeds=no actions=yes]] -[[!inline pages="community/gsoc/project_ideas/libgtop" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/maxpath" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/gnat" show=0 feeds=no actions=yes]] +[[!inline pages="community/gsoc/project_ideas/gccgo" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/hardware_libs" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/cdparanoia" show=0 feeds=no actions=yes]] -[[!inline pages="community/gsoc/project_ideas/perl" show=0 feeds=no actions=yes]] +[[!inline pages="community/gsoc/project_ideas/perl_python" show=0 feeds=no actions=yes]] +[[!inline pages="community/gsoc/project_ideas/testsuites" show=0 feeds=no actions=yes]] +[[!inline pages="community/gsoc/project_ideas/testing_framework" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/libcap" show=0 feeds=no actions=yes]] [[!inline pages="community/gsoc/project_ideas/xattr" show=0 feeds=no actions=yes]] +[[!inline pages="community/gsoc/project_ideas/valgrind" show=0 feeds=no actions=yes]] +[[!inline pages="community/gsoc/project_ideas/gcc_asan" show=0 feeds=no actions=yes]] +[[!inline pages="community/gsoc/project_ideas/driver_glue_code" show=0 feeds=no actions=yes]] +[[!inline pages="community/gsoc/project_ideas/dtrace" show=0 feeds=no actions=yes]] +[[!inline pages="community/gsoc/project_ideas/libdiskfs_locking" show=0 feeds=no actions=yes]] diff --git a/community/gsoc/project_ideas/debian_installer.mdwn b/community/gsoc/project_ideas/debian_installer.mdwn index ca10a61e..37dcc72d 100644 --- a/community/gsoc/project_ideas/debian_installer.mdwn +++ b/community/gsoc/project_ideas/debian_installer.mdwn @@ -10,6 +10,11 @@ is included in the section entitled [[!meta title="Port the Debian Installer to the Hurd"]] +[!] Jérémie Koenig has been working on this as a [[Google Summer of Code +2010|2010]] project. + +--- + The primary means of distributing the Hurd is through Debian GNU/Hurd. However, the installation CDs presently use an ancient, non-native installer. The situation could be much improved by making sure that the newer *Debian @@ -21,6 +26,21 @@ Some preliminary work has been done, see The goal is to have the Debian Installer fully working on the Hurd. It requires relatively little Hurd-specific knowledge. +A lot of the "non-Linux support" part of the project has already been done thanks to a previous GSoC, so at least no ground reason should bar the project. A lot of the required udebs are already in Debian or are pending upload, so that building an image and booting it does already work. A preliminary list of what remains is + + * Add initrd support to GNU Mach, unless youpi does it before :) This should not be very complicated by re-using the iopl driver code. + * hurdify genext2fs to handle 4096 block size by default (see bug #562999) and support translator entries. + * Port busybox. This needs to be synchronized with kfreebsd people, who have probably already done some work, but that seemingly still hasn't been merged. In the meanwhile, youpi has a version with most of it disabled so a d-i image can actually be built. + * Port keyboard-setup to configure the xkb driver of the Hurd console + +As a starting point to get a grasp at how the debian installer is built, students might wish to look at the current Debian installer source and build process on Linux: + + * svn co svn://svn.debian.org/d-i/trunk/installer/ + * cd installer/build + * make build_monolithic + +The same can be done on hurd-i386 but a few non-uploaded packages are needed, see http://people.debian.org/~sthibault/hurd-i386/README-d-i + Possible mentors: Samuel Thibault (youpi) -Exercise: Try to get one piece of the installer running on Hurd. +Exercise: Fix a couple of Hurd issues in busybox. diff --git a/community/gsoc/project_ideas/disk_io_performance.mdwn b/community/gsoc/project_ideas/disk_io_performance.mdwn index bb047308..ae634709 100644 --- a/community/gsoc/project_ideas/disk_io_performance.mdwn +++ b/community/gsoc/project_ideas/disk_io_performance.mdwn @@ -1,34 +1,48 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="Disk I/O Performance Tuning"]] +[[!tag open_issue_hurd]] + The most obvious reason for the Hurd feeling slow compared to mainstream -systems like GNU/Linux, is very slow harddisk access. +systems like GNU/Linux, is a low I/O system performance, in particular very +slow hard disk access. The reason for this slowness is lack and/or bad implementation of common -optimisation techniques, like scheduling reads and writes to minimalize head +optimization techniques, like scheduling reads and writes to minimize head movement; effective block caching; effective reads/writes to partial blocks; -reading/writing multiple blocks at once; and read-ahead. The +[[reading/writing multiple blocks at once|open_issues/performance/io_system/clustered_page_faults]]; and +[[open_issues/performance/io_system/read-ahead]]. The [[ext2_filesystem_server|hurd/translator/ext2fs]] might also need some -optimisations at a higher logical level. +optimizations at a higher logical level. The goal of this project is to analyze the current situation, and implement/fix -various optimisations, to achieve significantly better disk performance. It +various optimizations, to achieve significantly better disk performance. It requires understanding the data flow through the various layers involved in -disk acces on the Hurd ([[filesystem|hurd/virtual_file_system]], +disk access on the Hurd ([[filesystem|hurd/virtual_file_system]], [[pager|hurd/libpager]], driver), and general experience with -optimising complex systems. That said, the killing feature we are definitely -missing is the read-ahead, and even a very simple implementation would bring +optimizing complex systems. That said, the killing feature we are definitely +missing is the [[open_issues/performance/io_system/read-ahead]], and even a very simple implementation would bring very big performance speedups. +Here are some real testcases: + + * [[open_issues/performance/io_system/binutils_ld_64ksec]]; + + * running the Git testsuite which is mostly I/O bound; + + * use [[TopGit]] on a non-toy repository. + + Possible mentors: Samuel Thibault (youpi) Exercise: Look through all the code involved in disk I/O, and try something diff --git a/community/gsoc/project_ideas/download_backends.mdwn b/community/gsoc/project_ideas/download_backends.mdwn index 749597a6..f794e814 100644 --- a/community/gsoc/project_ideas/download_backends.mdwn +++ b/community/gsoc/project_ideas/download_backends.mdwn @@ -19,7 +19,7 @@ Download protocols like FTP, HTTP, BitTorrent etc. are very good candidates for this kind of modularization: a program could simply use the download functionality by accessing FTP, HTTP etc. translators. -There is already an ftpfs traslator in the Hurd tree, as well as an [httpfs +There is already an ftpfs translator in the Hurd tree, as well as an [httpfs translator on hurdextras](http://www.nongnu.org/hurdextras/#httpfs); however, these are only suitable for very simple use cases: they just provide the actual file contents downloaded from the URL, but no additional status information @@ -35,7 +35,7 @@ The goal of this project is to design a suitable interface, implement it for at least one download protocol, and adapt apt-get (or some other program) to use this as a backend. -This task requires some design skills and some knowlegde of internet protocols, +This task requires some design skills and some knowledge of internet protocols, to create a suitable interface. Translator programming knowledge will have to be obtained while implementing it. diff --git a/community/gsoc/project_ideas/driver_glue_code.mdwn b/community/gsoc/project_ideas/driver_glue_code.mdwn index 04efe202..8581c7cb 100644 --- a/community/gsoc/project_ideas/driver_glue_code.mdwn +++ b/community/gsoc/project_ideas/driver_glue_code.mdwn @@ -1,42 +1,73 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] -[[!meta title="New Driver Glue Code"]] +[[!meta title="New Driver Framework"]] -Although a driver framework in userspace would be desirable, presently the Hurd -uses kernel drivers in the microkernel, -[[GNU_Mach|microkernel/mach/gnumach]]. (And changing this would be far beyond a -GSoC project...) +[[!tag stable_URL]] -The problem is that the drivers in GNU Mach are presently old Linux drivers -(mostly from 2.0.x) accessed through a glue code layer. This is not an ideal -solution, but works quite OK, except that the drivers are very old. The goal of -this project is to redo the glue code, so we can use drivers from current Linux -versions, or from one of the free BSD variants. +The Hurd presently uses hardware drivers +implemented in the microkernel, [[GNU_Mach|microkernel/mach/gnumach]]. +These drivers are old Linux drivers (mostly from 2.0.x) +accessed through a glue code layer. +This is not an ideal solution, but works quite OK, +except that the drivers are extremely old by now. +Thus we need a new framework, +so we can use drivers from current Linux versions instead, +or perhaps from one of the free BSD variants. -While it would be certainly possible to create custom glue code again, a more -sustainable and probably also easier approch is to use -[ddekit](http://demo.tudos.org/dsweeper_tutorial.html) instead -- it already -does the hard work of providing an environment where the foreign drivers can -run, and has the additional advantage of being externally maintained. +This is [[!GNU_Savannah_task 5488]]. +[[open issues/user-space device drivers]]. +[[open issues/device drivers and io systems]]. -This is a doable, but pretty involved project. Previous experience with driver -programming probably is a must. (No Hurd-specific knowledge is required, -though.) +The most promising approach for getting newer drivers seems to be [[DDE]]: +it already does the hard work of providing an environment +where the foreign drivers can run, +and offers the additional benefit of being externally maintained. +DDE also offers the necessary facilities +for running all drivers in separate userspace processes, +which is more desirable than drivers running in the microkernel. -This is [[!GNU_Savannah_task 5488]]. +[[Zheng Da|zhengda]] has already done considerable work on this. +The basic framework for using DDE in the Hurd is present, +and network card drivers are already working very well. +However, this work isn't fully integrated in the Hurd yet. +The additional kernel interfaces that were created for this +are still prototypes, and will need to be reworked. +Also, there is no build system for automatically compiling +all Linux network card drivers in one go. + +Other types of drivers are missing so far. +Support for IDE drivers has been partially implemented, +but isn't fully working yet. +To fully replace the old in-kernel drivers, +further infrastructure will be necessary +to make userspace disk drivers usable for the root filesystem. + +Some other subsystems are missing or incomplete in DDE itself, +and will require additional work that is not specific to the Hurd implementation. + +The goal of this task is to fix at least one of the mentioned major shortcomings: +rework the kernel interfaces; +provide a streamlined build system for the drivers; +finish IDE support; +or implement support for some other subsystem. +<!-- should probably provide separate task descriptions for each... --> + +This is a doable, but pretty involved project. +Previous experience with driver programming probably is a must. +To be able to work on the framework, +the student will also have to get a good understanding of certain aspects of Hurd, +such as memory management for example. -Possible mentors: Samuel Thibault (youpi) +Possible mentors: Zheng Da, Samuel Thibault (youpi) -Exercise: Take a driver for some newer piece of hardware (e.g. Intel e1000 -ethernet) from a recent system, and try to port it to run in the existing -driver framework in GNU Mach. Completing the port might be too involved for the -exercise; but it's pretty likely that you will find something else to improve -in the glue code while working on this... +Exercise: Get one of the not yet integrated Linux network card drivers to work. +(Note: This should be straightforward, +once you have the framework properly built and set up...) diff --git a/community/gsoc/project_ideas/dtrace.mdwn b/community/gsoc/project_ideas/dtrace.mdwn index 93c2a5f3..6261c03e 100644 --- a/community/gsoc/project_ideas/dtrace.mdwn +++ b/community/gsoc/project_ideas/dtrace.mdwn @@ -1,22 +1,25 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2011 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] -[[!meta title="dtrace Support"]] +[[!meta title="Kernel Instrumentation"]] + +[[!tag open_issue_gnumach]] One of the main problems of the current Hurd implementation is very poor -performance. While we have a bunch of ideas what could cause the performance +[[open_issues/performance]]. While we have a bunch of ideas what could cause the performance problems, these are mostly just guesses. Better understanding what really causes bad performance is necessary to improve the situation. For that, we need tools for performance measurements. While all kinds of more -or less specific profiling tools could be convieved, the most promising and +or less specific [[open_issues/profiling]] tools could be conceived, the most promising and generic approach seems to be a framework for logging certain events in the running system (both in the microkernel and in the Hurd servers). This would allow checking how much time is spent in certain modules, how often certain @@ -24,23 +27,36 @@ situations occur, how things interact, etc. It could also prove helpful in debugging some issues that are otherwise hard to find because of complex interactions. -The most popular framework for that is Sun's dtrace; but there might be others. -The student has to evaluate the existing options, deciding which makes most -sense for the Hurd; and implement that one. (Apple's implementation of dtrace -in their Mach-based kernel might be helpful here...) - -This project requires ability to evaluate possible solutions, and experience -with integrating existing components as well as low-level programming. +The most popular kernel instrumentation framework is Sun's dtrace, +originally written for Solaris, +but also adopted by some other systems. +However, the GPL-incompatible license means it can't be used in Linux, +and thus Linux developers created their own frameworks instead: +first [[SystemTap]], and now [[LTTng]]. + +In 2008, Andrei Barbu did initial work on kernel probes for the Hurd. +However, not all of his patches got merged, +because some turned out not to be fully functional. +Also, he didn't get around to work on userspace probes, +nor on a nice frontend for writing test scripts employing the probes. + +The goal of this project is to make the instrumentation framework +more usable and complete, +and to better integrate it in the Hurd. +For that, the student will have to work +on some real profiling and/or debugging tasks, +and fix any shortcomings he encounters in the framework. + +This is a pretty involved task. +Previous experience with low-level programming is a must; +and it also requires a good grasp on interactions in complex systems. + +To work on this project, +the student will have to get familiar with GNU Mach. +(The microkernel employed by the Hurd.) +Some understanding of other aspects of the Hurd will also be required, +depending on the exact nature of the profiling/debugging performed. Possible mentors: Samuel Thibault (youpi) -Exercise: In lack of a good exercise directly related to this taks, just pick -one of the kernel-related or generally low-level tasks from the bug/task -trackers on savannah, and make a go at it. You might not be able to finish the -task in a limited amount of time, but you should at least be able to make a -detailed analysis of the issue. - -*Status*: Andei Barbu was working on -[SystemTap](http://csclub.uwaterloo.ca/~abarbu/hurd/) for GSoC 2008, but it -turned out too Linux-specific. He implemented kernel probes, but there is no -nice frontend yet. +Exercise: Use the existing probes to perform some simple measurement. diff --git a/community/gsoc/project_ideas/file_locking.mdwn b/community/gsoc/project_ideas/file_locking.mdwn index b6b393f9..206d4d7d 100644 --- a/community/gsoc/project_ideas/file_locking.mdwn +++ b/community/gsoc/project_ideas/file_locking.mdwn @@ -1,27 +1,31 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2012 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] -[[!meta title="Fix File Locking"]] +[[!meta title="Fix and Complete File Locking Support"]] -Over the years, [[UNIX]] has aquired a host of different file locking mechanisms. +Over the years, [[UNIX]] has acquired a host of different file locking mechanisms. Some of them work on the Hurd, while others are buggy or only partially implemented. This breaks many applications. The goal is to make all file locking mechanisms work properly. This requires finding all existing shortcomings (through systematic testing and/or checking for known issues in the bug tracker and mailing list archives), and fixing -them. +them. The biggest missing feature is record locking, i.e. the lockf variant, +which needs a complete implementation. This task will require digging into parts of the code to understand how file locking works on the Hurd. Only general programming skills are required. +A preliminary patch is [[!GNU_Savannah_patch 332 desc="available"]]. + Possible mentors: Samuel Thibault (youpi) Exercise: Find one of the existing issues, either by looking at the task/bug diff --git a/community/gsoc/project_ideas/gcc_asan.mdwn b/community/gsoc/project_ideas/gcc_asan.mdwn new file mode 100644 index 00000000..229c46ec --- /dev/null +++ b/community/gsoc/project_ideas/gcc_asan.mdwn @@ -0,0 +1,21 @@ +[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="Port GCC's AddressSanitizer to the Hurd"]] + +[[!tag open_issue_gcc]] + +See the entry on the [[open_issues/code_analysis]] page. + +See also the [[valgrind]] task. + +A follow-up project is porting GCC's ThreadSanitizer. + +Possible mentors: Thomas Schwinge (tschwinge) diff --git a/community/gsoc/project_ideas/gccgo.mdwn b/community/gsoc/project_ideas/gccgo.mdwn new file mode 100644 index 00000000..54b20754 --- /dev/null +++ b/community/gsoc/project_ideas/gccgo.mdwn @@ -0,0 +1,34 @@ +[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="Porting Google Go (GCC: gccgo)"]] + +The goal of this project is to make the [Google Go programming +language](http://golang.org/) available on GNU/Hurd in its [[GCC]] *gccgo* +implementation. + +Presumably less work will be needed on the language's frontend itself, but +rather on the supporting libraries. + +Apart from a solid knowledge of the [[POSIX]] API, working knowledge of the +Google Go programming language is a must. Some Hurd knowledge will have to be +acquired while working on the project. + +Designing and implementing [[language_bindings]] is a follow-up project. + +Possible mentors: Ian Lance Taylor: gccgo bits, [[Thomas Schwinge +(tschwinge)|tschwinge]]: Hurd bits. + +Exercise: Fix one of the problems preventing *gccgo* from working on the Hurd. + +--- + +[[Open Issue page|open_issues/gccgo]]. [Entry in the GCC +wiki](http://gcc.gnu.org/wiki/SummerOfCode#gccgo_hurd). diff --git a/community/gsoc/project_ideas/gnat.mdwn b/community/gsoc/project_ideas/gnat.mdwn index b7f2ea62..ba34cc9c 100644 --- a/community/gsoc/project_ideas/gnat.mdwn +++ b/community/gsoc/project_ideas/gnat.mdwn @@ -1,24 +1,32 @@ -[[!meta copyright="Copyright © 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2011, 2012 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] -[[!meta title="Porting GNAT"]] +[[!meta title="Porting GNAT (GCC)"]] -The GNU Ada Translator (GNAT) isn't available for the Hurd so far. There are -also a number of other Debian packages depending on GNAT, and thus not -buildable on the Hurd. +An initial port of the GNU Ada Translator (GNAT) is available for the Hurd. The goal of this project is getting GNAT fully working in Debian GNU/Hurd. It -requires implementing some explicitely system-specific stuff in GNAT, and maybe -fixing a few other problems. Good knowledge of Ada is a must; some Hurd -knowledge will have to be aquired while working on the project. +requires implementing some explicitly system-specific stuff in GNAT (mostly in +its runtime libraries), and for that also address a number of issues in Hurd +and other libraries. Knowledge of Ada is a must; some Hurd +knowledge will have to be acquired while working on the project. -Possible mentors: Samuel Thibault (youpi) +Designing and implementing [[language_bindings]] is a follow-up project. -Exercise: Fix one of the problems preventing GNAT from working on the Hurd. +Possible mentors: [[Samuel Thibault (youpi)|samuelthibault]], [[Thomas Schwinge +(tschwinge)|tschwinge]]. + +Exercise: Fix one of the known issues of GNAT on the Hurd. + +--- + +[[Open Issue page|open_issues/gnat]]. [Entry in the GCC +wiki](http://gcc.gnu.org/wiki/SummerOfCode#gnat_hurd). diff --git a/community/gsoc/project_ideas/gnumach_cleanup.mdwn b/community/gsoc/project_ideas/gnumach_cleanup.mdwn index e75c9d3e..4aef0d1b 100644 --- a/community/gsoc/project_ideas/gnumach_cleanup.mdwn +++ b/community/gsoc/project_ideas/gnumach_cleanup.mdwn @@ -11,7 +11,7 @@ is included in the section entitled [[!meta title="GNU Mach Code Cleanup"]] Although there are some attempts to move to a more modern microkernel -alltogether, the current Hurd implementation is based on +altogether, the current Hurd implementation is based on [[GNU_Mach|microkernel/mach/gnumach]], which is only a slightly modified variant of the original CMU [[microkernel/Mach]]. @@ -19,7 +19,7 @@ Unfortunately, Mach was created about two decades ago, and is in turn based on even older BSD code. Parts of the BSD kernel -- file systems, [[UNIX]] [[mechanism]]s like processes and signals, etc. -- were ripped out (to be implemented in [[userspace_servers|hurd/translator]] instead); while other mechanisms were -added to allow implementing stuff in userspace. +added to allow implementing stuff in user space. ([[Pager_interface|microkernel/mach/external_pager_mechanism]], [[microkernel/mach/IPC]], etc.) diff --git a/community/gsoc/project_ideas/language_bindings.mdwn b/community/gsoc/project_ideas/language_bindings.mdwn index bf75c805..d9a426be 100644 --- a/community/gsoc/project_ideas/language_bindings.mdwn +++ b/community/gsoc/project_ideas/language_bindings.mdwn @@ -1,15 +1,19 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011, 2012 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="Bindings to Other Programming Languages"]] +<!-- See also open_issues/gccgo, open_issues/gnat, open_issues/perl, and +open_issues/python. --> + The main idea of the Hurd design is giving users the ability to easily modify/extend the system's functionality ([[extensible_system|extensibility]]). This is done by creating [[filesystem_translators|hurd/translator]] and other @@ -19,7 +23,7 @@ However, in practice this is not as easy as it should, because creating translators and other servers is quite involved -- the interfaces for doing that are not exactly simple, and available only for C programs. Being able to easily create simple translators in RAD languages is highly desirable, to -really be able to reap the advantages of the Hurd architecture. +really be able to reap the [[advantages]] of the Hurd architecture. Originally Lisp was meant to be the second system language besides C in the GNU system; but that doesn't mean we are bound to Lisp. Bindings for any popular @@ -30,20 +34,24 @@ Several approaches are possible when creating such bindings. One way is simply to provide wrappers to all the available C libraries ([[hurd/libtrivfs]], [[hurd/libnetfs]] etc.). While this is easy (it requires relatively little consideration), it may not be the optimal solution. It is preferable to hook in at a lower level, thus -being able te create interfaces that are specially adapted to make good use of +being able to create interfaces that are specially adapted to make good use of the features available in the respective language. -These more specialised bindings could hook in at some of the lower level +These more specialized bindings could hook in at some of the lower level library interfaces ([[hurd/libports]], [[hurd/glibc]], etc.); use the [[microkernel/mach/MIG]]-provided [[microkernel/mach/RPC]] stubs directly; or even create native stubs directly from the interface definitions. The -[[lisp_bindings_created_by_Flavio_Cruz|flaviocruz]] in last year's GSoC mostly -use the latter approach, and can serve as a good example. +[[lisp_bindings_created_by_Flavio_Cruz|flaviocruz]] as his [[2008 GSoC +project|2008]] mostly use the latter approach, and can serve as a good example. +In his [[2011 GSoC project|2011]], Jérémie Koenig designed and began +implementing an object-oriented interface; see his [[Java status +page|user/jkoenig/java]] for details. There is another possible reason for preferring lower-level bindings: Presently, the Hurd server libraries use the cthreads threading library, which -predates the pthread standard prevalent today. There is a pthread library for -the Hurd as well, but it's not possible to use both cthreads and pthreads in +predates the pthread standard prevalent today. There is a +[[pthread library for the Hurd|libpthread]] +as well, but it's not possible to use both cthreads and pthreads in the same executable. Thus, until [[porting_the_Hurd_libraries_to_pthreads|community/gsoc/project_ideas/pthreads]] is finished, implementing bindings for any language that uses pthreads (in the @@ -64,4 +72,4 @@ There was also some previous work on [Perl bindings](http://www.nongnu.org/hurdextras/#pith), which might serve as a reference if you want to work on Perl. -Possible mentors: Anatoly A. Kazantsev (jim-crow) for Python +Possible mentors: Anatoly A. Kazantsev (anatoly) for Python diff --git a/community/gsoc/project_ideas/lexical_dot-dot.mdwn b/community/gsoc/project_ideas/lexical_dot-dot.mdwn index f0b8db7c..e0dabc01 100644 --- a/community/gsoc/project_ideas/lexical_dot-dot.mdwn +++ b/community/gsoc/project_ideas/lexical_dot-dot.mdwn @@ -22,7 +22,7 @@ resolution. Some application already use lexical resolution internally for that reason. It is generally agreed that many problems could be avoided if the standard filesystem lookup calls used lexical resolution as well. The compatibility -problems probably would be negligable. +problems probably would be negligible. The goal of this project is to modify the filename lookup mechanism in the Hurd to use lexical resolution, and to check that the system is still fully @@ -31,7 +31,7 @@ mechanism. See also [[!GNU_Savannah_bug 17133]]. -Possible mentors: ? +Possible mentors: Carl Fredrik Hammar (cfhammar) Exercise: This project requires changes to the name lookup mechanism in the Hurd-related glibc parts, as well as the Hurd servers. Thus, the exercise task diff --git a/community/gsoc/project_ideas/libcap.mdwn b/community/gsoc/project_ideas/libcap.mdwn index 10ca508e..18c49c48 100644 --- a/community/gsoc/project_ideas/libcap.mdwn +++ b/community/gsoc/project_ideas/libcap.mdwn @@ -1,12 +1,12 @@ -[[!meta copyright="Copyright © 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2010 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="Implementing libcap"]] @@ -14,7 +14,7 @@ libcap is a library providing the API to access POSIX capabilities. These allow giving various kinds of specific privileges to individual users, without giving them full root permissions. -Although the Hurd design should faciliate implementing such features in a quite +Although the Hurd design should facilitate implementing such features in a quite natural fashion, there is no support for POSIX capabilities yet. As a consequence, libcap is not available on the Hurd, and thus various packages using it can not be easily built in Debian GNU/Hurd. @@ -31,6 +31,10 @@ Some knowledge of POSIX capabilities will need to be obtained, and for the latter part also some knowledge about the Hurd architecture. This project is probably doable without previous experience with either, though. +David Hedberg applied for this project in 2010, +and though he didn't go through with it, +he fleshed out many [[details]]. + Possible mentors: Samuel Thibault (youpi) Exercise: Make libcap compile on Debian GNU/Hurd. It doesn't need to actually diff --git a/community/gsoc/project_ideas/libcap/details.mdwn b/community/gsoc/project_ideas/libcap/details.mdwn new file mode 100644 index 00000000..85695978 --- /dev/null +++ b/community/gsoc/project_ideas/libcap/details.mdwn @@ -0,0 +1,766 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="Details on implementing libcap"]] + + +This is the proposal submitted by David Hedberg for GSoC 2010 (who opted +to go with another mentoring organization), adapted from an initial +proposal and several amendments into a single document, but the idea +is to further adapt it to a more neutral design document over time. + + +# The proposal + +### Quick description of POSIX capabilities + +The original suggestion can be found [[here|libcap]]. +POSIX capabilities never actually entered the POSIX standard but was +left as a draft. Linux has nevertheless implemented this draft, and +there are reasons for doing the same in the Hurd - a more fine grained +control of rights leading to increased security being one of them. + +POSIX capabilities are give on a per-process basis, and basically allows +splitting up those rights usually granted to root into smaller and more +specific rights. Examples of capabilities are CAP_CHOWN and CAP_KILL, +overriding certain restrictions on chown and allowing the process to +kill processes with different UID's, respectively. + +Each process is given three sets with capabilities: the Permitted set +(P), the Effective set (E) and the Inheritable set (I). The effective +set contains the capabilities that are currently active. The permitted +set contains the capabilities that the process has the right to use. +The inheritable set contains the capabilities that can be inherited +by children to the process. A process can drop capabilities from its +permitted set, but not set them. The effective set and the inheritable +set can be changed freely as long as they stay subsets of the permitted +set. + +Capabilities can also be set on executables. When executed, the resulting +process is given the capabilities both as defined by the parent process +and by the capabilities set on the file (formula below), resulting in +what might be explained as a fine-grained setuid. Implementing this +requires support for *xattr* or similar. + +Some applications that are currently using capabilities are samba, ntp, +vsftp, pure-ftpd, proftpd, squid, asterisk and dovecot. + + +### A quick description of capabilities in Linux + +Each process has a three bit fields representing each of the three +sets (P, E and I). Each bit field is currently built up of two (32 +bit) integers to be able to hold the 33 currently defined capabilities +(see linux/capability.h). Each process further has a bounding set which +bounds the permitted set. Two [[system call]]s handles the setting and getting +of capabilities; *capset* and *capget*. Some related functionality +can also be controlled by calling *prctl*: the right to read/drop the +bounding capabilities (PR_CAPBSET_READ/PR_CAPBSET_DROP) and whether +or not the process should keep its capabilities when a change is made +to the threads UID's (PR_SET_KEEPCAPS/PR_GET_KEEPCAPS). User space +applications are expected(recommended?) to use libcap to take advantage +of the functionality provided. Some applications also use libcap-ng +which is "intended to make programming with POSIX capabilities much +easier than the traditional libcap library". Most applications seem +to be using the original libcap, however. + + +## Implementing libcap + +The exercise for this assignment was to get the libcap used in +Linux to compile under the Hurd. This was accomplished using the +latest git version of libcap from (..), corresponding to libcap +2.19. The changes were simple and amounted to simply removing the +dependency on some Linux-specific headers and creating stubs for +capset, capget and prctl described above. This suggests that porting +this library to the Hurd once the proper functionality is in place +could be relatively simple. The patch is available +[here](https://alioth.debian.org/tracker/index.php?func=detail&aid=312442&group_id=30628&atid=410472 "Alioth"). +One could also consider implementing the three missing functions in the +Hurd (or Hurd glibc) which would allow the usage of the Linux libcap +without modifications. As the Linux libcap maintainer might or might +not be interested in making libcap multi platform, this approach might +be preferable. + + +## Implementing POSIX capabilities in the Hurd + +As I am still reading up on how things fit together in the Hurd this is +very likely to contain some misunderstandings and be at least partly off +the mark. I intend to have grasped this by the time I start working on +it however, if I were to be given the chance. Below are two possible +approaches as I have understood them after some reading and discussions +on #hurd@freenode. + + +### The basics, Approach 1: Special UID's + +Let each capability be represented by a specific UID. One could imagine +reserving a range of the possible uid_t's for this purpose. The euids +vector in the authhandle struct could perhaps hold the effective set while +the auids vector could hold the permitted set as these seem to roughly +correspond to eachother in intent. This leaves the Inheritable set. +One solution could be to store the inheritable set in the agids vector, +but that does not seem to be a very natural nor nice solution. One could +extend the authhandle struct with an additional vector, but one would then +need to also extend the auth interface with RPC's to be able to modify +and access it. Another possibility is to store all the capabilities +in the same idvec and use separate defines for the the different sets +(CAP_P_CHMOD, CAP_E_CHMOD, CAP_I_CHMOD). This does not seem like a +good solution. + +Storing each capability in its own uid_t might also arguably be somewhat +wasteful, although this is probably of secondary concern (if at all). +One could also imagine that legacy applications might be confused, +although I am not sure I can see any obvious problems. What happens +when a process have only capability id's? + + +### The basics, Approach 2: Extend the auth interface + +This approach would expand the auth interface and extend +the auth server with another set of RPC's for capabilities +(*auth_getcaps*, *auth_makecaps*, *auth_user_authenticate* and +*auth_server_authenticate*), mirroring those currently declared for id's. +This would obviously require changes to more parts of the Hurd for +processes to be able to take advantage of capabilities, but as the logic +behind handling capabilities and that behind handling user id's might +not be completely comparable, this might make for a cleaner solution. +It would also remove the problem of having to sensibly map all the +three capability sets onto semantically differing sets of user/group +ids, something that might be even more important if we were to also +implement something like the bounding sets used in Linux or perhaps +other related functionality. We are also not limited to having to store +the capabilities as id vectors, although doing so would perhaps still +make sense. + + +### The individual capabilities + +Implementing the individual capabilities will probably have to be thought +of on a case-by-case basis. Taking chown (in *libdiskfs*) as an example, +the exact approach would differ slightly depending on how the approach +taken to implement capabilities. If the first approach were to be taken, +the UID's (and thus the capabilities) of the caller would already be +available to the function through the *iouser* struct contained in the +*protid* struct given as the first argument. Implementing capabilities +would then simply be a matter of checking for the special UID's. If the +second approach were to be taken, one would need to perhaps expand the +iouser struct to contain information about the capabilities. + +Just like Linux has defined many Linux-specific capabilities - some of +which could certainly be useful also applied to the Hurd - one could +also imagine extending the POSIX capability system also for Hurd specific +purposes. + + +## Some applications using capabilities + +#### Samba 3 + +Uses CAP_MKNOD and CAP_LEASE in smbd to only keep the necessary abilities. +Documentation mentions CAP_LINUX_IMMUTABLE as a way to protect files +from being deleted. Can also use a couple of IRIX specific capabilities +(CAP_NETWORK_MGT and CAP_DEVICE_MGT) as alternatives to the Linux-specific +ones if running on IRIX. + + +#### ntpd + +Checks if capabilities are supported, more precisely if CAP_SYS_TIME, +CAP_SETUID, CAP_SETGID, CAP_SYS_CHROOT and CAP_NET_BIND_SERVICE are +supported. If they are supported, it uses prctl with PR_SET_KEEPCAPS +to keep privileges across setuid() and then drops root. After done with +CAP_SETUID, CAP_SETGID, CAP_SYS_CHROOT it drops every capability except +CAP_SYS_TIME and, if needed, CAP_NET_BIND_SERVICE. + + +#### vsftpd + +Uses CAP_CHOWN, CAP_NET_BIND_SERVICE when using the "one process" +security model (typically disabled by default). + + +#### proftpd-basic + +Provides support for capabilities from mod_cap. Uses CAP_USE_CHOWN, +CAP_USE_DAC_OVERRIDE, CAP_USE_DAC_READ_SEARCH and CAP_USE_AUDIT_WRITE. +Distribution contains README.capabilities with some explanations. +Also ships with their own libcap for some reason, based on libcap 1.10. + + +#### dovecot + +Keeps CAP_CHOWN, CAP_SYS_CHROOT, CAP_SETUID, CAP_SETGID, +CAP_NET_BIND_SERVICE, CAP_DAC_OVERRIDE for proper operations, drops +the rest. + + +#### bind9 + +Reasons for each capability are clearly noted in comments in update.c +in linux_initialprivs() and linux_minprivs(). initialprivs drops all +capabilities and proceeds to set CAP_NET_BIND_SERVICE, CAP_SYS_CHROOT, +CAP_SETUID, CAP_SETGID, CAP_DAC_READ_SEARCH and CAP_CHOWN. minprivs only +sets CAP_NET_BIND_SERVICE and CAP_SYS_RESOURCE. + + +#### pulseaudio + +Mentions CAP_NICE (CAP_SYS_NICE), but does not appear to be using it +(anymore?). Seems to use libcap to drop caps, however. + + +#### pinentry + +Checks if CAP_IPC_LOCK is available and "uses it" to gain only the +ability to lock memory when needed. + + +#### zsh + +Comes with a module "caps" which contains "[b]uiltins for manipulating +POSIX.1e (POSIX.6) capability (privilege) sets." Most useful here is the +"cap" builtin, which makes it possible to change the shell's process +capability sets. This might be useful for testing. + + +#### inetutils (ping,traceroute) + +Does not use capabilities explicitly, but is nevertheless a useful +example of how file capabilities could be used. ping and traceroute +are currently installed suid root since they need to be able to open +raw sockets. With file capabilities, this could be accomplished by +instead setting the capability CAP_NET_RAW on the two executables, +thus giving the utilities almost only the specific rights they need. + + +## The capabilities + +The above might give some hint as to what capabilities should be +prioritized. One assumption I have made is that the goal of this project +is to implement, as far as possible, the same functionality as what is +present in Linux. No effort has (so far) been made to look into possible +applications specific to the Hurd. + +A few of the above mentioned applications also explicitly uses +PR_SET_KEEPCAPS (through prctl()) to specify that capabilities should +be passed on to children. This means that the implementation in the +Hurd should take this into account. + +I have below done a preliminary classification of the capabilities +as defined in Linux capability.h into four "classes": Network, File +management, "glibc -> mach" and Other. There are also some capabilities +that either depend on functionality not implemented or are too Linux +specific. I have not described each capability in detail as looking +at the comments in capability.h and reading in capabilities(7) are +better sources. + + +### The Networking Class + +These would mostly affect pfinet. The general picture seem to be that +pfinet currently uses a boolean (int) isroot in struct sock_user to keep +track of the credentials of the caller. This would need to be expanded +somehow to keep track of the separate capabilities. + +CAP_NET_BIND_SERVICE: Allow binding to TCP/UDP sockets below 1024 +CAP_NET_RAW: Allow use of RAW and PACKET sockets. +CAP_NET_BROADCAST: "Allow broadcasting, listen to multicast" +CAP_NET_ADMIN: This seem to be a bit of a "catch-all" for network-related +administration. + + +### The Files Management Class + +The description of CAP_CHOWN in the original proposal should apply to +(most of?) these. That is, modify the iouser struct. At least libdiskfs +should be modified, but the same or similar modifications might need to +be made to several servers (libnetfs..? The actual servers implementing +the filesystem?) + +CAP_CHOWN: "Make arbitrary changes to file UIDs and GIDs" +CAP_FOWNER: allow chmod, utime, .. for files not owned. +CAP_FSETID: Don't clear setuid/setgid when a file is modified. +CAP_DAC_OVERRIDE and +CAP_DAC_READ_SEARCH: Bypasses file/directory read/write/execute permission +checks. ( hurdexec.c, file-access.c, .. ? ) +CAP_MKNOD: allows usage of "the privileged aspects of mknod()". Does this +one make sense in the Hurd? + + +### The (glibc -> gnumach) Class + +These seem to be implemented in glibc by direct calls to gnumach. +If they should be implemented, maybe a proxy in the Hurd is needed? + +CAP_SYS_TIME: manipulate the system clock, set real-time clock. +CAP_IPC_LOCK: mlock, mlockall, mmap, shmctl +CAP_KILL: No permission checks for killing processes +CAP_SYS_NICE: setpriority/getpriority for arbitrary processes. + + +### The Rest Class + +CAP_SYS_CHROOT: Allows usage of chroot(). +It's either really simple (not needed in the case of the Hurd) or really +difficult. Needs some figuring out. One of the calls that should be +high-priority. +CAP_SYS_ADMIN: General administration rights. Seemingly sprinkled out +into just about everything. Quick grep through the Linux sources gives +440 hits in .c-files. +CAP_SYS_BOOT: Allow use of reboot(). +glibc calls init:startup_reboot.. +CAP_SETGID: Allows usage of setgid,setgroups and "forged gids on socket +credentials passing" +CAP_SETUID: Allows usage of set*uid and "forged pids on socket credentials +passing" +CAP_SYS_TTY_CONFIG: vhangup, some other places. vhangup() is a stub in +the Hurd, but maybe the console server is otherwise affected? +CAP_SYS_RESOURCE: Override various limits. (quota, reserved space etc. +on ext2, interrupt frequencies, consoles,...). According to "The Critique" +mach performs no resource accounting so some of these might be moot to +implement, while others still apply. +CAP_SYS_RAWIO Allow raw input/output. Sprinkled in many places, +device drivers among others. Many of these will probably be difficult +to implement. +CAP_SETPCAP: This one has (or had?) two different usages in Linux: +If file capabilities are not supported it gives the right to grant +or remove capabilities from the permitted set of arbitrary processes. +If file capabilities are supported, it allows for removing capabilities +from the bounding set of the current process. As the Hurd implementation +won't have file capabilities initially it might make sense to implement +this if possible. If bounding sets are implemented this should probably +be the way provided to modify them. + + +### Unimplementable + +*(At this point in time, as far as I can determine)* + +CAP_LINUX_IMMUTABLE: depends on chattr etc. working. +CAP_SETFCAP: depends on xattr's +CAP_SYS_PACCT: acct() missing in the Hurd. +CAP_SYS_MODULE, CAP_SYS_PTRACE, CAP_MAC_OVERRIDE, CAP_MAC_ADMIN, +CAP_AUDIT_WRITE, CAP_AUDIT_CONTROL, CAP_LEASE + + +## Priority when implementing + +The two most used capabilities as unscientifically judged from +the selection of applications above are CAP_NET_BIND_SERVICE and +CAP_CHOWN, suggesting that implementing the "network class" and the +"file management" class of capabilities as classified above might be a +good start. These also, to me, seem to be easier classes to implement. +CAP_NET_ADMIN might need some extra work. + +Second most common were CAP_SYS_CHROOT, CAP_SETGID and CAP_SETUID. I am +not completely clear on how these should be handled. + +Assuming those are out of the way, CAP_IPC_LOCK, CAP_SYS_TIME, CAP_KILL +and CAP_SYS_NICE might be a good choice to tackle if possible. This +might, if I have understood things correctly, involve writing a proxy +Hurd server for these calls in mach. + +CAP_SYS_ADMIN, CAP_SYS_RESOURCE and CAP_SYS_RAWIO. These contains a bit +of "everything" (ADMIN being the worse one), meaning that experience +and infrastructure gained from implementing the previous capabilities +might come in handy. CAP_SYS_RAWIO might be difficult; it can be found +inside many drivers in the Linux source. + + +## Additional general details + +[This article](http://www.ibm.com/developerworks/library/l-posixcap.html) +contains a good general description of how capabilities in Linux +works. As there will be no file capabilities in the Hurd initially, +an approach emulating the behavior Linux exhibits when SECURE_NOROOT +and SECURE_NO_SETUID_FIXUP are *not* set seems to be a good start. +This is called the "root-user-is-privileged" model in the article, +and somewhat simplified it means that processes started by root, or +setuid-root, is given all capabilities no matter what capabilities the +parent might or might not hold at the time of execution. Quoting verbatim +from the article: + +> * When SECURE_NOROOT is not set, then when a process executes a file, +> the new capability sets may be calculated as though the file had some +> file capability sets set fully populated. In particular: +> +> * The file inheritable and permitted sets will be full on if the +> process's real or effective uid is 0 (root) or the file is setuid +> root. +> +> * The file effective set will be full on if the process's effective +> uid is root or the file is setuid root. +> +> +> * When SECURE_NO_SETUID_FIXUP is not set, then when a process switches +> its real or effective uids to or from 0, capability sets are further +> shifted around: +> +> * If a process switches its effective uid from 0 to non-0, then its +> effective capability set is cleared. +> +> * If a process switches its real, effective, or saved uids from at +> least one being 0 to all being non-zero, then both the permitted +> and effective capabilities are cleared. +> +> * If a process sets its effective uid from non-zero to 0, then the +> effective capabilities are set equal to the permitted capabilities. + +The capabilities of the resulting process are determined by the following +formulas, again taken from the article, with p for Process and f for file: + +> pI' = pI +> pP' = fP | (fI & pI) +> pE' = pP' & fE + +The security under the above described model, being what at least some +of the applications I listed in my last comment employs, is basically +the following (also detailed somewhat in the same article): + +* Execute process as root (or setuid) to gain all capabilities. + +* Use the prctl [[system call]] to enable keepcaps for the process + (same(?) effect as enabling SECURE_NO_SETUID_FIXUP for the process). + keepcaps should be off by default. + +* setuid to a non-root user, and by doing so losing the possibility to + regain capabilities by simply starting a new process. + +* Drop all the capabilities except those few actually needed. + + +## Infrastructure details - Special UIDs approach + +The auth server must somehow keep track of three sets of capabilities. +I suggest keeping these three sets in three additional idvec's in the +authhandle struct, and will for the purpose of this description name +these pcaps (permitted), ecaps (effective) and icaps (inheritable). +This will simplify keeping track of the internal logic somewhat. +In addition to this, there is a need to keep track of the "keepcaps" flag +as described above. I suggest representing this with an int keepcaps +in the same struct. + +1. Expand authhandle struct with three additional idvecs and one integer. +Fix static functions handling the struct, such as destroy_authhandle. + +2. Implement the necessary logic in auth_makeauth to handle capabilities. + +Problems: +Assume that all capabilities are mapped onto uids, given names on the form +uid_<capability>, for example uid_cap_net_raw. Assume that the presence +of an uid in euids suggest that the capability is in the effective set +of the process, that the presence of this uid in auids suggests that it +is in the permitted set of the process, and that the presence of this +uid in aguids suggest that it is in the inheritable set of the process. +That they are internally stored in separate idvec's can be ignored as +an implementation detail. + +* The UID's have as it is different meanings depending on where in the + array they are positioned, and certain clients seem to rely on this. + The first UID in euids is the effective uid, and the first and second + UIDs in auids are the real and saved UIDS respectively. At least + some users of makeauth would need to made aware of capabilities, + for example setuid in glibc. + +* Setting/getting the keepcaps-flag is also a bit of a problem. To avoid + changing the auth interface yet another special UID could be used + for this purpose, although that seems to be really stretching it. + The cleaner solution would probably be to expand the interface with + something along the lines of auth_setkeepcaps/auth_getkeepcaps. + This interface would only be used by prctl. + +Another problem with this approach is that it seems a bit difficult +to oversee the affects that using other RPC's like fsys_getroot and +io_restrict_auth might have on capabilities. + + +## Infrastructure details - "extend-interfaces" approach + +This approach has started to seem like the better way to me, as the +usage of capabilities becomes more explicit through the entire "chain", +perhaps making it somewhat more easy to understand all the interactions. + +I suggest something like the following new interface methods: + +*** + + +### The auth interface + + routine auth_getauth_caps ( + handle: auth_t; + out euids: idarray_t; + out auids: idarray_t; + out egids: idarray_t; + out agids: idarray_t; + out ecaps: idarray_t; + out pcaps: idarray_t; + out icaps: idarray_t); + + routine auth_makeauth_caps ( + handle: auth_t; + other_handles: portarray_t; + euids: idarray_t; + auids: idarray_t; + egids: idarray_t; + agids: idarray_t; + ecaps: idarray_t; + pcaps: idarray_t; + icaps: idarray_t; + flags: int; /* keepcaps.. ? */ + out newhandle: mach_port_make_send_t); + + routine auth_server_authenticate_caps ( + handle: auth_t; + sreplyport reply: mach_port_poly_t; + rendezvous: mach_port_send_t; + newport: mach_port_poly_t; + out euids: idarray_t; + out auids: idarray_t; + out egids: idarray_t; + out agids: idarray_t; + out ecaps: idarray_t; + out pcaps: idarray_t; + out icaps: idarray_t); + + +### The io interface + + routine io_restrict_auth_caps ( + io_object: io_t; + RPT + out new_object: mach_port_send_t; + uids: idarray_t SCP; + gids: idarray_t SCP; + ecaps: idarray_t SCP); + + +### The fsys interface + + routine fsys_getroot_caps ( + fsys: fsys_t; + RPT + #ifdef FSYS_GETROOT_UREPLY + ureplyport ureply: mig_reply_port_t; + #endif + dotdot_node: mach_port_send_t; + gen_uids: idarray_t; + gen_gids: idarray_t; + out ecaps: idarray_t; + out pcaps: idarray_t; + out icaps: idarray_t; + flags: int; + out do_retry: retry_type; + out retry_name: string_t; + out file: mach_port_send_t); + +*** + +These are meant to be able to replace the old methods with +capability-aware methods, instead of merely complementing them. +The replacing work could then be made a more gradual process. Steps: + +* Extend authhandle with the same data members as in the UID-case. + +* Implement new _caps-functions according to described interface + extensions above, refactor code a bit to share common uid-handling + logic. Both makeauth's should drop all capabilities if switching from + uid 0 without having keepcaps. For example, keepcaps should be unset + by default. + +* Fix glibc. Extend hurd_id_data in hurd/id.h to store capabilities, + switch to capability aware functions where necessary. + +* io-reauthenticate. Fix implementations to use + auth_server_authenticate_caps instead. For this we also need somewhere + to save the caps, so it ties in with for example the extension of + iouser as mentioned in the details. + +* fsys_getroot. Implement fsys_getroot_caps in libdiskfs, trans, + libtreefs, libtrivs, libnetfs. Fix users of function in libdiskfs, + libfshelp, settrans, libtreefs, clookup. + +* io_restrict_auth. Implement io_restrict_auth_caps in libdiskfs, + libtreefs, libtrivfs, libnetfs, boot. Fix users in utils(symlink, + firmlink), libtrivs, term, clookup + +Among the problems might be that there are a lot of arguments that +needs to be passed around, and that it seems somewhat ugly to end up +with function names referencing caps in particular. + +Below some more about the specific capabilities. This should in large +be common to the two approaches above. + + +## Actually handing out the capabilities to process + +This seems like a good job for the file_exec route in the fs interface. +Quoting from the comments above the definition in fs.defs: "*Necessary +initialization, including authentication changes associated with set[ug]id +execution must be handled by the filesystem*". The capability-granting +functionality should to be added in at least the implementations in +libdiskfs and libnetfs as far as I can determine, and should be "easy +enough" once the infrastructure for implementing the file-related +capabilities (CAP_CHOWN etc.) are in place. This also seem to make +sense considering the future possibility for file capabilities. + + +## Some implementation details of individual capabilities. + +### Net-related capabilities. + +This turned out to be a bit less work than I had thought, as the +imported Linux code already seem to contain all the necessary checks. +What remains to do to implement all of these capabilities is mostly a +matter of putting some infrastructure in place. + +* In struct sock_user (pfinet.h), change isroot for idvec + caps. Alternatively, add idvec caps. + +* Change the function make_sock_user in socket.c to take an idvec caps + as a parameter and properly set the given caps to the corresponding + idvec in sock_user. + +* Fix users of make_sock_user: S_io_reauthenticate, S_io_restrict_auth, + S_socket_create, S_socket_accept. This should be doable with the + current infrastructure. For example, S_socket_create currently + sets isroot in the new sock_user from the corresponding member in + the trivfs_protid struct. This does not present a problem however, + as the latter struct also provides access to an iouser (iohelp.h) + from which the needed uids of the user should be available. + +* Fix up parts of source from Linux, modify task_struct add idvec, + modify prepare_current to take the caps idvec instead of isroot. + Re-implement the existing function capable(int cap) to actually check + for the capability passed as an argument instead of just return isroot. + +* Change a few isroot's at 3 other places in the code to check for + capabilities. Since these places all have access to isroot and thus by + implication the sock_user, they also have access to the new capability + information; no restructuring necessary. + + +### File-related capabilities + +While there are a lot of servers in the Hurd, I am not sure all of these +capabilities actually make sense to implement in all of them. + + +#### CAP_CHOWN + +Implementing this in libdiskfs should take care of it where it makes +sense. Servers using libdiskfs uses iouser from libiohelp to hold +user credentials. As noted above, this struct is already capable of +holding our capabilities as uid's or is otherwise extended to contain +the necessary idvecs if using the second general approach. Adding a +check along the lines of *idvec_contains(uid_cap_chown)* in function +diskfs_S_file_chown (file-chown.c) should be all that's needed. + +In libnetfs, netfs_attempt_chown is declared as a function that the +server using the library must implement. Any checks for chown rights +are however most likely performed on the server side, suggesting that +there is nothing we can do here to implement CAP_CHOWN. Even if we do +need to add something, an iouser containing the necessary information +to implement the checks in a manner analogous to that in libdiskfs seems +to be passed to each important function. + + +#### CAP_DAC_* + +These might actually make sense to implement in more servers, and the +logic seems somewhat involved. Need to add the necessary checks to +at least file_check_access, file_exec in libdiskfs. file_exec also in +libnetfs, probably. Possibly changes also in other places. + +The main difficulties overall seem to lie in getting the infrastructure +properly in place rather than implementing most of the individual +capabilities, and I have updated the schedule a bit in an attempt to +reflect this. + + +## Schedule updating + +The more I look into this the less time it seems likely to take to +do the work. Below is my best estimation at the moment, and I have +basically adjusted everything to what I think is more likely estimations. +If this is correct I would be more or less around midterm. I haven't +gone completely to the individual level as that doesn't seem to make +sense, but what is clustered together are either related capabilities +or a collection of capabilities classified roughly with regards to how +much I know about the area and how many different rights they control. +It's not my intention to "slack off" or anything, so if this estimation +were to be correct I could perhaps take a look at the xattrs-patch, +or spend the rest of my time fixing PATH_MAX-issues. Then again, maybe +there is some great difficulty hidden somewhere. + + +#### Some justifications: + +Dummy libcap, more or less done. +*1 day* (making sure it "fails gracefully" shouldn't really take longer than this) + +Application for testing, the beginnings of a fairly extensive "test suit" on Linux. +*2 days* + +Basic infrastructure. +*5 days*, depends on the chosen approach, but it is probably wise to +reserve at least a bunch of days for this. + +Implementations of prctl/capset/capget in libshouldbeinlibc, +or a port of libcap to the Hurd in any other way. +*5 days* + +CAP_NET_BIND_SERVICE, CAP_NET_RAW, CAP_NET_BROADCAST, CAP_NET_ADMIN +*4 days*, as noted above this should be easy, but it might uncover bugs +in the newly implemented infrastructure for example. + +CAP_CHOWN,CAP_FOWNER,CAP_FSETID +*2 days*, I think these only needs to be done in libdiskfs + +CAP_DAC_OVERRIDE,CAP_DAC_READ_SEARCH +*4 days*, these might need changes to various servers + +CAP_SYS_TIME,CAP_IPC_LOCK,CAP_KILL +CAP_SYS_NICE,CAP_SYS_CHROOT,CAP_SYS_BOOT +*2 weeks*, these are varied and I'm not that sure exactly how each should +be tackled so some research is needed. + +CAP_SETGID,CAP_SETUID,CAP_SYS_TTY_CONFIG +*4 days* + +CAP_SYS_ADMIN,CAP_SYS_RESOURCE,CAP_SYS_RAWIO +*2 weeks*, these too are pretty varied and some might need some individual +researching + +CAP_SETPCAP +*1 day* + + +## Schedule + +24/5 Start coding +25/5 Dummy libcap ready for use. +27/5 The beginnings of a "test suite", written on Linux. + 1/6 Basic infrastructure in place + 6/6 Dummy libcap extended with real functionality to make use of + implemented capability and infrastructure, or the Hurd adapted for + compatibility with Linux libcap. +10/6 The "network class" of capabilities implemented. +12/6 CAP_CHOWN, CAP_FOWNER, CAP_FSETID +16/6 CAP_DAC_OVERRIDE, CAP_DAC_READ_SEARCH +30/6 CAP_SYS_TIME, CAP_IPC_LOCK, CAP_KILL, CAP_SYS_NICE, + CAP_SYS_CHROOT, CAP_SYS_BOOT + 4/7 CAP_SETGID,CAP_SETUID,CAP_SYS_TTY_CONFIG +12/7 "Mentors and students can begin submitting mid-term evaluations" +16/7 GSoC Mid-term evaluations deadline. +18/7 CAP_SYS_ADMIN, CAP_SYS_RESOURCE, CAP_SYS_RAWIO +19/7 CAP_SETPCAP diff --git a/community/gsoc/project_ideas/libdiskfs_locking.mdwn b/community/gsoc/project_ideas/libdiskfs_locking.mdwn index 2a08b387..faac8bd9 100644 --- a/community/gsoc/project_ideas/libdiskfs_locking.mdwn +++ b/community/gsoc/project_ideas/libdiskfs_locking.mdwn @@ -1,15 +1,18 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="Fix libdiskfs Locking Issues"]] +[[!tag open_issue_hurd]] + Nowadays the most often encountered cause of Hurd crashes seems to be lockups in the [[hurd/translator/ext2fs]] server. One of these could be traced recently, and turned out to be a lock inside [[hurd/libdiskfs]] that was taken @@ -18,17 +21,21 @@ faulty paths causing these lockups. The task is systematically checking the [[hurd/libdiskfs]] code for this kind of locking issues. To achieve this, some kind of test harness has to be implemented: For -exmple instrumenting the code to check locking correctness constantly at -runtime. Or implementing a unit testing framework that explicitely checks +example instrumenting the code to check locking correctness constantly at +runtime. Or implementing a [[open_issues/unit_testing]] framework that explicitly checks locking in various code paths. (The latter could serve as a template for -implementing unit checks in other parts of the Hurd codebase...) +implementing unit tests in other parts of the Hurd codebase...) -(A systematic code review would probably suffice to find the existing locking +(A [[systematic code review|open_issues/security]] would probably suffice to find the +existing locking issues; but it wouldn't document the work in terms of actual code produced, and thus it's not suitable for a GSoC project...) -This task requires experience with debugging locking issues in multithreaded -applications. +This task requires experience with debugging locking issues in +[[multithreaded|open_issues/multithreading]] applications. + +Tools have been written for automated [[open_issues/code_analysis]]; these can help to +locate and fix such errors. Possible mentors: Samuel Thibault (youpi) diff --git a/community/gsoc/project_ideas/libgtop.mdwn b/community/gsoc/project_ideas/libgtop.mdwn index ef1b2bcb..41897a1f 100644 --- a/community/gsoc/project_ideas/libgtop.mdwn +++ b/community/gsoc/project_ideas/libgtop.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2010, 2012 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,12 +11,16 @@ is included in the section entitled [[!meta title="Porting libgtop"]] +/!\ On 2012-05-05 Andjos reported (commit +web.git:8061106f2d1f15fa9a54947bc45d4cba68d89bba) that this task has already +been completed. + libgtop is a library used by many applications (especially GNOME applications) to abstract the system-specific methods for obtaining information about the current state of the system -- processes running, system load etc. -A [[Linux-compatible_procfs|madhusudancs]] implementation has been created -during GSoC 2008, and should cover a large part of the functionality of +A Linux-compatible [[hurd/translator/procfs]] is available +and should cover a large part of the functionality of libgtop. However, not all necessary information is exported via /proc (even on Linux); there are some bits still missing in the Hurd procfs implementation; and there are a couple of bugs that need to be fixed to make it fully usable. @@ -24,9 +29,9 @@ The goal of this project is a fully functional libgtop in Debian GNU/Hurd. Some application(s) using it also need to be ported, e.g. gnome-system-monitor. Some bits of this work are easy, others need some digging into Hurd internals. -This task doesn't require any specific previous knowlegde (besides of general +This task doesn't require any specific previous knowledge (besides of general C/UNIX programming skills of course); but during the course of the project, -some knowlegde about Hurd internals will have to be obtained, along with a bit +some knowledge about Hurd internals will have to be obtained, along with a bit of Debian stuff. Possible mentors: Samuel Thibault (youpi) diff --git a/community/gsoc/project_ideas/maxpath.mdwn b/community/gsoc/project_ideas/maxpath.mdwn index 5be8917f..4a1314c2 100644 --- a/community/gsoc/project_ideas/maxpath.mdwn +++ b/community/gsoc/project_ideas/maxpath.mdwn @@ -13,7 +13,7 @@ is included in the section entitled POSIX describes some constants (or rather macros) like PATH_MAX/MAXPATHLEN and similar, which may be defined by the system to indicate certain limits. Many people overlook the *may* though: Systems only should define them if they -actually have such fixed limits. The Hurd, following the GNU Coding Standards, +actually have such fixed limits (see [limits.h](http://pubs.opengroup.org/onlinepubs/009695399/basedefs/limits.h.html)). The Hurd, following the GNU Coding Standards, tries to avoid this kind of arbitrary limits, and consequently doesn't define the macros. diff --git a/community/gsoc/project_ideas/mtab.mdwn b/community/gsoc/project_ideas/mtab.mdwn index 045533e6..a60a8038 100644 --- a/community/gsoc/project_ideas/mtab.mdwn +++ b/community/gsoc/project_ideas/mtab.mdwn @@ -22,7 +22,7 @@ is no central place keeping track of mounts. As a consequence, there is currently no easy way to obtain a listing of all mounted file systems. This also means that commands like `df` can only work on -explicitely specified mountpoints, instead of displaying the usual listing. +explicitly specified mountpoints, instead of displaying the usual listing. One possible solution to this would be for the translator startup mechanism to update the `mtab` on any `mount`/`unmount`, like in traditional systems. @@ -31,7 +31,7 @@ with passive translators, i.e., translators that are not presently running, but set up to be started automatically whenever the node is accessed? Probably these should be counted among the mounted filesystems; but how to handle the `mtab` updates for a translator that is not started yet? Generally, being -centralized and event-based, this is a pretty unelegant, non-hurdish solution. +centralized and event-based, this is a pretty inelegant, non-hurdish solution. A more promising approach is to have `mtab` exported by a special translator, which gathers the necessary information on demand. This could work by @@ -64,10 +64,10 @@ other filtering rules might be desirable. After taking decisions on the outstanding design questions, the student will implement both the actual [[mtab_translator|hurd/translator/mtabfs]], and the -necessery interface(s) for gathering the data. It requires getting a good +necessary interface(s) for gathering the data. It requires getting a good understanding of the translator mechanism and Hurd interfaces in general. -Possible mentors: Olaf Buddenhagen (antrik) +Possible mentors: Olaf Buddenhagen (antrik), Carl Fredrik Hammar (cfhammar) Exercise: Make some improvement to any of the existing Hurd translators. Especially those in [hurdextras](http://www.nongnu.org/hurdextras/) are often diff --git a/community/gsoc/project_ideas/namespace-based_translator_selection.mdwn b/community/gsoc/project_ideas/namespace-based_translator_selection.mdwn index d40d73ac..f668b6f2 100644 --- a/community/gsoc/project_ideas/namespace-based_translator_selection.mdwn +++ b/community/gsoc/project_ideas/namespace-based_translator_selection.mdwn @@ -1,14 +1,21 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2012 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] -[[!meta title="Namspace-based Translator Selection"]] +[[!meta title="Namespace-based Translator Selection"]] + +[!] [[Sergiu Ivanov|scolobb]] has been working *voluntarily* on this task an +inofficial GSoC 2008 participant. Not all the desired functionality is in +place yet, though. + +--- The main idea behind the Hurd is to make (almost) all system functionality user-modifiable ([[extensible_system|extensibility]]). This includes a @@ -26,7 +33,7 @@ file, and presents a virtual file with the uncompressed contents. Or the other way around. Or a translator that presents an [[XML_file_as_a_directory_tree|hurd/translator/xmlfs]]. Or an mbox as a set of individual files for each mail ([[hurd/translator/mboxfs]]); or ever further -breaking it down into headers, body, attachements... +breaking it down into headers, body, attachments... This gets even more powerful when translators are used as building blocks for larger applications: A mail reader for example doesn't need backends for @@ -40,7 +47,7 @@ There are a few problems with the way translators are set, though. For one, once a translator is set on a node, you always see the translated content. If you need the untranslated contents again, to do a backup for example, you first need to remove the translator again. Also, having to set a translator -explicitely before accessing the contents is pretty cumbersome, making this +explicitly before accessing the contents is pretty cumbersome, making this feature almost useless. A possible solution is implementing a mechanism for selecting translators @@ -61,7 +68,7 @@ place of the original ones. In the long run it's probably desirable to have the mechanism implemented in the standard name lookup mechanism, so it will be available globally, and avoid -the overhead of a proxy; but for the beginnig the proxy solution is much more +the overhead of a proxy; but for the beginning the proxy solution is much more flexible. The goal of this project is implementing a prototype proxy; perhaps also a @@ -75,8 +82,3 @@ Possible mentors: Olaf Buddenhagen (antrik) Exercise: Try to make some modification to the existing unionfs and/or firmlink translators. (More specific suggestions welcome... :-) ) - -*Status*: Sergiu Ivanov has been working *voluntarily* on -[[namespace-based_translator_selection|scolobb]], as an inofficial GSoC 2008 -participant! Not all the desired functionality is in place yet; work is -ongoing. diff --git a/community/gsoc/project_ideas/namespace-based_translator_selection/discussion.mdwn b/community/gsoc/project_ideas/namespace-based_translator_selection/discussion.mdwn new file mode 100644 index 00000000..befd680a --- /dev/null +++ b/community/gsoc/project_ideas/namespace-based_translator_selection/discussion.mdwn @@ -0,0 +1,64 @@ +[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + + +# IRC, freenode, #hurd, 2012-04-22 + + <youpi> btw, I was wondering, when working on namespace mangling, did they + think about automatitioning ? + <youpi> autopartitioning, I meant + <youpi> i.e. with a foo.img file, open foo.img,,part1 + <braunr> what are you referring to with namespace mangling + <youpi> and voila + <youpi> I don't remember the exact term they used + <braunr> you mean there is a hurd library that parses names and can direct + to different services depending on part of the name ? + <youpi> namespace-based_translator_selection + <youpi> yes + <braunr> i thought it only handled directories + <braunr> well, the classical path representation + * civodul finds it ugly + <youpi> civodul: because of potential conflict, and the not-too-nice ",," + part? + <youpi> actually I wonder whether using directory access would be nicer + <youpi> i.e. you have a foo.gz, just open foo.gz/gunzip to get the unzipped + content + <youpi> and for foo.img.gz, open foo.img.gz/gunzip/part/1 + <civodul> youpi: because of the interpretation of special chars in file + names + <civodul> users should be free to use any character they like in file names + <civodul> foo.gz/gunzip looks nicer to me + <youpi> ok, so we agree + <youpi> that said, the user could choose the separator + <youpi> the namespace can be not run by root for everybody, but just for + your shell, run by yourself + <antrik> civodul: the user can't use any character anyways... '/' and '\0' + are reserved :-P + <civodul> antrik: '/' isn't quite reserved on the Hurd :-) + <civodul> you could implement dir_lookup such that it does something + special about it + <civodul> (server-side) + <antrik> civodul: as for overloading '/', although I haven't thought it + through entirely, I guess that would work for nodes that present as files + normally. however, it would *not* work for directory nodes + <antrik> which would be quite a serious limitation IMHO + <antrik> I can think of various kinds of useful directory translators + <antrik> what's more, one of the main use cases I originally had in mind is + a policy filter + <antrik> you could pass a directory name with a appropriate filter applied + to tar for example, so it wouldn't try to follow any translators + <antrik> I don't see why taking an obscure prefix like ,, would be much of + a problem in practice anyways + <antrik> (also, it doesn't strictly prevent the user from having such file + names... you just need to escape it if accessing such files through the + namespace multiplexer. though admittedly that would need some special + handling in *some* programs to work properly) diff --git a/community/gsoc/project_ideas/nfs.mdwn b/community/gsoc/project_ideas/nfs.mdwn index 683287f7..d4980279 100644 --- a/community/gsoc/project_ideas/nfs.mdwn +++ b/community/gsoc/project_ideas/nfs.mdwn @@ -1,22 +1,23 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2012 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="Improved NFS Implementation"]] The Hurd has both NFS server and client implementations, which work, but not -very well: File locking doesn't work properly (at least in conjuction with a +very well: File locking doesn't work properly (at least in conjunction with a GNU/Linux server), and performance is extremely poor. Part of the problems could be owed to the fact that only NFSv2 is supported so far. (Note though that locking on the Hurd is problematic in general, not only in -conjuction with NFS -- see the [[file_locking]] task.) +conjunction with NFS -- see the [[file_locking]] task.) This project encompasses implementing NFSv3 support, fixing bugs and performance problems -- the goal is to have good NFS support. The work done in @@ -31,6 +32,9 @@ has been done for a former GSoC application -- it might give you some pointers. But don't take any of the statements made there for granted -- check the facts yourself! +A bigger subtask is the [[libnetfs: `io_map`|open_issues/libnetfs_io_map]] +issue. + This task, [[!GNU_Savannah_task 5497]], has no special prerequisites besides general programming skills, and an interest in file systems and network protocols. diff --git a/community/gsoc/project_ideas/package_manager.mdwn b/community/gsoc/project_ideas/package_manager.mdwn index 43e53f7c..23304f6b 100644 --- a/community/gsoc/project_ideas/package_manager.mdwn +++ b/community/gsoc/project_ideas/package_manager.mdwn @@ -37,7 +37,7 @@ came about. There are no global databases of any kind. (Some things might require caching for better performance, but this must happen transparently.) The core of this approach is formed by [[hurd/translator/stowfs]], which -creates a traditional unix directory structure from all the files in the +creates a traditional Unix directory structure from all the files in the individual package directories. But this only handles the lowest level of package management. Additional mechanisms are necessary to handle stuff like dependencies on other packages. diff --git a/community/gsoc/project_ideas/perl.mdwn b/community/gsoc/project_ideas/perl.mdwn deleted file mode 100644 index bfe1968e..00000000 --- a/community/gsoc/project_ideas/perl.mdwn +++ /dev/null @@ -1,30 +0,0 @@ -[[!meta copyright="Copyright © 2009 Free Software Foundation, Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] - -[[!meta title="Improving Perl Support"]] - -Perl is available on the Hurd, but there are quite a lot of test suite -failures. These could be caused by problems in the system-specific -implementation bits of Perl, and/or shortcomings in the actual system -functionality which Perl depends on. - -The goal of this project is to fix all of these problems if possible, or at -least some of them. Some issues might require digging quite deep into Hurd -internals, while others are probably easy to fix. - -Note that while some Perl knowledge is probably necessary to understand what -the test suite failures are about, the actual work necessary to fix these -issues is mostly C programming -- in the implementation of Perl and/or the -Hurd. - -Possible mentors: Samuel Thibault (youpi) - -Exercise: Make some improvement to Perl support on the Hurd, e.g. fixing one of -the known test suite failures. diff --git a/community/gsoc/project_ideas/perl_python.mdwn b/community/gsoc/project_ideas/perl_python.mdwn new file mode 100644 index 00000000..a51393f1 --- /dev/null +++ b/community/gsoc/project_ideas/perl_python.mdwn @@ -0,0 +1,41 @@ +[[!meta copyright="Copyright © 2009, 2010, 2011 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="Improving Perl or Python Support"]] + +<!-- See also open_issues/perl and open_issues/python. --> + +Perl and Python are available on the Hurd, but there are quite a lot of test suite +failures. These could be caused by problems in the system-specific +implementation bits of Perl/Python, and/or shortcomings in the actual system +functionality which Perl/Python depends on. + +The student applying for this project can pick either Perl or Python, +whichever he is more comfortable with. +(Perl is higher priority though; and there are more failures too.) + +The goal then is to fix all of the problems with the chosen language if possible, or at +least some of them. Some issues might require digging quite deep into Hurd +internals, while others are probably easy to fix. + +Note that while some Perl/Python knowledge is probably necessary to understand what +the test suite failures are about, the actual work necessary to fix these +issues is mostly C programming -- in the implementation of Perl/Python and/or the +Hurd. + +Possible mentors: Samuel Thibault (youpi) + +Exercise: Take a stab at one of the testsuite failures, +and write a minimal testcase exposing the underlying problem. +Actually fixing it would be a bonus of course -- +but as it's hard to predict which issues will be easy and which will be tricky, +we will already be satisfied if the student makes a good effort. +(We hope to see some discussion of the problems in this case though :-) ) diff --git a/community/gsoc/project_ideas/procfs.mdwn b/community/gsoc/project_ideas/procfs.mdwn index 85eec43c..ac5fa6d8 100644 --- a/community/gsoc/project_ideas/procfs.mdwn +++ b/community/gsoc/project_ideas/procfs.mdwn @@ -1,15 +1,25 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2011, 2012 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="procfs"]] +[!] Madhusudan.C.S has implemented a new, fully functional +[[procfs|madhusudancs]] as a [[GSoC 2008 project|2008]]. + +[!] This was not the end of the story: [[jkoenig's +`procfs`|hurd/translator/procfs]] is yet another re-written and +improved version. + +--- + Although there is no standard (POSIX or other) for the layout of the `/proc` pseudo-filesystem, it turned out a very useful facility in GNU/Linux and other systems, and many tools concerned with process management use it. (`ps`, `top`, @@ -33,13 +43,10 @@ interfaces.) This project requires learning [[hurd/translator]] programming, and understanding some of the internals of process management in the Hurd. It should not be too hard coding-wise; and the task is very nicely defined by the -exising Linux `/proc` interface -- no design considerations necessary. +existing Linux `/proc` interface -- no design considerations necessary. **Note**: We already have several applications for this task. Possible mentors: Olaf Buddenhagen (antrik) Exercise: Add or fix one piece in the existing procfs translator. - -*Status*: Madhusudan.C.S has implemented a new, fully functional [[procfs|madhusudancs]] for -GSoC 2008. He is still working on some outstanding issues. diff --git a/community/gsoc/project_ideas/pthreads.mdwn b/community/gsoc/project_ideas/pthreads.mdwn index 9c703d36..2270c774 100644 --- a/community/gsoc/project_ideas/pthreads.mdwn +++ b/community/gsoc/project_ideas/pthreads.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,15 +11,18 @@ is included in the section entitled [[!meta title="Convert Hurd Libraries and Servers to pthreads"]] +[[!tag open_issue_libpthread]] + The Hurd was originally created at a time when the [pthreads standard](http://www.opengroup.org/onlinepubs/009695399/basedefs/pthread.h.html) didn't exist yet. Thus all Hurd servers and libraries are using the old [[cthreads|hurd/libcthreads]] package that came with [[microkernel/Mach]], -which is not compatible with [[pthreads|hurd/libpthread]]. +which is not compatible with pthreads. Not only does that mean that people hacking on Hurd internals have to deal with a non-standard thread package, which nobody is familiar with. Although a -pthreads implementation for the Hurd was created in the meantime, it's not +[[pthreads implementation for the Hurd|libpthread]] +was created in the meantime, it's not possible to use both cthreads and pthreads in the same program. Consequently, pthreads can't presently be used in any Hurd servers -- including translators. diff --git a/community/gsoc/project_ideas/secure_chroot.mdwn b/community/gsoc/project_ideas/secure_chroot.mdwn index a433e8d1..bfaf330b 100644 --- a/community/gsoc/project_ideas/secure_chroot.mdwn +++ b/community/gsoc/project_ideas/secure_chroot.mdwn @@ -1,26 +1,27 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="Secure chroot Implementation"]] As the Hurd attempts to be (almost) fully [[UNIX]]-compatible, it also implements a -`chroot()` system call. However, the current implementation is not really +`chroot()` [[system call]]. However, the current implementation is not really good, as it allows easily escaping the `chroot`, for example by use of [[passive_translators|hurd/translator]]. Many solutions have been suggested for this problem -- ranging from simple -workaround changing the behaviour of passive translators in a `chroot`; -changing the context in which passive translators are exectuted; changing the +workaround changing the behavior of passive translators in a `chroot`; +changing the context in which passive translators are executed; changing the interpretation of filenames in a chroot; to reworking the whole passive -translator mechanism. Some involving a completely different approch to -`chroot` implementation, using a proxy instead of a special system call in the +translator mechanism. Some involving a completely different approach to +`chroot` implementation, using a proxy instead of a special [[system call]] in the filesystem servers. See <http://tri-ceps.blogspot.com/2007/07/theory-of-filesystem-relativity.html> @@ -38,7 +39,7 @@ new mechanisms. (Translators.) More important than the actual code is the documentation of what he did: he must be able to defend why he chose a certain approach, and explain why he believes this approach really secure. -Possible mentors: ? +Possible mentors: Carl Fredrik Hammar (cfhammar) Exercise: It's hard to come up with a relevant exercise, as there are so many possible solutions... Probably best to make an improvement to one of the diff --git a/community/gsoc/project_ideas/server_overriding.mdwn b/community/gsoc/project_ideas/server_overriding.mdwn index 42edf287..c35d88de 100644 --- a/community/gsoc/project_ideas/server_overriding.mdwn +++ b/community/gsoc/project_ideas/server_overriding.mdwn @@ -14,7 +14,7 @@ The main idea of the Hurd is that every user can influence almost all system functionality ([[extensible_system|extensibility]]), by running private Hurd servers that replace or proxy the global default implementations. -However, running such a cumstomized subenvironment presently is not easy, +However, running such a customized subenvironment presently is not easy, because there is no standard mechanism to easily replace an individual standard server, keeping everything else. (Presently there is only the [[hurd/subhurd]] method, which creates a completely new system instance with a completely @@ -40,7 +40,7 @@ mechanism would have to check an override table on each lookup, and apply the desired replacement whenever a match is found. Another approach would be server-side overrides. Again there are various -variants. The actual servers themself could provide a mechanism to redirect to +variants. The actual servers themselves could provide a mechanism to redirect to other servers on request. (3) Or we could use some more generic server-side namespace overrides: Either all filesystem servers could provide a mechanism to modify the namespace they export to certain clients (4), or proxies could be @@ -68,7 +68,7 @@ discussing this topic, from a previous year's GSoC application -- see <http://lists.gnu.org/archive/html/bug-hurd/2007-06/msg00082.html>, <http://lists.gnu.org/archive/html/bug-hurd/2008-03/msg00039.html>. -Possible mentors: Olaf Buddenhagen (antrik) +Possible mentors: Olaf Buddenhagen (antrik), Carl Fredrik Hammar (cfhammar) Exercise: Come up with a glibc patch that allows overriding one specific standard server using method (1). diff --git a/community/gsoc/project_ideas/smp.mdwn b/community/gsoc/project_ideas/smp.mdwn new file mode 100644 index 00000000..e17c2ccf --- /dev/null +++ b/community/gsoc/project_ideas/smp.mdwn @@ -0,0 +1,16 @@ +[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="SMP"]] + + +# IRC, freenode, #hurd, 2012-09-30 + + <braunr> i expect smp to be our next gsoc project diff --git a/community/gsoc/project_ideas/sound.mdwn b/community/gsoc/project_ideas/sound.mdwn index b92f76da..8411831b 100644 --- a/community/gsoc/project_ideas/sound.mdwn +++ b/community/gsoc/project_ideas/sound.mdwn @@ -36,7 +36,7 @@ Possible mentors: Samuel Thibault (youpi) Exercise: This project requires kernel (driver framework) hacking as well as some Hurd server hacking; so the exercise should involve either of these, or even both. You could for example port some newer driver to run in the existing -framework (see the [[device_driver|driver_glue_code]] project descrption), or +framework (see the [[device_driver|driver_glue_code]] project description), or try to make some fix(es) to the [unfinished random device implementation](http://savannah.gnu.org/patch/?6088) created by Michael Casadevall. diff --git a/community/gsoc/project_ideas/tcp_ip_stack.mdwn b/community/gsoc/project_ideas/tcp_ip_stack.mdwn index b56bff51..331336ac 100644 --- a/community/gsoc/project_ideas/tcp_ip_stack.mdwn +++ b/community/gsoc/project_ideas/tcp_ip_stack.mdwn @@ -21,7 +21,7 @@ more flexibly. Rather than just having the standard socket interface, plus some lower-level hooks for special needs, there are explicit (perhaps filesystem-based) interfaces at all the individual levels; special application can just directly access the desired layer. All kinds of packet filtering, -routing, tunneling etc. can be easily achieved by stacking compononts in the +routing, tunneling etc. can be easily achieved by stacking components in the desired constellation. Implementing a complete modular network stack is not feasible as a GSoC @@ -34,7 +34,7 @@ make such a split later on feasible. This is [[!GNU_Savannah_task 5469]]. -Possible mentors: ? +Possible mentors: zhengda Exercise: You could try making some improvement to the existing pfinet implementation; or you could work towards running some existing userspace diff --git a/community/gsoc/project_ideas/testing_framework.mdwn b/community/gsoc/project_ideas/testing_framework.mdwn new file mode 100644 index 00000000..ff9899d9 --- /dev/null +++ b/community/gsoc/project_ideas/testing_framework.mdwn @@ -0,0 +1,55 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled +[[GNU Free Documentation License|/fdl]]."]]"""]] + +[[!meta title="Automated Testing Framework"]] + +Hurd development would benefit greatly from automated tests. +Unit tests should be added for all the major components +(Mach; Hurd servers and libraries). +Also, functional tests can be performed on various levels: +Mach; individual servers; and interaction of several servers. + +(The highest level would actually be testing libc functionality, +which in turn uses the various Hurd mechanisms. +glibc already comes with a test suite -- +though it might be desirabe to add some extra tests +for exercising specific aspects of the Hurd...) + +Our page on [[automated testing|open_issues/unit_testing]] collects some relevant material. + +The Goal of this task is to provide testing frameworks +that allow automatically running tests +as part of the Hurd and Mach build processes. +The student will have to create the necessary infrastrucure, +and a couple of sample tests employing it. +Ideally, all the aspects mentioned above should be covered. +At least some have to be ready for use and upstream merging +before the end of the summer. + +(As a bonus, +in addition to these explicit tests, +it would be helpful to integrate some methods +for testing [[locking validity|libdiskfs_locking]], +performing static code analysis etc.) + +This task probably requires some previous experience +with unit testing of C programs, +as well as dealing with complex build systems. +No in-depth knowledge about any specific parts of the Hurd should be necessary, +but some general understanding of the Hurd architecture +will have to be aquired to complete this project. +This makes it a very good project +to get started on Hurd development :-) + +Possible mentors: ? + +Exercise: Create a program performing some simple test(s) on the Hurd or Mach code. +It doesn't need to be integrated in the build process yet -- +a standalone progrem with it's own Makefile is fine for now. diff --git a/community/gsoc/project_ideas/testing_framework/discussion.mdwn b/community/gsoc/project_ideas/testing_framework/discussion.mdwn new file mode 100644 index 00000000..b01d13c3 --- /dev/null +++ b/community/gsoc/project_ideas/testing_framework/discussion.mdwn @@ -0,0 +1,272 @@ +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_documentation]] + +freenode, #hurd channel, 2011-03-05: + + <nixness> what about testing though? + <nixness> like sort of "what's missing? lets write tests for it so that + when someone gets to implementing it, he knows what to do. Repeat" + project + <antrik> you mean creating an automated testing framework? + <antrik> this is actually a task I want to add for this year, if I get + around to it :-) + <nixness> yeah I'd very much want to apply for that one + <nixness> cuz I've never done Kernel hacking before, but I know that with + the right tasks like "test VM functionality", I would be able to write up + the automated tests and hopefully learn more about what breaks/makes the + kernel + <nixness> (and it would make implementing the feature much less hand-wavy + and more correct) + <nixness> antrik, I believe the framework(CUnit right?) exists, but we just + need to write the tests. + <antrik> do you have prior experience implementing automated tests? + <nixness> lots of tests! + <nixness> yes, in Java mostly, but I've played around with CUnit + <antrik> ah, great :-) + <nixness> here's what I know from experience: 1) write basic tests. 2) + write ones that test multiple features 3) stress test [option 4) + benchmark and record to DB] + <youpi> well, what we'd rather need is to fix the issues we already know + from the existing testsuites :) + +[[GSoC project propsal|community/gsoc/project_ideas/testsuites]]. + + <nixness> youpi, true, and I think I should check what's available in way + of tests, but if the tests are "all or nothing" then it becomes really + hard to fix your mistakes + <youpi> they're not all or nothing + <antrik> youpi: the existing testsuites don't test specific system features + <youpi> libc ones do + <youpi> we could also check posixtestsuite which does too + +[[open_issues/open_posix_test_suite]]. + + <antrik> AFAIK libc has very few failing tests + +[[open_issues/glibc]]. + + <youpi> err, like twenty? + <youpi> € grep -v '^#' expected-results-i486-gnu-libc | wc -l + <youpi> 67 + <youpi> nope, even more + <antrik> oh, sorry, I confused it with coreutils + <pinotree> plus the binutils ones, i guess + <youpi> yes + +[[open_issues/binutils#weak]]. + + <antrik> anyways, I don't think relying on external testsuites for + regression testing is a good plan + <antrik> also, that doesn't cover unit testing at all + <youpi> why ? + <youpi> sure there can be unit testing at the translator etc. level + <antrik> if we want to implement test-driven development, and/or do serious + refactoring without too much risk, we need a test harness where we can + add specific tests as needed + <youpi> but more than often, the issues are at the libc / etc. level + because of a combination o fthings at the translator level, which unit + testing won't find out + * nixness yewzzz! + <nixness> sure unit testing can find them out. if they're good "unit" tests + <youpi> the problem is that you don't necessarily know what "good" means + <youpi> e.g. for posix correctness + <youpi> since it's not posix + <nixness> but if they're composite clever tests, then you lose that + granularity + <nixness> youpi, is that a blackbox test intended to be run at the very end + to check if you're posix compliant? + <antrik> also, if we have our own test harness, we can run tests + automatically as part of the build process, which is a great plus IMHO + <youpi> nixness: "that" = ? + <nixness> oh nvm, I thought there was a test stuie called "posix + correctness" + <youpi> there's the posixtestsuite yes + <youpi> it's an external one however + <youpi> antrik: being external doesn't mean we can't invoke it + automatically as part of the build process when it's available + <nixness> youpi, but being internal means it can test the inner workings of + certain modules that you are unsure of, and not just the interface + <youpi> sure, that's why I said it'd be useful too + <youpi> but as I said too, most bugs I've seen were not easy to find out at + the unit level + <youpi> but rather at the libc level + <antrik> of course we can integrate external tests if they exist and are + suitable. but that that doesn't preclude adding our own ones too. in + either case, that integration work has to be done too + <youpi> again, I've never said I was against internal testsuite + <antrik> also, the major purpose of a test suite is checking known + behaviour. a low-level test won't directly point to a POSIX violation; + but it will catch any changes in behaviour that would lead to one + <youpi> what I said is that it will be hard to write them tight enough to + find bugs + <youpi> again, the problem is knowing what will lead to a POSIX violation + <youpi> it's long work + <youpi> while libc / posixtestsuite / etc. already do that + <antrik> *any* unexpected change in behaviour is likely to cause bugs + somewher + <youpi> but WHAT is "expected" ? + <youpi> THAT is the issue + <youpi> and libc/posixtessuite do know that + <youpi> at the translator level we don't really + <youpi> see the recent post about link() + +[link(dir,name) should fail with +EPERM](http://lists.gnu.org/archive/html/bug-hurd/2011-03/msg00007.html) + + <youpi> in my memory jkoenig pointed it out for a lot of such calls + <youpi> and that issue is clearly not known at the translator level + <nixness> so you're saying that the tests have to be really really + low-level, and work our way up? + <youpi> I'm saying that the translator level tests will be difficult to + write + <antrik> why isn't it known at the translator level? if it's a translator + (not libc) bug, than obviously the translator must return something wrong + at some point, and that's something we can check + <youpi> because you'll have to know all the details of the combinations + used in libc, to know whether they'll lead to posix issues + <youpi> antrik: sure, but how did we detect that that was unexpected + behavior? + <youpi> because of a glib test + <youpi> at the translator level we didn't know it was an unexpected + behavior + <antrik> gnulib actually + <youpi> and if you had asked me, I wouldn't have known + <antrik> again, we do *not* write a test suite to find existing bugs + <youpi> right, took one for the other + <youpi> doesn't really matter actually + <youpi> antrik: ok, I don't care then + <antrik> we write a test suite to prevent future bugs, or track status of + known bugs + <youpi> (don't care *enough* for now, I mean) + <nixness> hmm, so write something up antrik for GSoC :) and I'll be sure to + apply + <antrik> now that we know some translators return a wrong error status in a + particular situation, we can write a test that checks exactly this error + status. that way we know when it is fixed, and also when it breaks again + <antrik> nixness: great :-) + <nixness> sweet. that kind of thing would also need a db backend + <antrik> nixness: BTW, if you have a good idea, you can send an application + for it even if it's not listed among the proposed tasks... + <antrik> so you don't strictly need a writeup from me to apply for this :-) + <nixness> antrik, I'll keep that in mind, but I'll also be checking your + draft page + <nixness> oh cool :) + <antrik> (and it's a well known fact that those projects which students + proposed themselfs tend to be the most successful ones :-) ) + * nixness draft initiated + <antrik> youpi: BTW, I'm surprised that you didn't mention libc testsuite + before... working up from there is probably a more effective plan than + starting with high-level test suites like Python etc... + <youpi> wasn't it already in the gsoc proposal? + <youpi> bummer + <antrik> nope + +freenode, #hurd channel, 2011-03-06: + + <nixness> how's the hurd coding workflow, typically? + +*nixness* -> *foocraft*. + + <foocraft> we're discussing how TDD can work with Hurd (or general kernel + development) on #osdev + <foocraft> so what I wanted to know, since I haven't dealt much with GNU + Hurd, is how do you guys go about coding, in this case + <tschwinge> Our current workflow scheme is... well... is... + <tschwinge> Someone wants to work on something, or spots a bug, then works + on it, submits a patch, and 0 to 10 years later it is applied. + <tschwinge> Roughly. + <foocraft> hmm so there's no indicator of whether things broke with that + patch + <foocraft> and how low do you think we can get with tests? A friend of mine + was telling me that with kernel dev, you really don't know whether, for + instance, the stack even exists, and a lot of things that I, as a + programmer, can assume while writing code break when it comes to writing + kernel code + <foocraft> Autotest seems promising + +See autotest link given above. + + <foocraft> but in any case, coming up with the testing framework that + doesn't break when the OS itself breaks is hard, it seems + <foocraft> not sure if autotest isolates the mistakes in the os from + finding their way in the validity of the tests themselves + <youpi> it could be interesting to have scripts that automatically start a + sub-hurd to do the tests + +[[hurd/subhurd#unit_testing]]. + + <tschwinge> foocraft: To answer one of your earlier questions: you can do + really low-level testing. Like testing Mach's message passing. A + million times. The questions is whether that makes sense. And / or if + it makes sense to do that as part of a unit testing framework. Or rather + do such things manually once you suspect an error somewhere. + <tschwinge> The reason for the latter may be that Mach IPC is already + heavily tested during normal system operation. + <tschwinge> And yet, there still may be (there are, surely) bugs. + <tschwinge> But I guess that you have to stop at some (arbitrary?) level. + <foocraft> so we'd assume it works, and test from there + <tschwinge> Otherwise you'd be implementing the exact counter-part of what + you're testing. + <tschwinge> Which may be what you want, or may be not. Or it may just not + be feasible. + <foocraft> maybe the testing framework should have dependencies + <foocraft> which we can automate using make, and phony targets that run + tests + <foocraft> so everyone writes a test suite and says that it depends on A + and B working correctly + <foocraft> then it'd go try to run the tests for A etc. + <tschwinge> Hmm, isn't that -- on a high level -- have you have by + different packages? For example, the perl testsuite depends (inherently) + on glibc working properly. A perl program's testsuite depends on perl + working properly. + <foocraft> yeah, but afaik, the ordering is done by hand + +freenode, #hurd channel, 2011-03-07: + + <antrik> actually, I think for most tests it would be better not to use a + subhurd... that leads precisely to the problem that if something is + broken, you might have a hard time running the tests at all :-) + <antrik> foocraft: most of the Hurd code isn't really low-level. you can + use normal debugging and testing methods + <antrik> gnumach of course *does* have some low-level stuff, so if you add + unit tests to gnumach too, you might run into issues :-) + <antrik> tschwinge: I think testing IPC is a good thing. as I already said, + automated testing is *not* to discover existing but unknown bugs, but to + prevent new ones creeping in, and tracking progress on known bugs + <antrik> tschwinge: I think you are confusing unit testing and regression + testing. http://www.bddebian.com/~hurd-web/open_issues/unit_testing/ + talks about unit testing, but a lot (most?) of it is actually about + regression tests... + <tschwinge> antrik: That may certainly be -- I'm not at all an expert in + this, and just generally though that some sort of automated testing is + needed, and thus started collecting ideas. + <tschwinge> antrik: You're of course invited to fix that. + +IRC, freenode, #hurd, 2011-03-08 + +(After discussing the [[open_issues/anatomy_of_a_hurd_system]].) + + <antrik> so that's what your question is actually about? + <foocraft> so what I would imagine is a set of only-this-server tests for + each server, and then we can have fun adding composite tests + <foocraft> thus making debugging the composite scenarios a bit less tricky + <antrik> indeed + <foocraft> and if you were trying to pass a composite test, it would also + help knowing that you still didn't break the server-only test + <antrik> there are so many different things that can be tested... the + summer will only suffice to dip into this really :-) + <foocraft> yeah, I'm designing my proposal to focus on 1) make/use a + testing framework that fits the Hurd case very well 2) write some tests + and docs on how to write good tests + <antrik> well, doesn't have to be *one* framework... unit testing and + regression testing are quite different things, which can be covered by + different frameworks diff --git a/community/gsoc/project_ideas/testsuites.mdwn b/community/gsoc/project_ideas/testsuites.mdwn new file mode 100644 index 00000000..9ca6fe3e --- /dev/null +++ b/community/gsoc/project_ideas/testsuites.mdwn @@ -0,0 +1,62 @@ +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="Fix Compatibility Problems Exposed by Testsuites"]] + +A number of software packages come with extensive testsuites. +Some notable ones are [[open_issues/glibc]], gnulib, Perl, +Python, GNU Coreutils, and glib. +While these testsuites were written mostly to track regressions in the respective packages, +some of the tests fail on the Hurd in general. + +There is also the [[Open POSIX Testsuite|open_issues/open_posix_test_suite]] +which is more of a whole system interface testing suite. + +Then, there is the [[open_issues/File_System_Exerciser]] which we can use to +test our file system servers for conformity. + +While in some cases these might point to wrong usage of system interfaces, +most of the time such failures are actually caused by shortcomings in Hurd's implementation of these interfaces. +These shortcomings are often not obvious in normal use, +but can be directly or indirectly responsible for all kinds of failures. +The testsuites help in isolating such problems, +so they can be tracked down and fixed. + +This task thus consists in running some of the mentioned testsuites +(and/or any other ones that come to mind), +and looking into the causes of failures. +The goal is to analyze all failures in one or more of the listed testsuites, +to find out what shortcomings in the Hurd implementation cause them (if any), +and to fix at least some of these shortcomings. + +Note that this task somewhat overlaps with the [[Perl/Python task|perl_python]] listed above. +Here the focus however is not on improving the support for any particular program, +but on fixing general problems in the Hurd. + +This is a very flexible task: +while less experienced students should be able to tackle at least a few of the easier problems, +other issues will be challenging even for experienced hackers. +No specific previous knowledge is required for this task; +only fairly decent C programming skills. +While tracking down the various issues, +the student will be digging into the inner workings of the Hurd, +and thus gradually gaining the knowledge required for Hurd development in general. + +A complementary task is adding a proper [[open_issues/unit_testing]] framework +to the GNU Hurd's code base, and related packages. + +Possible mentors: Samuel Thibault (youpi) + +Exercise: Take a stab at one of the testsuite failures, +and write a minimal testcase exposing the underlying problem. +Actually fixing it would be a bonus of course -- +but as it's hard to predict which issues will be easy and which will be tricky, +we will already be satisfied if the student makes a good effort. +(We hope to see some discussion of the problems in this case though :-) ) diff --git a/community/gsoc/project_ideas/tmpfs.mdwn b/community/gsoc/project_ideas/tmpfs.mdwn index 69adef0f..c38c6da8 100644 --- a/community/gsoc/project_ideas/tmpfs.mdwn +++ b/community/gsoc/project_ideas/tmpfs.mdwn @@ -1,21 +1,27 @@ -[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2012 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="Fix tmpfs"]] +[!] [[Maksym_Planeta]] has been making good progress here; status is tracked at +[[here|hurd/translator/tmpfs/discussion]]. + +--- + In some situations it is desirable to have a file system that is not backed by actual disk storage, but only by anonymous memory, i.e. lives in the RAM (and possibly swap space). A simplistic way to implement such a memory filesystem is literally creating a -ramdisk, i.e. simply allocating a big chunck of RAM (called a memory store in +ramdisk, i.e. simply allocating a big chunk of RAM (called a memory store in Hurd terminology), and create a normal filesystem like ext2 on that. However, this is not very efficient, and not very convenient either (the filesystem needs to be recreated each time the ramdisk is invoked). A nicer solution is @@ -37,7 +43,7 @@ implementation. It requires digging into some parts of the Hurd, including the [[pager_interface|hurd/libpager]] and [[hurd/translator]] programming. This task probably doesn't require any design work, only good debugging skills. -Possible mentors: ? +Possible mentors: Carl Fredrik Hammar (cfhammar) Exercise: Take a look at tmpfs and try to fix one of the existing issues. Some of them are probably not too tricky; or you might discover something else you diff --git a/community/gsoc/project_ideas/unionfs_boot.mdwn b/community/gsoc/project_ideas/unionfs_boot.mdwn index a801290f..d9f1a9e1 100644 --- a/community/gsoc/project_ideas/unionfs_boot.mdwn +++ b/community/gsoc/project_ideas/unionfs_boot.mdwn @@ -11,7 +11,7 @@ is included in the section entitled [[!meta title="Allow Using unionfs Early at Boot"]] In [[UNIX]] systems, traditionally most software is installed in a common directory -hierachy, where files from various packages live beside each other, grouped by +hierarchy, where files from various packages live beside each other, grouped by function: user-invokable executables in `/bin`, system-wide configuration files in `/etc`, architecture specific static files in `/lib`, variable data in `/var`, and so on. To allow clean installation, deinstallation, and upgrade of @@ -42,4 +42,4 @@ Completing this task will require gaining a very good understanding of the Hurd boot process and other parts of the design. It requires some design skills also to come up with a working mechanism. -Possible mentors: ? +Possible mentors: Carl Fredrik Hammar (cfhammar) diff --git a/community/gsoc/project_ideas/valgrind.mdwn b/community/gsoc/project_ideas/valgrind.mdwn new file mode 100644 index 00000000..e9e94857 --- /dev/null +++ b/community/gsoc/project_ideas/valgrind.mdwn @@ -0,0 +1,83 @@ +[[!meta copyright="Copyright © 2009, 2010, 2011 Free Software Foundation, +Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="Porting Valgrind to the Hurd"]] + +[[!tag open_issue_gnumach open_issue_hurd]] + +[Valgrind](http://valgrind.org/) is an extremely useful debugging tool for memory errors. +(And some other kinds of hard-to-find errors too.) +Aside from being useful for program development in general, +a Hurd port will help finding out why certain programs segfault on the Hurd, +although they work on Linux. +Even more importantly, it will help finding bugs in the Hurd servers themselfs. + +To keep track of memory use, +Valgrind however needs to know how each [[system call]] affects the validity of memory regions. +This knowledge is highly kernel-specific, +and thus Valgrind needs to be explicitely ported for every system. + +Such a port involves two major steps: +making Valgrind understand how kernel traps work in general on the system in question; +and how all the individual kernel calls affect memory. +The latter step is where most of the work is, +as the behaviour of each single [[system call]] needs to be described. + +Compared to Linux, +[[microkernel/Mach]] (the microkernel used by the Hurd) has very few kernel traps. +Almost all [[system call]]s are implemented as [[RPC]]s instead -- +either handled by Mach itself, or by the various [[Hurd servers|hurd/translator]]. +All RPCs use a pair of `mach_msg()` invocations: +one to send a request message, and one to receive a reply. +However, while all RPCs use the same `mach_msg()` trap, +the actual effect of the call varies greatly depending on which RPC is invoked -- +similar to the `ioctl()` call on Linux. +Each request thus must be handled individually. + +Unlike `ioctl()`, +the RPC invocations have explicit type information for the parameters though, +which can be retrieved from the message header. +By analyzing the parameters of the RPC reply message, +Valgrind can know exactly which memory regions are affected by that call, +even without specific knowledge of the RPC in question. +Thus implementing a general parser for the reply messages +will already give Valgrind a fairly good approximation of memory validity -- +without having to specify the exact semantic of each RPC by hand. + +While this should make Valgrind quite usable on the Hurd already, it's not perfect: +some RPCs might return a buffer that is only partially filled with valid data; +or some reply parameters might be optional, +and only contain valid data under certain conditions. +Such specific semantics can't be deduced from the message headers alone. +Thus for a complete port, +it will still be necessary to go through the list of all known RPCs, +and implement special handling in Valgrind for those RPCs that need it. + +The goal of this task is at minimum to make Valgrind grok Mach traps, +and to implement the generic RPC handler. +Ideally, specific handling for RPCs needing it should also be implemented. + +Completing this project will require digging into Valgrind's handling of [[system call]]s, +and into Hurd RPCs. +It is not an easy task, but a fairly predictable one -- +there shouldn't be any unexpected difficulties, +and no major design work is necessary. +It doesn't require any specific previous knowledge: +only good programming skills in general. +On the other hand, +the student will obtain a good understanding of Hurd RPCs while working on this task, +and thus perfect qualifications for Hurd development in general :-) + +Possible mentors: Samuel Thibault (youpi) + +Exercise: As a starter, +students can try to teach valgrind a couple of Linux ioctls, +as this will make them learn how to use the read/write primitives of valgrind. diff --git a/community/gsoc/project_ideas/virtualization.mdwn b/community/gsoc/project_ideas/virtualization.mdwn index c7403f70..bd67718b 100644 --- a/community/gsoc/project_ideas/virtualization.mdwn +++ b/community/gsoc/project_ideas/virtualization.mdwn @@ -25,18 +25,18 @@ desired constellations. The goal is to create a set of powerful tools for managing at least one desirable virtualization scenario. One possible starting point could be the [[hurd/subhurd]]/[[hurd/neighborhurd]] mechanism, which allows a second almost totally -independant instance of the Hurd in parallel to the main one. +independent instance of the Hurd in parallel to the main one. While subhurd allow creating a complete second system instance, with an own set of Hurd servers and [[UNIX]] daemons and all, there are also situations where it is -desirable to have a smaller subenvironment, living withing the main system and +desirable to have a smaller subenvironment, living within the main system and using most of its facilities -- similar to a chroot environment. A simple way to create such a subenvironment with a single command would be very helpful. It might be possible to implement (perhaps as a prototype) a wrapper using existing tools (chroot and [[hurd/translator/unionfs]]); or it might require more specific tools, -like some kind of unionfs-like filesytem proxy that mirrors other parts of the -filesystem, but allows overriding individual locations, in conjuction with +like some kind of unionfs-like filesystem proxy that mirrors other parts of the +filesystem, but allows overriding individual locations, in conjunction with either chroot or some similar mechanism to create a subenvironment with a different root filesystem. @@ -73,4 +73,4 @@ Completing this project will require gaining a very good understanding of the Hurd architecture and spirit. Previous experience with other virtualization solutions would be very helpful. -Possible mentors: Olaf Buddenhagen (antrik) +Possible mentors: Olaf Buddenhagen (antrik), Carl Fredrik Hammar (cfhammar) diff --git a/community/gsoc/project_ideas/vm_tuning.mdwn b/community/gsoc/project_ideas/vm_tuning.mdwn index 9e802188..ecc5f9f4 100644 --- a/community/gsoc/project_ideas/vm_tuning.mdwn +++ b/community/gsoc/project_ideas/vm_tuning.mdwn @@ -14,7 +14,7 @@ Hurd/[[microkernel/Mach]] presently make very bad use of the available physical system. Some of the problems are inherent to the system design (the kernel can't distinguish between important application data and discardable disk buffers for example), and can't be fixed without fundamental changes. Other -problems however are an ordinary lack of optimisation, like extremely crude +problems however are an ordinary lack of optimization, like extremely crude heuristics when to start paging. (See <http://lists.gnu.org/archive/html/bug-hurd/2007-08/msg00034.html> for example.) Many parameters are based on assumptions from a time when typical machines had like 16 MiB of RAM, or simply have been set to @@ -23,7 +23,7 @@ arbitrary values and never tuned for actual use. The goal of this project is to bring the virtual memory management in Hurd/Mach closer to that of modern mainstream kernels (Linux, FreeBSD), by comparing the implementation to other systems, implementing any worthwhile improvements, and -general optimisation/tuning. It requires very good understanding of the Mach +general optimization/tuning. It requires very good understanding of the Mach VM, and virtual memory in general. This project is related to [[!GNU_Savannah_task 5489]]. diff --git a/community/gsoc/project_ideas/xattr.mdwn b/community/gsoc/project_ideas/xattr.mdwn index 55961547..8ec4c42e 100644 --- a/community/gsoc/project_ideas/xattr.mdwn +++ b/community/gsoc/project_ideas/xattr.mdwn @@ -21,7 +21,7 @@ fields in the inode to store Hurd-specific metadata: most notable passive translator settings. As these fields are Hurd-specific, they can't be accessed by the standard methods from Linux for example, so it's not possible to fully work with a Hurd filesystem on GNU/Linux (copy, backup etc.); and also, even -when on Hurd, only tools that explicitely support the Hurd-specific information +when on Hurd, only tools that explicitly support the Hurd-specific information can handle them. Using extended attributes instead of custom fields for the Hurd-specific @@ -38,8 +38,8 @@ Completing this project will require digging into some parts of the Hurd, but it should be quite doable without previous Hurd experience. Some experience with xattrs might help a bit, but shouldn't be really necessary either. -Some previous work on xattr support is available in [[!GNU_Savannah_patch -5126]], and might serve as a starting point. +Some previous work on xattr support is [[available|open_issues/xattr]], and +might serve as a starting point. Possible mentors: Samuel Thibault (youpi) diff --git a/community/gsoc/student_application_form.mdwn b/community/gsoc/student_application_form.mdwn index 84070cbf..ba339dc9 100644 --- a/community/gsoc/student_application_form.mdwn +++ b/community/gsoc/student_application_form.mdwn @@ -21,7 +21,8 @@ title yourself of course -- but surely this isn't hard, if you were able to come up with your own project idea :-) Submitting the application form is only part of the deal: we expect a few other -things on top of that. Lacking these, the application is not complete, and we +things on top of that, as explained below. This is important, so please take +it seriously -- without these things, the application is not complete, and we won't consider it. One of the things we expect is that you contact us directly as soon as possible @@ -30,41 +31,44 @@ One of the things we expect is that you contact us directly as soon as possible in particular allows for very informal conversations. (Note though that we are not all in the same time zone, and people generally -don't stare at the IRC screen all the time -- it can take quite a long time -until somebody replies: even several hours. Don't get discouraged by that: Just +don't stare at the IRC screen all the time: it can take quite a long time +until somebody replies -- even several hours. Don't get discouraged by that. Just be patient and hang on, or try again later.) Contacting us as soon as possible is crucial, as regular communication is the single most important factor for a successful GSoC project. We need to see that -you are able and willing to talk to us regularily. Also it allows us getting to -know you much better than the application form alone could. +you are able and willing to talk to us regularly. Also, we get to +know you much better this way than what the application form alone would allow us to. -You shouldn't be at a loss for reason to contact us. You ought to discuss your -project and application with us for exmple: You will gain a much better idea -about the project, our expectations etc. -- in short, you will be able to -submit a better application right from the beginnig, saving both yourself and -us some tedious roundtrips :-) +You shouldn't be at a loss for reasons to contact us. You ought to discuss your +project and application with us for example -- you will gain a much better idea +about the project, our expectations etc. In short, you will be able to +submit a better application right from the beginning, saving both yourself and +us some tedious round trips :-) Also, if you really want to get involved with the Hurd project, there are -surely many things you will want to know :-) All in all, you should have ample +surely many things you will want to know -- after all, it's a fascinating +project, with a fascinating architecture etc., right? :-) + +All in all, you should have ample causes to get in touch during the application period. Bonus points if you also participate in discussions not directly related to your project. The other thing necessary to complete your application is making a change to -some part of the Hurd code, and submitting a patch with the change. (If you are +some part of the Hurd code, and submitting a patch implementing that change. (If you are not sure what that means, ask us!) This is important, as it shows that you have everything set up to start hacking -on the project (source code, tool chain, testing environment etc.), and that -you have all kinds of qualifications necessary to successfully finish the -project: general programming abilities; working in the Hurd environment; +on the project (source code, tool chain, testing environment etc.); and that +you have all kinds of qualifications necessary to successfully finish your +project: general programming skills; working in the Hurd environment; submitting patches and reacting to feedback; finding and/or asking for any information you need; and so on. Don't get us wrong: We absolutely do *not* demand that you have and know all this up front. After all, the idea of GSoC is to *introduce* you to free software development in general, and to our project specifically :-) We are -willing to help you with anything you will need to create the patch -- you just +eager to help you with anything you will need to create the patch -- you just need to ask! We actively encourage you to contact us whenever you have any doubts. Don't be @@ -72,7 +76,7 @@ afraid that we will think worse of you when you ask too much. On the contrary: this is an occasion for you to show us that whenever there is something you don't know yet, you are able to learn quickly, and know how to ask for help :-) -As for the kind of change we want: Ideally, it would be some real improvement +As for the kind of change we want: ideally, it would be some real improvement (bug fix or new feature) in a part of the Hurd related to the specific project you want to work on. (This is not always possible though -- in that case, a useful change to some unrelated part of the Hurd would also do, or perhaps some @@ -101,9 +105,17 @@ you are unsure about something. And now that you are prepared to face the enemy, here we go :-) +* Please supply your contact information here: full name, email address, IRC +nick, Jabber ID, phone number, etc. -- anything we might need to recognize you +and to keep in touch. + +* Introduce yourself: who are you, where are you from, what do you do, how did +you get here... Don't write a long essay here -- just a bunch of basic facts +you think we should know, so we get some idea whom we are talking to :-) + * Please describe the task of the project you want to work on, in your own words. Be as specific as possible. It's not sufficient to rephrase the -description from the project ideas page; show us that you actually understand +description from the project ideas page -- show us that you actually understand what this task involves! Read the available documentation (and possibly code) if necessary. And don't hesitate to ask us if you have any doubts :-) @@ -116,7 +128,7 @@ ready to be merged to mainline. Experience shows that adding the "final touches" necessary for that, tends to take up quite a lot of time -- there are always some bugs here and there, some misunderstandings about how things are supposed to work, build system issues, missing documentation, forgotten bits, -and so on. Thus, the schedule should suppose that a larger part of the main +and so on. Thus, the schedule should assume that a larger part of the main implementation work will be done by midterm! Also note that by the beginning of the summer session, you need to be able to @@ -162,12 +174,12 @@ the application process. * Do you have a permanent internet connection, especially during the time of the summer session? Are you able and willing to hang out on the Hurd IRC -channel regularily? (As in: Running the IRC client more or less permanently and +channel regularly? (As in: Running the IRC client more or less permanently and checking for activity now and then.) If it turns out that your mentor lives in a different time zone, could you shift your day/night rhythm to better match that of your mentor and other Hurd developers? -Hint: Hanging out on the channel regularily during the application process +Hint: Hanging out on the channel regularly during the application process would be a good start :-) * When does your university term end, when are your exams, and when does the @@ -190,8 +202,8 @@ intend to make sure you will be able to dedicate sufficient time to your project nevertheless? Please be open about this, and also mention things you are not yet sure about. -We can be flexible about time arrangements; but we need to know about any -possible obstacles up front. +We can be flexible about time arrangements; but we absolutely need to know about any +possible obstacles up front. Surprises on that score are not acceptable. * How do you intend to make sure that your code will keep on being maintained and supported properly after the end of the GSoC program? diff --git a/community/gsoc/xorg_ideas.mdwn b/community/gsoc/xorg_ideas.mdwn index 406d370d..26177345 100644 --- a/community/gsoc/xorg_ideas.mdwn +++ b/community/gsoc/xorg_ideas.mdwn @@ -8,51 +8,11 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -## libpciaccess Support for GNU Hurd - -xserver 1.5 introduces libpciaccess as a new method for handling PCI devices. -The purpose was to get rid of the messy old PCI code in the server, instead -using a new implementation that is encapsulated in a library, and uses external -PCI access interfaces provided by the operating systems. - -As libpciaccess is written from scratch, support for various operating systems -needs to be reimplemented individually, writing specific backends for each. -This task is about creating such a backend for GNU Hurd. - -Writing a "proper" native backend for the Hurd is rather involved, as there is -no kernel PCI interface so far -- it has to be created first. It would also -have the disadvantage of being somewhat tied to the specific driver framework: -The present framework is pretty dated (using some ancient Linux drivers with a -layer of glue code); and when this is replaced, the PCI interface will have to -be updated as well. - -Alternatively, a libpciaccess backend simply doing old-style register poking -from user space could be implemented. (Either from scratch, or by reusing some -existing PCI code from the old server or some BSD kernel.) That approach is not -as elegant; it would be somewhat fragile and limited, like the old PCI code in -the server. On the other hand, it is much simpler to implement; and it would -also benefit other non-mainstream platforms that can't afford -creating/maintaining a native backend. - -The goal of this project is getting a recent xorg server, using libpciaccess, -to run on the Hurd. - -This task probably doesn't require any previous X programming knowledge, though -a bit of experience with PCI programming will probably help. Some Hurd-specific -knowledge will have to be obtained while working on it -- especially if -implementing a proper kernel PCI interface. The task is pretty involved in the -latter case, but shouldn't be too hard if taking the user space register poking -approach. - -Exercise: Implement a stub backend for libpciaccess, which doesn't really do -anything useful, but compiles fine on the Hurd. - - ## VT Switching for GNU Hurd While XFree86 was first ported to the Hurd more than a decade ago, and there -are updates now and then to make newer versions of Xorg run as well (see also -the libpciaccess task), the support is quite rudimentary: in particular, there +are updates now and then to make newer versions of Xorg run as well, +the support is quite rudimentary: in particular, there is no support for switching back to the text console while X is running. Implementing this requires creating an interface between the X server and the @@ -74,7 +34,7 @@ The Direct Rendering Manager (DRM) is a kernel driver component taking care of graphics hardware access. Originally, it only took care of the 3D acceleration unit, and was used mostly by the DRI (Direct Rendering Infrastructure) in Mesa. -About two years ago, the developers came to the conclusion that a more robust +A few years ago, the developers came to the conclusion that a more robust and functional graphics stack requires the kernel driver to take care of other graphics access as well: mode setting in particular. (Essentially what the old KGI project proposed, see <http://www.kgi-project.org>.) Also, with the new GEM @@ -90,7 +50,7 @@ microkernel idea -- we want to run the drivers as priviledged user space server processes, rather than actual kernel modules. This task is about doing the first steps for porting the DRM to the Hurd. This -can be done by taking one of the existing DRM modesetting drivers (Intel or +can be done by taking one of the existing DRM modesetting drivers (Intel, Nouveau (Nvidia), or Radeon), trying to get parts of it running as a Hurd server, and porting/implementing necessary pieces of the general DRM framework as needed along the way. diff --git a/community/meetings.mdwn b/community/meetings.mdwn index 9c88418e..d715066e 100644 --- a/community/meetings.mdwn +++ b/community/meetings.mdwn @@ -1,22 +1,35 @@ -[[!meta copyright="Copyright © 2007, 2008, 2009 Free Software Foundation, -Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2009, 2010, 2011, 2012 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="Meetings with Hurd developer attendance"]] # Upcoming + +## In the Future + + * [[FOSDEM_2013]] * [[Self-organised]] + # Past + * [[GNU Hackers Meeting, 2012, Düsseldorf|ghm2012]] + * [[FOSDEM_2012]] + * [[FrOSCon_2011]] + * [[GNU Hackers Meeting, 2011, Paris|ghm2011]] + * [[FOSDEM_2011]] + * [[DebConf10]] + * [[GNU Hackers Meeting, 2010, Den Haag|ghm2010]] + * [[FOSDEM_2010]] * [[EuroSys_2009]] * [[FOSDEM_2008]] * [[Weekend_at_stesie's|stesie_2007-10-12]] diff --git a/community/meetings/debconf10.mdwn b/community/meetings/debconf10.mdwn new file mode 100644 index 00000000..3b83a8cc --- /dev/null +++ b/community/meetings/debconf10.mdwn @@ -0,0 +1,27 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="DebConf10"]] + +<http://debconf10.debconf.org/> + + * {{$banck_hurd}} + + +[[!ymlfront data=""" + +banck_hurd: + + "presentation (including video) by Michael Banck: [*Debian GNU/Hurd -- Past. + Present. And + Future?*](http://penta.debconf.org/dc10_schedule/events/595.en.html) + ([slides](http://people.debian.org/~mbanck/debian-hurd.pdf))" + +"""]] diff --git a/community/meetings/fosdem_2010.mdwn b/community/meetings/fosdem_2010.mdwn new file mode 100644 index 00000000..9def3c1c --- /dev/null +++ b/community/meetings/fosdem_2010.mdwn @@ -0,0 +1,86 @@ +[[!meta copyright="Copyright © 2006, 2007, 2008, 2009, 2010 Free Software +Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled +[[GNU Free Documentation License|/fdl]]."]]"""]] + +[[!meta title="FOSDEM 2010"]] + +<http://fosdem.org/2010> + +FOSDEM will take place on February 6th/7th at the Université Libre de +Bruxelles. + + +# Who And When + +[[!table class="table_style_1" data=""" +"Name","Attending","Arrival","Return","Share room with us" +"Anatoly A. Kazantsev","no","","","" +"Andrei Barbu","no","","","" +"Arne Babenhauserheide","no","","","" +"[[Bas Wijnen|baswijnen]]","Yes","Friday evening","Sunday afternoon","no" +"Ben Asselstine","yes","?","?","yes (but no dates confirmed)" +"Carl Fredrik Hammar","no","","","" +"Colin Leitner","no","","","" +"Emilio Pozuelo Monfort","probably not","","","" +"Flavio Cruz","no","","","" +"[[Gianluca Guida|GianlucaGuida]]","yes","Fr.","Mo.","yes" +"Guillem Jover","no","","","" +"Madhusudan C.S.","?","","","" +"Marcus Brinkmann","yes","fr","mo","yes" +"[[Michael Banck|MichaelBanck]]","yes","Fr., early afternoon","Su., night","yes" +"Neal Walfield","yes","Fr","Mo","yes" +"Olaf Buddenhagen","yes","fr","mo","yes" +"Pino Toscano","no","","","" +"[[Samuel Thibault|SamuelThibault]]","no","","","" +"Sergiu Ivanov","no","","","" +"[[Soeren Schulze|SoerenSchulze]]","yes","Fr","Mo","yes" +"[[Stefan Siegl|stesie]]","no","","","" +"[[Thomas Schwinge|tschwinge]]","yes","Fr., 10:35","Mo., 18:25","yes" +"Vasily Sartakov","maybe","","","" +"Zheng Da","no","","","" +"""]] + + +# Where (Accommodation) + +This year, we'll stay in the [Hotel Astrid](http://www.astridhotel.be/), +together with the FSFE folks. The following data (without Ben's) has been +forwarded to them: + +[[!table class="table_style_1" data=""" + , Ben , Gianluca, Marcus, Michael, Neal, Olaf, Thomas, Total +Fri 5, **?**, 1 , 1 , 1 , 1 , 1 , 1 , 6 +Sat 6, **?**, 1 , 1 , 1 , 1 , 1 , 1 , 6 +Sun 7, **?**, 1 , 1 , , 1 , 1 , 1 , 5 +"""]] + + +# What + +On Saturday, Bas will be giving a [talk about *Iris*, his new +kernel](http://fosdem.org/2010/schedule/events/emb_iris) (Sat., 18:00, Embedded +Developer Room). + +On Sunday, one can join the other folks at the [Alt-OS Developer +Room](http://fosdem.org/2010/schedule/devrooms/altos) (also see the [Haiku +FOSDEM2010AltOSDevroomSchedule +page](http://dev.haiku-os.org/wiki/FOSDEM2010AltOSDevroomSchedule)) where at +least some of us will [spend their +time](http://lists.gnu.org/archive/html/bug-hurd/2009-12/msg00080.html). + +At this very place, Olaf will be giving two presentations: [*Why is Anyone +Still Working on the GNU +Hurd?*](http://fosdem.org/2010/schedule/events/altos_hurd) (Sun., 10:30, Alt-OS +Developer Room), and [*Porting KGI graphics drivers from Linux to GNU +Hurd*](http://fosdem.org/2010/schedule/events/altos_kgi_hurd) (Sun., 13:00, +Alt-OS Developer Room). + +There'll be further GNU folks around; [Mini GNU Hackers Meeting at FOSDEM +Brussels 2010](http://www.gnu.org/ghm/2010/fosdem/). diff --git a/community/meetings/fosdem_2011.mdwn b/community/meetings/fosdem_2011.mdwn new file mode 100644 index 00000000..8522e10f --- /dev/null +++ b/community/meetings/fosdem_2011.mdwn @@ -0,0 +1,36 @@ +[[!meta copyright="Copyright © 2006, 2007, 2008, 2009, 2010, 2011 Free Software +Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="FOSDEM 2011"]] + +<http://fosdem.org/2011> + +FOSDEM will take place on February 5th/6th at the Université Libre de +Bruxelles. + + +# Who and When + +[[!table class="table_style_1" data=""" +"Name","Attending","Arrival","Return","Share room with us" +"Emilio Pozuelo Monfort","yes"," "," "," " +"Marcus Brinkmann","yes"," "," "," " +"Michael Banck","yes"," "," "," " +"Neal Walfield","yes","Friday noon","Monday noon","yes" +"Richard Braun","no","n/a","n/a","n/a" +"[[Thomas Schwinge|tschwinge]]","no","n/a","n/a","n/a" +"Wouter van Heyst","yes"," "," "," " +"""]] + + +# What + +<http://lists.gnu.org/archive/html/bug-hurd/2010-12/msg00019.html> diff --git a/community/meetings/fosdem_2012.mdwn b/community/meetings/fosdem_2012.mdwn new file mode 100644 index 00000000..8143e236 --- /dev/null +++ b/community/meetings/fosdem_2012.mdwn @@ -0,0 +1,34 @@ +[[!meta copyright="Copyright © 2006, 2007, 2008, 2009, 2010, 2011, 2012 Free +Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="FOSDEM 2012"]] + +<http://fosdem.org/2012> + +FOSDEM will take place on February 4th/5th at the Université Libre de +Bruxelles. + + +# Who and When + +[[!table class="table_style_1" data=""" +"Name","Attending","Arrival","Return","Share room with us" +"[[Maksym Planeta]]","no" +"Olaf Buddenhagen","yes","","","yes" +"Richard Braun","no" +"Svante Signell","no" +"[[Thomas Schwinge|tschwinge]]","no" +"""]] + + +# Multiserver, Microkernel-Based Operating Systems Devroom + +[Announcement](http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00137.html). diff --git a/community/meetings/fosdem_2013.mdwn b/community/meetings/fosdem_2013.mdwn new file mode 100644 index 00000000..4acf0c90 --- /dev/null +++ b/community/meetings/fosdem_2013.mdwn @@ -0,0 +1,58 @@ +[[!meta copyright="Copyright © 2012 Free +Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="FOSDEM 2013"]] + +<http://fosdem.org/2013> + +FOSDEM will take place on February 2th/3th at the Université Libre de +Bruxelles. + + +# Who and When + +[[!table class="table_style_1" data=""" +"Name","Attending","Arrival","Return","Share room with us" +"Samuel Thibault","yes","","","yes" +"""]] + + +# Multiserver, Microkernel-Based Operating Systems Devroom + +[Announcement](https://lists.fosdem.org/pipermail/microkernel-devroom/2012-October/000054.html). + +Talk proposal: + +title: The GNU/Hurd architecture, nifty features, and latest news + +Full name: Samuel Thibault + +Short abstract: +GNU/Hurd aims at being a general-purpose Operating System with a strong emphasis +on flexibility and freedom for the user, and thus based on a design made +of a micro-kernel surrounded by a hird of userland servers. It has however +a long-term vaporware reputation; development has indeed been relatively +dormant for some time, but recent regain of interest has brought interesting +improvements and stabilization, to the point that there will be a non-official +release of the GNU/Hurd variant of Debian Wheezy, with about 75% of the Debian +packages, including classical graphical desktop application (gnome, gnumeric, +firefox, ...) + +This talk will present to GNU/Hurd in general and its "translator" mechanism +which replaces the traditional notion of filesystem by userland processes, so +as to provide strong flexibility to users and administrators, and we will demo +it live. The subhurd/neighborhurd mechanism, a very natural way to provide +virtualization container support on GNU/Hurd, will also be presented. We will +also present recent developments, notably in terms of DDE device drivers run as +userland processes, and discuss about maintenance of DDE. + +Duration: +30m, 45m? diff --git a/community/meetings/froscon_2011.mdwn b/community/meetings/froscon_2011.mdwn new file mode 100644 index 00000000..b15140d6 --- /dev/null +++ b/community/meetings/froscon_2011.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="FrOSCon, 2011, Sankt Augustin, Germany"]] + +<http://www.froscon.de/> + + * [Arch Hurd booth](http://www.froscon.de/en/exhibitors/projekte.html#c1413) diff --git a/community/meetings/ghm2010.mdwn b/community/meetings/ghm2010.mdwn new file mode 100644 index 00000000..c8a9d8c3 --- /dev/null +++ b/community/meetings/ghm2010.mdwn @@ -0,0 +1,26 @@ +[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="GNU Hackers Meeting, 2010, Den Haag"]] + +<http://www.gnu.org/ghm/2010/denhaag/> + + * {{$walfield_hurd}} + + +[[!ymlfront data=""" + +walfield_hurd: + + "video of the presentation by Neal Walfield: [*GNU/Hurd: It's About Freedom + (Or: Why you should + care)*](http://audio-video.gnu.org/video/ghm2010/GNU-Hurd_-_Its_About_Freedom,_Or_Why_you_should_care.ogv)" + +"""]] diff --git a/community/meetings/ghm2011.mdwn b/community/meetings/ghm2011.mdwn new file mode 100644 index 00000000..8e77d500 --- /dev/null +++ b/community/meetings/ghm2011.mdwn @@ -0,0 +1,27 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="GNU Hackers Meeting, 2011, Paris"]] + +<http://www.gnu.org/ghm/2011/paris/> + + * {{$thibault_hurd}} + + +[[!ymlfront data=""" + +thibault_hurd: + + "presentation by Samuel Thibault: [*GNU/Hurd, aka. Extensibility from the + Ground*](http://www.gnu.org/ghm/2011/paris/#outline-container-2-5) + ([slides](http://www.gnu.org/ghm/2011/paris/slides/samuel-thibault-hurd.pdf), + [video](http://audio-video.gnu.org/video/ghm2011/Samuel_Thibault-GNU_Hurd.ogv))" + +"""]] diff --git a/community/meetings/ghm2012.mdwn b/community/meetings/ghm2012.mdwn new file mode 100644 index 00000000..0e3c8cd5 --- /dev/null +++ b/community/meetings/ghm2012.mdwn @@ -0,0 +1,13 @@ +[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="GNU Hackers Meeting, 2012, Düsseldorf"]] + +<http://www.gnu.org/ghm/2012/ddorf/> diff --git a/community/weblogs.mdwn b/community/weblogs.mdwn index 50791549..28f413eb 100644 --- a/community/weblogs.mdwn +++ b/community/weblogs.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,8 +11,14 @@ is included in the section entitled Weblogs from Hurd programmers and enthusiasts. +[[!map +pages="community/weblogs/* and !community/weblogs/*/*" +show=title]] + +--- + [[!inline -pages="community/weblogs/*/* and !*/discussion" +pages="community/weblogs/*/* and !community/weblogs/*/*/*" show=0 actions=no rootpage="community/weblogs" postformtext="Add a new user named:"]] diff --git a/community/weblogs/ArneBab.mdwn b/community/weblogs/ArneBab.mdwn index 4ef8a329..1e80e5a8 100644 --- a/community/weblogs/ArneBab.mdwn +++ b/community/weblogs/ArneBab.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2009 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -39,7 +39,7 @@ See you in the Hurd! ----- My Blog ----- [[!inline -pages="community/weblogs/ArneBab/* and !*/discussion" +pages="community/weblogs/ArneBab/* and !community/weblogs/ArneBab/*/*" show=0 actions=no rootpage="community/weblogs/ArneBab" postformtext="Add a new entry named:"]] diff --git a/community/weblogs/ArneBab/2011-04-06-application-pyhurd.mdwn b/community/weblogs/ArneBab/2011-04-06-application-pyhurd.mdwn new file mode 100644 index 00000000..b0e57bfb --- /dev/null +++ b/community/weblogs/ArneBab/2011-04-06-application-pyhurd.mdwn @@ -0,0 +1,148 @@ +Python Bindings for the Hurd (PyHurd) +========================== + +## Contact information + +- Name: Arne Babenhauserheide +- E-Mail Address: arne_bab@web.de +- IRC-nick: ArneBab @ freenode +- Jabber-ID: arne@jabber.fsfe.org +- Phone-number: XXXXXXXXX +- GnuPG key: http://draketo.de/inhalt/ich/pubkey.txt + +## Who I am + +I am a physics student from Heidelberg, Germany, a passionate free software user and roleplayer, and I started contributing to the Hurd in minor ways about 5 years ago. Now my coding skills are good enough (and I have enough time) that I feel ready to tackle a GSoC project - and I want to take the chance GSoC offers and do a focussed effort for contributing to free software before I am no longer a student. I married 4 years ago and now have a 5½ month old son whoose happy laughing can make you forget everything around you - or at least it does that to me, but what else could you expect to hear from his father about him ;) + +## Project + +For this years GSoC I want to turn the currently rudimentary Python Bindings of the Hurd into a complete Python-library for low-level Hurd and Mach hacking with high level functionality to allow for easy creation of complex applications. Particularly it should make it possible to utilize the whole Python standard library for translators. + +## Preliminary Schedule + + * *Community bonding period.* + Read up on the current C-interface to the Hurd and Cython. Especially grok the Hurd hacking guide. Add docstrings to all existing source files (where they are missing) explaining what they do. Add auto-generated API-docs. Deliverable: Easy to understand API-docs of the current PyHurd, a simple way to generate them from the sources automatically. + * *May 23.* + Coding starts. + * *May 30.* + Finished a basic Hello World translator, naively implementing the necessary Mach parts directly in the translator. + 1. A simple program which gets a Mach port and can receive messages on that port. It has to get and hold its port at startup and send a reply port, needs to use mach_msg to get the messages, should be able to deallocate the port and must have a kill condition (for example 10 received messages). + 2. stdout functionality, to print all Mach messages (for debugging and to make sure that they really get received entirely). + 3. a parser for the Mach read file message similar to trivfs\_S\_io\_read + * *June 6.* + Moved the functionality for reading into a simple API using decorators to define actions + and ported Hello World to use it: + + """Show Hello World.""" + from translator import repres + @repres.text.event + def on_text_read(size): + return "Hello, World!"[:size] + * *June 13.* + Implemented single file read-write in the API. Added a simple writethrough translator. The API code is nicely commented. + * *June 20.* + Access Control and file attributes. Added lock_file translator which just adjusts the effective file access codes and can be used to lock a file as long as the translator is used. Might be useful for quick testing. + * *June 27.* + Translator commandline arguments and testing. + * *July 4.* + Translator: Overlay with backend store: write changes to a different file. Makes any file writeable, but keeps the changes only visible for the user who set up the translator. Effectively single-file unionmount. + * *July 11.* + Mid-term: trivfs in python works: It is possible to write translators in Python with relative ease. + * *July 18.* + More complex, specialized and helper translator libraries, along with example translators. This should recreate some of the Hurd libraries for Python and add convenience options. + * *July 25.* + Full featured setttrans in Python. + * *August 1.* + Redesigned and realized an updated controlling API with the existing direct Cython bindings. + * *August 8.* + More translators and integrating into the build system. + * *August 15.* + Suggested Pencils down. The translator API is easy to use, there are many example translators and there is a full featured settrans command in Python using the easier controlling API which shows how to control the Hurd directly from Python. The code is pushed to <https://github.com/ArneBab/PyHurd> and a git repo at <http://git.savannah.gnu.org/cgit/hurd> and integrated into the build system with a switch to enable building PyHurd. + * *August 22.* + Firm pencils down. + + +## Initial Fix + +Initial Fix: Making PyHurd build again under Cython 0.14.1. Sent as patch series to bug-hurd@gnu.org + +## Detailed answers + +### What I have to learn, and what I already know + +I need to dive into the detailed interfaces of the Hurd to get a better understanding of the exact requirements for a well usable Python interface, especially for higher level functionality, and read up more on working with Cython. + +I already know Python and I did design my share of interfaces for my own hobby projects ([TextRPG][], [Fungus][], [evolve-keyboard-layout][] and others). + +[TextRPG]: https://bitbucket.org/ArneBab/textrpg/ +[Fungus]: https://bitbucket.org/ArneBab/fungus +[evolve-keyboard-layout]: https://bitbucket.org/ArneBab/evolve-keyboard-layout + +Also I know the functionality and design of the Hurd from a user perspective and can code in C and C++. + +### Why did you choose this project idea? What do you consider most appealing about it? + +FIrstoff: It is about making it possible for me to hack on the Hurd using my favorite programming language. + +Also I can learn more about accessing low-level interfaces directly (as opposed to just using higher level abstractions) and grok the ins and outs of creating Python extensions - into which I wanted to dive for a long time now. + +And I helped getting the project running and am intrigued by how far it can be pushed. + +### Have you been involved in any free software ("Open Source") projects yet? Which projects, how long, and in what way have you been involved? Have you been active in the Hurd project/Hurd community before? + +I worked on documentation and news for the Hurd, wrote two plugins and the usage guide for Mercurial and created a bunch of personal Python projects. Also I generally try to nudge other Hurd developers into the direction of actually getting the system useful for people (and communicating its strengths) - and do the same for the freenet project. + +In my opinion, my major contribution to the Hurd is the Month of the Hurd, a try at fixing the Hurds reputation for never being finished. To achieve that goal, the Month of the Hurd only lists actually testable successes for which I can easily describe how they get the Hurd closer to its vision, ideally those which are already committed. + +### Please briefly describe the Hurd, including the goals, architecture etc. Also, what makes you interested in the Hurd? Why do you want to work on it? What is your vision of it's future development? + +The Hurd offers much greater freedom for users compared to Linux, because every user can change his/her environment to a much greater extent. + +Also it allows for easier low-level tinkering, making it possible for hobby-hackers to work on stuff which in linux requires dabbling with kernel-sources. Also it makes it much easier to test these low-level work, so a community can spawn which informally shares low-level hacks, giving a much bigger momentum for low-level work. + +And it allows for containment of potentially dangerous applications using subhurds. As a very simple example, I can open a webbrowser without giving it access to the internet and just add that capability later, when I really want to go online (as opposed to just showing local files). + +But mainly: + + settrans -a ftp\: /hurd/hostmux /hurd/ftpfs / + dpkg -i ftp://ftp.gnu.org/…/*.deb + +And that’s only the beginning. + +### Are you subscribed to the bug-hurd@gnu.org mailing list? (See http://lists.gnu.org/mailman/listinfo/bug-hurd ) + +Yes :) + +### Do you have a permanent internet connection, especially during the time of the summer session? Are you able and willing to hang out on the Hurd IRC channel regularly? (As in: Running the IRC client more or less permanently and checking for activity now and then.) If it turns out that your mentor lives in a different time zone, could you shift your day/night rhythm to better match that of your mentor and other Hurd developers? + +Yes, a permanent internet connection as well as a permanently running computer. Since I’m used to also work later in the evening (on hobby projects), the time zone should not be a major issue. + +### When does your university term end, when are your exams, and when does the next term begin? + +I have a clean timetable for the summer: No exams anymore. + +### How much time do you intend to spend on your GSoC project per day/week during the summer months? + +I plan to spend at least 40 hours per week on the PyHurd. + +### What other major activities will you engage in during the summer? (Moving apartments, longer vacations, other obligations, etc.) If any, how do you intend to make sure you will be able to dedicate sufficient time to your project nevertheless? + +Finding a job for after the GSoC. This should not take too much time, all in all, but rather mean short out-times now and then. + +### How do you intend to make sure that your code will keep on being maintained and supported properly after the end of the GSoC program? + +My main plan to keep it maintained is to comment it cleanly, and naturally to keep using the Hurd and PyHurd itself, so any breakage will bother me personally. + +Also i want to get it merged into the main git repositories, so it is directly accessible for later developers. + +### Anything else you want to add to your application? + +I’d love to work on PyHurd, because it grips me more and more. For example a high level API might get as simple as + + from translator.source.text import * + from translator.repres.tree import * + def source_text_changed(text): … (adapt tree object) + def repres_tree_changed(tree): … (adapt text object) + → 2-way connectingk,5 + writeonly is then done by simply leaving out the definition for the source_<whatever>_changed. + source is the node below and repres is the translated node diff --git a/community/weblogs/ArneBab/2011-04-06-application-python-test.mdwn b/community/weblogs/ArneBab/2011-04-06-application-python-test.mdwn new file mode 100644 index 00000000..4c20afa1 --- /dev/null +++ b/community/weblogs/ArneBab/2011-04-06-application-python-test.mdwn @@ -0,0 +1 @@ +Cancelled. See [[community/weblogs/ArneBab/2011-04-06-application-pyhurd]] instead. diff --git a/community/weblogs/ArneBab/Hurd-showcase-qemu-image.mdwn b/community/weblogs/ArneBab/Hurd-showcase-qemu-image.mdwn new file mode 100644 index 00000000..00d09094 --- /dev/null +++ b/community/weblogs/ArneBab/Hurd-showcase-qemu-image.mdwn @@ -0,0 +1,111 @@ +I’m currently preparing a qemu image for the Hurd which allows testing the capabilities of the Hurd with as little effort as possible. + +**Work in progress. These are my in-development notes.** + +For that I want to use: + +* An up to date debian image (no longer online, but I have a copy here). +* My [Hurd Intro](http://bitbucket.org/ArneBab/hurd_intro), +* Translators from [hurd-extras](http://www.nongnu.org/hurdextras/) and the [incubator](http://git.savannah.gnu.org/cgit/hurd/incubator.git/), and naturally +* a lot of apt-get update; apt-get upgrade and apt-get dist-upgrade :) (all worked flawlessly). + +## Working + +### Generally + + # ssh with public key + apt-get install random-egd + ssh-keygen + + # build tools + apt-get install build-essential + +### StoreIO + + # mount an iso image + mount foo.iso bar -t iso9660fs + # see myfile as device + settrans foo /hurd/storeio myfile + # so that means I can pack a complete chroot (300MB) into a file with storeio and ext2fs — giselher + + # nfs mount anywhere (TODO: check this with antrik) + mount server:/home /home -t nfs + settrans /home /hurd/nfs server:/home + +## In Progress + +### Hurdextras + + hg clone <hurdextras repo> + +### httpfs + + # pkg-config is needed to avoid “PKG_CHECK_MODULES syntax error near unexpected token `HTTPFS,'” + # pkg-config must be installed before you run autoreconf. + apt-get install autoconf autoconf-archive libxml2-dev pkg-config + autoreconf -i -f + ./configure + make + make install + + settrans -ac gnu /usr/local/httpfs www.gnu.org/ + # (breaks, because libxml2 needs pthreads → work to do.) + # (what we need: pthreads in translators. → see the [work of Barry](https://savannah.gnu.org/task/?func=detailitem&item_id=5487)) + # check: for i in `objdump -x /usr/local/bin/httpfs |grep NEEDED| sed s/.*NEEDED//`; do echo $i; objdump -x /usr/lib/$i | grep pthread; objdump -x /lib/$i | grep pthread; done + +### Tarfs + + apt-get install zip libz-dev libbz2-dev + git clone git://git.sv.gnu.org/hurd/incubator.git tarfs + cd tarfs/ + git checkout tarfs/master + cd tarfs + make + make install + # works, though with warnings. + + settrans -ca new /hurd/tarfs -cz test/intro.tar.gz + cp repos/intro/README new/ + settrans -g new + tar -tf test/intro.tar.gz + # works + + tar -cf test/intro.tar repos/intro + settrans -ac t /hurd/tarfs test/intro.tar + # (settrans: /hurd/tarfs: Translator died :( ⇒ more work to do ) + +### nsmux + + git clone git://git.sv.gnu.org/hurd/incubator.git nsmux + cd nsmux/ + git checkout -b nsmux origin/nsmux + + apt-get install autoconf autoconf-archive + autoreconf -i -f + ./configure + make + make install + + cd ../.. + mkdir test + touch test/hello + settrans -ca test2 /usr/local/bin/nsmux test + # tar -cvf test/intro.tar repos/hurd_intro + cat test2/hello + cat test2/hello,,hello + # Hello, World! + +### clisp + + git clone git://git.sv.gnu.org/hurd/incubator.git clisp + cd clisp/ + git checkout -b clisp origin/clisp + + apt-get install texi2html + make + make install + + +### debugging Translators + + rpctrace diff --git a/community/weblogs/ArneBab/What_a_Hurd_release_should_be_able_to_do_for_me.mdwn b/community/weblogs/ArneBab/What_a_Hurd_release_should_be_able_to_do_for_me.mdwn new file mode 100644 index 00000000..51ef2a85 --- /dev/null +++ b/community/weblogs/ArneBab/What_a_Hurd_release_should_be_able_to_do_for_me.mdwn @@ -0,0 +1,40 @@ +I thought a bit about what I’d need from Hurd to use it for some of my real life tasks. + +My desktop has to be able to do everything it does now, and that under high load, so it currently is no useful target for the Hurd. + +But then I have an OLPC XO sitting here, and I use it mostly for work and for clearly defined tasks. As such it seems natural to me to check, what the Hurd would have to be able to do to support my workflow on the OLPC. + +### What I need + +* Writing text and programming Python with emacs. - *works*. +* Use Mercurial for my versiontracked stuff. - *works*. +* Reading websites with emacs and w3m or with lynx. - *works*. +* Use SSH to go on my desktop and on the university machine. - *should work*. +* Run X11 with dwm and emacs. - *should work*. +* Boot Hurd on the OLPC from a USB stick. - *not yet*? +* Support networking over wlan and wpa_supplicant. - *not yet*? Might DDE kit help? +* Listen to music with Quod Libet in X11. - *not yet*. Needs audio support. + +### What would be nice + +* Run a Gentoo system. - not *really* needed, but nice to update my system with the same tools. +* Watch videos with mplayer. - unlikely. Even with Linux as kernel watching videos pushes my XO to the limit. But this is not essential. + + +So, as soon as Debian GNU/Hurd (or Arch Hurd) supports the things I need, I’ll put it on a USB-stick and use it for coding and writing. + +To be frank: I’d likely even use it without audio-support. I have an mp3 player and can feed it via USB. So the essential features for me are: + +### Essential features + +* Writing text and programming Python with emacs. - works. +* Use Mercurial for my versiontracked stuff. - works. +* Use SSH to go on my desktop and on the university machine. - should work. +* Boot Hurd on the OLPC from a USB stick. - not yet? +* Support networking over wlan and wpa_supplicant. - not yet? Might DDE kit help? + +### Conclusion + +The Hurd doesn’t yet do everything I need for my OLPC, but it isn’t that far away either. Grub already gets [ported to OLPC](http://grub.enbug.org/OLPC), so what’s missing to make the Hurd a work system for me are just *booting on OLPC from USB stick* and *wlan-support on OLPC*. + +All the rest I need for work is already in place. diff --git a/community/weblogs/ArneBab/how-i-write-a-qoth.mdwn b/community/weblogs/ArneBab/how-i-write-a-qoth.mdwn new file mode 100644 index 00000000..87b1f07d --- /dev/null +++ b/community/weblogs/ArneBab/how-i-write-a-qoth.mdwn @@ -0,0 +1,44 @@ +I just read on the hurd IRC channel (chat: #hurd at irc.freenode.net), that people consider my work valuable (I knew that, and I think that myself, but it is still nice to hear), so I want to dispell any possible myth about it :) + +What I do is not hard - at least not anymore, since I created a simple structure for it (But it still takes time). + +First I open up the relevant mailing lists for the quarter. I get them from [[contributing/web_pages/news/writing_the_qoth]]. Normally I just use the following: + +* <http://lists.gnu.org/archive/html/bug-hurd/YYYY-MM/threads.html> +* <http://lists.debian.org/debian-hurd/YYYY/MM/> + +Then I copy them 3 times and use M-x replace-string (in emacs) to adjust them to the correct months. + +Additionally I open the Arch Hurd news: + +* <http://www.archhurd.org/news.php> +* <http://planet.archhurd.org/> + +Having all those news at hand, I read every thread-starter and every news-item. For each of them I first check if I understand them (no use trying to explain something I don’t get myself) and if they provide a way for people to test what they improved (however complex that might be), then I + +* note the name of the main contributor(-s), +* write a line of text what it does (often partly copied from the news-item), +* add a link to the news-item, a code-repo or a patch and +* a note how that new development helps achieve the goals_of_the_Hurd (see [[contributing/web_pages/news/writing_the_qoth]] for details). + +With that list of short news I go into [[contributing/web_pages/news/qoth_next]]. + +Now I identify 2 to 4 main news items by some kind of “helps the Hurd most when more people know it”, “biggest change” and similar fudgery :) + +Finally I sort all the news items by intuition, crude logic I develop on-the-fly writing and the goal of making the qoth read somewhat like nice prose. + +On the way to that I commit every little to medium step. I never know when I have to abort due to an interruption (I’m sure tschwinge loves my super-non-atomic horrible-to-review commits :-) - but better that than losing work == time, and I try to prefix the commit-messages with “news:” so he knows that it’s useless to review them as in-flight-patches…). + +Having finished the text (usually after 3 to 6 hours of overall work), I send it by mail to bug-hurd: <http://lists.gnu.org/archive/html/bug-hurd/> + +After about a week I incorporate the comments from there and publish the qoth as described in [[contributing/web_pages/news/writing_the_qoth]]. + +Then tschwinge reviews it, does some last-minute changes and pushes it from the staging wiki to the website. + +And that’s it. + +I hope this small insight was interesting to you. Happy hacking and have fun with the Hurd! + +-- Arne Babenhauserheide + +PS: Writing this blog entry took about 20 minutes. The raw text is longer than a qoth, but it is much faster to write, because it avoids the main time-eater: Gathering the info with the necessary references to make sure that people can test what’s in here. diff --git a/community/weblogs/ArneBab/metadata-and-program-communication-akonadi-nepomuk-dbus.mdwn b/community/weblogs/ArneBab/metadata-and-program-communication-akonadi-nepomuk-dbus.mdwn new file mode 100644 index 00000000..41c82c83 --- /dev/null +++ b/community/weblogs/ArneBab/metadata-and-program-communication-akonadi-nepomuk-dbus.mdwn @@ -0,0 +1,20 @@ +Just ideas for more elegant implementations of dbus and akonadi/nepomuk using Hurd interfaces + +tagging: + + settrans ~/ /hurd/nsmux + ls ~/file,,metadata + +store in ~/.metadata + +network store: search for .metadata + +All metadata: + + settrans meta /hurd/metadata --show-store + +dbus: + + settrans -a /dbus /hurd/dbus + +Programs just add an active translator in /dbus: /dbus/org.… → receives dbus calls in-process. diff --git a/community/weblogs/ArneBab/niches_for_the_hurd.mdwn b/community/weblogs/ArneBab/niches_for_the_hurd.mdwn index ff169b0a..5febe7b6 100644 --- a/community/weblogs/ArneBab/niches_for_the_hurd.mdwn +++ b/community/weblogs/ArneBab/niches_for_the_hurd.mdwn @@ -94,16 +94,16 @@ sounds like an interesting option.*" *"While I believe this can be applied to any kind of applications, I'm personally most interested in more efficient and powerful desktop environments -- these considerations are in fact what got me seriously -interested in the Hurd. +interested in the Hurd.* -Even more specifically, I've done most considerations (though by far not +*Even more specifically, I've done most considerations (though by far not all) on modular web browsing environments. Those interested can read up -some of my thoughts on this: +some of my thoughts on this:* http://sourceforge.net/mailarchive/message.php?msg_name=20080909073154.GB821%40alien.local -(Just skip the text mode browsing stuff -- the relevant part is the long +*(Just skip the text mode browsing stuff -- the relevant part is the long monologue at the end... I really should put these ideas into my blog.)"* @@ -127,7 +127,17 @@ or even: * cp ftp://foo/bar/ogg play -that's KDEs fabled network transparency on the filesystem / shell level. +that's KDEs fabled network transparency on the filesystem / shell level (where it belongs to be desktop agnostic). + +* add temporary filesystems anywhere via `settrans -a NODE /hurd/ext2fs` + +* On-demand mounted filesystems via a passive translator which unmounts the filesystem when it isn’t used for some time. + +* make everything temporarily writeable without really changing it via [[hurd/translator/unionfs]]. Store the changes on an external device. + +* Read tar archives and mbox files via `ls foo.tar.gz,,tarfs` and `ls foo.mbox,,mboxfs`, respectively → [[hurd/translator/nsmux]]. + +* Use stuff like the new akonady (personal information) framework in KDE more efficiently from the shell. Reality check @@ -149,7 +159,7 @@ aren't possible. * elegantly mount iso images and similar as unprivileged user. - Other useful stuff: - * Install deb-packages from an ftp server via 'dpkg -iO ftp://foo/bar/*.deb' + * Install deb-packages from an ftp server via 'dpkg -iO ftp://foo/bar/package.deb' * remount a filesystem readonly as regular user: fsysopts /foo -r * give a process additional group and user permissions at runtime: $ groups @@ -187,10 +197,6 @@ aren't possible. # once the subhurd closes. $ subdo --no-lasting-changes ./virus -- Running parts of the Hurd on different computers, maybe even with shared servers on -dedicated hardware (Cloud Computing when the servers can be made to migrate from -between computers). Maybe this should be placed in "need a lot of coding". - - subhurds for quickly adapting the whole system without bothering others. - Define your personal environment via translators, so you can easily take it with @@ -222,8 +228,6 @@ traditional systems in this reagard, but in fact surpass them. - The possibility to create more efficient and powerful desktop environments. -- Multicore systems (need to fixup Mach for SMP) - - Currently to offer CPU time to some project (like the World Community Grid), it is necessary to install a program from them, and they can then do only what that proram allows them to - which leads to reinventing a processing environment instead of just @@ -240,6 +244,12 @@ level. Programs only have to display the given files and quickly update the state of their own files, so the programs stay very easy. The translator could notify the program when something changes. +- Multicore systems (need to fixup Mach for SMP) + +- Running parts of the Hurd on different computers, maybe even with shared servers on +dedicated hardware (Cloud Computing when the servers can be made to migrate from +between computers). Maybe this should be placed in "need a lot of coding". + ### Unfeasible ideas @@ -320,13 +330,32 @@ Each participant: ("must", because in a community people can do what they perceive as important, and telling someone to stop what he's doing is no option (in my opinion)) +**Result: We talk about the niches we can already fulfill :)** + Things to do ------------ todo-item -> niches for which it is useful. +*This might be useful for the next GSoC.* + ### Easy -- Port debian packages to the Hurd -> mainly tinkerers, but also any other niche. +- Port debian packages to the Hurd -> currently mainly tinkerers, but also any other niche. In the long run this is necessary for every user. Easy start for devs. +- Document easier access to low-level functions via translators, one function at a time. -> tinkerers. +- get nsmux ready for regular users by setting it up in the LiveCDs by default. -> show tinkerers what it can do. +### Complex + +- A filesystem-based package manager: Unionmounting packages. With filterfs from nsmux packages any user should be able to selectively disable any package without affecting the system of others. Simple active translators can add packages. -> clean design and more freedom for tinkerers to setup test environments: “Does this also work with XY disabled?” +- Enable subhurds for regular users via a subdo command: A framework for confining individual applications. -> tinkerers for testing their work. +- Define your personal environment via translators, so you can easily take it with +you ⇒ system on a USB stick. Would work great with a filesystem based package manager. -> ? + +### Huge + +- Get Hurd/GNU Mach ready for efficient multicore usage. -> multicore +- Running parts of the Hurd on different computers, maybe even with shared servers on +dedicated hardware (Cloud Computing when the servers can be made to migrate from +between computers). -> multicore on steroids :) diff --git a/community/weblogs/ArneBab/porting-simple-packages.mdwn b/community/weblogs/ArneBab/porting-simple-packages.mdwn new file mode 100644 index 00000000..becea251 --- /dev/null +++ b/community/weblogs/ArneBab/porting-simple-packages.mdwn @@ -0,0 +1,72 @@ +# Quick porting guide for simple packages + +*If you want to help port a package with simple issues to the Hurd, please read on.* + +*just imagine joe C-doodler stumbling over some GNU philosophy and thinking “hey, I’ve got 2 free hours, why not help the Hurd?”* +*for him I’d like to have a guide (and for me, the faulty-memory-does-too-many-things :) )* + +*a short guide “how to do simple ports” broken down to command line level: how to get the list of simple packages (youpi told me that here), how to get the source, how to test the fix, how to submit the fix.* + +## Setup an instant Hurd development environment + +See [[Instant Development Environment|contributing#index5h2]] - just follow the command to get a Hurd running in Qemu. + +## Getting the list of failed packages + + wget http://people.debian.org/~sthibault/failed_packages.txt.gz + gunzip failed_packages.txt.gz + +## Finding a simple task + + grep PATH_MAX failed_packages.txt -B 2 + +Each of these packages is likely to be simple to fix. The output looks like this: + + … + -- + tex/lilypond_2.12.3-7 by buildd_hurd-i386-mozart [optional:uncompiled:bp{-100}:calprio{-63}:days{258}] + Reasons for failing: + > file-name.cc:88: error: 'PATH_MAX' was not declared in this scope + -- + … + +in this case, lilypond is the package. + +Other simple tasks can be found on [[hurd/porting/guidelines]]. + +## Downloading the package source and installing dependencies + + apt-get source PACKAGE + apt-get build-dep PACKAGE + +For example + + apt-get source lilypond + apt-get build-dep lilypond + +## Fix the package + +See [[hurd/porting/guidelines]] for help on fixing the package. + +Notes: + +* char path[4096] is evil. Use dynamic allocation (if you can). +* use stuff like if (x < sysconf(_SC_PATH_MAX)) {} +* if need be, make it conditional + +\#ifdef PATH_MAX +old, POSIX-violating code +\#else +GNU, better code +\#endif + +## Test the fix (compile, run tests) + + cd PACKAGE + dpkg-buildpackage -B + +Also check the packages README for instructions. + +## Submit the fix + +See [[hurd/running/debian/patch_submission]]. diff --git a/community/weblogs/ArneBab/tasks-for-the-hurd.mdwn b/community/weblogs/ArneBab/tasks-for-the-hurd.mdwn new file mode 100644 index 00000000..bf6224b2 --- /dev/null +++ b/community/weblogs/ArneBab/tasks-for-the-hurd.mdwn @@ -0,0 +1,63 @@ +Tasks for the Hurd +================== + +*These tasks are compiled from the + [[community/weblogs/ArneBab/niches_of_the_hurd]] and + [[community/weblogs/ArneBab/what_we_need]]. The first asked “where + can the Hurd find niches where it is the biggest fish in the pond, + and how?” while the second asked “what do we still need to make the + Hurd usable for most of its developers as system for their day-to-day + tasks?”.* + +*This might be useful for the next GSoC. Please feel free to edit + and/or migrate it mercilessly :)* + +### Easy + +- Port debian packages to the Hurd -> currently mainly tinkerers, but + also any other niche. In the long run this is necessary for every + user. Easy start for devs. +- Document easier access to low-level functions via translators, one + function at a time. -> tinkerers. +- get nsmux ready for regular users by setting it up in the LiveCDs by + default. -> show tinkerers what it can do. +- Test on modern machines. If it doesn’t work, file a bug: + [info](http://www.mail-archive.com/bug-hurd@gnu.org/msg19105.html). + + +### Complex + +- A filesystem-based package manager: Unionmounting packages. With + filterfs from nsmux packages any user should be able to selectively + disable any package without affecting the system of others. Simple + active translators can add packages. -> clean design and more + freedom for tinkerers to quickly setup test environments: “Does this + also work with XY disabled?” ⇒ rapid testing for different base + systems. +- Enable subhurds for regular users via a subdo command: A framework + for confining individual applications. -> tinkerers for testing + their work. +- Define your personal environment via translators, so you can easily + take it with you ⇒ system on a USB stick. Would work great with a + filesystem based package manager -> use the capabilities of a system + and all its installed packages without having to give up your own + custom environment. + +- Implement USB support, maybe using DDE or DDEkit -> prerequisite to system on USB. +- Add Wireless support, maybe via DDE. +- Add sound support via a sound translator. +- Add SATA support. +- Stabilize Xorg, so it can run fast for days. +- Add PPPoE capablilities. +- Debug NFS for climm, w3m and git. +- Port a full-featured browser (i.e. Firefox). +- (Graphical Desktop and switching between console and X) or full + featured high-resultion console which doesn’t need X (and emacs :) + ). + +### Huge + +- Get Hurd/GNU Mach ready for efficient multicore usage. -> multicore +- Running parts of the Hurd on different computers, maybe even with + shared servers on dedicated hardware (Cloud Computing when the servers + can migrate between computers). -> multicore on steroids :) diff --git a/community/weblogs/ArneBab/technical-advantages-of-the-hurd.mdwn b/community/weblogs/ArneBab/technical-advantages-of-the-hurd.mdwn new file mode 100644 index 00000000..35e55518 --- /dev/null +++ b/community/weblogs/ArneBab/technical-advantages-of-the-hurd.mdwn @@ -0,0 +1,56 @@ +Some technical advantages of the Hurd +===================================== + +*→ An answer to [just accept it, truth hurds](http://blog.flameeyes.eu/2011/05/15/just-accept-it-truth-hurds), where Flameeyes told his reasons for not liking the Hurd and asked for technical advantages (and claimed, that the Hurd does not offer a concept which got incorporated into other free software, contributing to other projects). Note: These are the points I see. Very likely there are more technical advantages which I don’t see well enough to explain them. Please feel free to [point them out](http://draketo.de/comment/reply/447#comment-form).* + +*__Information for potential testers:__ The Hurd is already usable, but it is not yet in production state. It progressed a lot during the recent years, though. Have a look at the [[status_report|hurd/status]] if you want to see if it’s already interesting for you.* + +Thanks for explaining your reasons. As answer: + +Firstoff: [FUSE](http://fuse.sourceforge.net/) is essentially an implementation of parts of the [[translator_system|hurd/documentation/translators]] (which is the main building block of the [Hurd](http://hurd.gnu.org)) to Linux, and NetBSD recently got a [port of the translators system of the Hurd](http://netbsd-soc.sourceforge.net/projects/hurdt/). That’s the main contribution to other projects that I see. + +On the bare technical side, the **translator-based filesystem** stands out: The filesystem allows for making arbitrary programs responsible for displaying a given node (which can also be a directory tree) and to start these programs on demand. To make them persistent over reboots, you only need to add them to the filesystem node (for which you need the right to change that node). Also you can start translators on any node without having to change the node itself, but then they are not persistent and only affect your view of the filesystem without affecting other users. These translators are called active, and you don’t need write permissions on a node to add them. +<!--break--> +The filesystem implements stuff like Gnome VFS (gvfs) and KDE **network transparency on the filesystem level**, so those are available for all programs. And you can add a new filesystem as simple user, just as if you’d just write into a file “instead of this node, show the filesystem you get by interpreting file X with filesystem Y” (this is what you actually do when setting a translator but not yet starting it (passive translator)). + +One practical advantage of this is that the following works: + + settrans -a ftp\: /hurd/hostmux /hurd/ftpfs / + dpkg -i ftp://ftp.gnu.org/path/to/*.deb + +This installs all deb-packages in the folder `path/to` on the FTP server. The shell sees normal directories (beginning with the directory “ftp:”), so shell expressions just work. + +You could even define a Gentoo mirror translator (`settrans mirror\: /hurd/gentoo-mirror`), so every program could just access mirror://gentoo/portage-2.2.0_alpha31.tar.bz2 and get the data from a mirror automatically: `wget mirror://gentoo/portage-2.2.0_alpha31.tar.bz2` + +Or you could add a unionmount translator to root which makes writes happen at another place. **Every user is able to make a readonly system readwrite** by just specifying where the writes should go. But the writes **only affect his view of the filesystem**. + +Starting a network process is done by a translator, too: The first time something accesses the network card, the network translator starts up and actually provides the device. This replaces most **initscripts in the Hurd: Just add a translator to a node**, and the service will persist over restarts. + +It’s a surprisingly **simple concept, which reduces the complexity of many basic tasks needed for desktop systems**. + +And at its most basic level, *Hurd is a set of protocols for messages which allow using the filesystem to coordinate and connect processes* (along with helper libraries to make that easy). + +Also it adds **POSIX compatibility to Mach** (while still providing access to the capabilities-based access rights underneath, if you need them). You can **give a process permissions at runtime** and take them away at will. For example you can start all programs without permission to use the network (or write to any file) and add the permissions when you need them. + + groups # → root + addauth -p $(ps -L) -g mail + groups # → root mail + +And then there are subhurds (essentially **lightweight virtualization** which allows cutting off processes from other processes without the overhead of creating a virtual machine for each process). But that’s an entire post of its own… + +And the fact that a translator is just a simple standalone program means that these can be shared and tested much more easily, opening up completely new options for lowlevel hacking, because it massively lowers the barrier of entry. + +And then there is the possibility of *subdividing memory management* and using different microkernels (by porting the Hurd layer, as partly done in the NetBSD port), but that is purely *academic* right now (search for *Viengoos* to see what its about). + + +So in short: *The translator system in the Hurd is a simple concept which makes many tasks easy, which are complex with Linux (like init, network transparency, new filesystems, …). Additionally there are capabilities, subhurds and (academic) memory management.* + +Best wishes, +Arne + +*PS: I decided to read flameeyes’ post as “please give me technical reasons to dispell my emotional impression”.* + +*PPS: If you liked this post, it would be cool if you’d flattr it: <a href="http://flattr.com/thing/273582/Some-technical-advantages-of-the-Hurd" target="_blank"> +<img src="http://api.flattr.com/button/flattr-badge-large.png" alt="Flattr this" title="Flattr this" border="0" /></a>* + +*PPPS: Additional information can be found in [Gaël Le Mignot’s talk notes](http://kilobug.free.fr/hurd/pres-en/abstract/html/), in [niches for the Hurd](http://www.gnu.org/software/hurd/community/weblogs/ArneBab/niches_for_the_hurd.html) and the [[GNU_Hurd_documentation_pages|hurd/documentation]].* diff --git a/community/weblogs/ArneBab/technical-advantages-of-the-hurd/discussion.mdwn b/community/weblogs/ArneBab/technical-advantages-of-the-hurd/discussion.mdwn new file mode 100644 index 00000000..49b64509 --- /dev/null +++ b/community/weblogs/ArneBab/technical-advantages-of-the-hurd/discussion.mdwn @@ -0,0 +1,248 @@ +## Followup discussion in IRC + +IRC, freenode, #hurd, 2011-05-15 + +<dl> +<dt><LibreMan></dt><dd> ArneBab: hi, I read the hurd rant by flameeyes and your response ... I'm following Hurd for some time and would like to ask some questions about it, would you mind? :)</dd> +<dt><ArneBab></dt><dd> please ask :)</dd> +<dt><ArneBab></dt><dd> I don’t mind (as long as I have the time - which I have right now)</dd> +<dt><LibreMan></dt><dd> ok, so essentially I'm trying to figure out, as flameeyes probably is, whether reasons behind developing Hurd are more philosophical/value based or are there real-world technical advantages to it as well</dd> +<dt><ArneBab></dt><dd> antrik: I think his original remark was meant as part-jokingly remark to an aquaintance - which seems fitting, when you keep in mind that flameeyes works very hard and very much on Gentoo, hardly the most popular distro (but the one I like most).</dd> +<dt><ArneBab></dt><dd> LibreMan: the reasons for working on the Hurd are a little bit different for every contributor.</dd> +<dt><ArneBab></dt><dd> (or rather: vastly different :) )</dd> +<dt><LibreMan></dt><dd> as I'm reading about it and your reposne as well, I'm not sure the techical advantages you list would have any real world effect on usability of the OS, do you think they would?</dd> +<dt><ArneBab></dt><dd> I think they would</dd> +<dt><LibreMan></dt><dd> ArneBab: yeah, sure ... my reasons for supporting Hurd are philosophical/value based ... I'll say that outright</dd> +<dt><ArneBab></dt><dd> for example you enter an FTP address in your filebrowser. No problem. Then you want to grep the file contents.</dd> +<dt><ArneBab></dt><dd> you go into the shell and first need to get the files (completely inconvenient)</dd> +<dt><-- npnth (~npnth@pdpc/supporter/active/npnth) hat das Netzwerk verlassen (Disconnected by services) +<dt><ArneBab></dt><dd> or you use gnome and kde programs, and both access the same URL, but cache 2 times.</dd> +<dt><LibreMan></dt><dd> ArneBab: isn't that solved by mounting it?</dd> +<dt><ArneBab></dt><dd> or you want to implement your own desktop and need to do that network transarency stuff yourself.</dd> +<dt><ArneBab></dt><dd> You can’t really mount everything - especially not without root rights.</dd> +<dt><ArneBab></dt><dd> and that’s just one aspect.</dd> +<dt><ArneBab></dt><dd> But that’s only the technical side (he only wanted to hear that)</dd> +<dt><LibreMan></dt><dd> the thing is all these advantages seem too trivial to support a wholly new OS to be developed ... but maybe I'm mistaken, that's why I'm asking, I would love to be good techical reasons for Hurd ... but are not aware of any so far</dd> +<dt><ArneBab></dt><dd> What interests me the most as that I as user can change my environment without affecting others.</dd> +<dt><ArneBab></dt><dd> s/as/is/</dd> +<dt><ArneBab></dt><dd> the main community part is (and I think I missed that), that any server is just a userspace program.</dd> +<dt><ArneBab></dt><dd> it can be exchanged just like any other program</dd> +<dt><antrik></dt><dd> ArneBab: yeah, I found the original remark after following the other links... though it's rather painful to trace the conversations :-)</dd> +<dt><LibreMan></dt><dd> ArneBab: yeah, I understand that ... but what practical advantage would that give me I do not see ... as a server administrator for example</dd> +<dt><ArneBab></dt><dd> I can write an improved filesystem and pass it to you for testing, and you test it only for a backup snapshot of your disk without rebooting.</dd> +<dt><ArneBab></dt><dd> As server admin, you don’t need to install all drivers users could need.</dd> +<dt><ArneBab></dt><dd> the users can just install what they need themselves.</dd> +<dt><antrik></dt><dd> it certainly didn't sound half-joking though... and if it was meant privately, identi.ca is clearly NOT the right place</dd> +<dt><ArneBab></dt><dd> LibreMan: You simply provide a base which reduces the number of things people need to install.</dd> +<dt><LibreMan></dt><dd> ArneBab: but do not I have to give them access to raw HW too then?</dd> +<dt><ArneBab></dt><dd> antrik: I prefer to always assume good faith :)</dd> +<dt><ArneBab></dt><dd> brb</dd> +<dt><ArneBab></dt><dd> child</dd> +<dt><ArneBab></dt><dd> re</dd> +<dt><LibreMan></dt><dd> well, I do not see that people not able to install their own drivers on a server would be any problem currently</dd> +<dt><LibreMan></dt><dd> that's my point ... is seems to solve "problems" that are not really actual real world problems ...</dd> +<dt><ArneBab></dt><dd> LibreMan: no, you just give them a safe device, where a server makes sure they don’t do illegal things.</dd> +<dt><antrik></dt><dd> LibreMan: most of the advantages are not directly visible, unless you do very specific things, where traditional systems impose limits</dd> +<dt><ArneBab></dt><dd> For me network transparency is a realworld problem</dd> +<dt><ArneBab></dt><dd> as is that I can’t give a program network access later</dd> +<dt><antrik></dt><dd> but it makes many things easier, which in the end will translate into advantages for everyone I believe</dd> +<dt><ArneBab></dt><dd> “log out and in again to play games”</dd> +<dt><ArneBab></dt><dd> (after adding yourself to the games group)</dd> +<dt><ArneBab></dt><dd> or better still: Always start with minimal rights and only add what is really needed.</dd> +<dt><ArneBab></dt><dd> There’s no reason why a program should have access to my audio hardware without me granting it.</dd> +<dt><ArneBab></dt><dd> That way I could even run malicious software without having to fear compromising my system.</dd> +<dt><antrik></dt><dd> LibreMan: I could come up with situations where it could help you as an administrator; but this is not really helpful. you won't really understand the advantages until you get into a specific situation that is hard to do on Linux for example, and much easier on the Hurd</dd> +<dt><LibreMan></dt><dd> ArneBab: well, then it becomes a tradeoff between security and user friendliness ... I do not think that problem is unsolvable currently, I think it is a design decision not to "solve it" as wast majority of users do not actually need or want it</dd> +<dt><ArneBab></dt><dd> LibreMan: Time and again I find myself sitting in front of my linux box and thinking “damn, this woul be so easy in the Hurd”</dd> +<dt><ArneBab></dt><dd> (I do most of my work on a Linux box)</dd> +<dt><ArneBab></dt><dd> Gentoo GNU/Linux with KDE and Emacs</dd> +<dt><LibreMan></dt><dd> antrik: yeah, could you give me an example of something that is hard on linux but easy under hurd? with real world implications for real use cases :)</dd> +<dt><ArneBab></dt><dd> LibreMan: Get a hg/git log of a repo on an ftp server</dd> +<dt><antrik></dt><dd> LibreMan: no, it's *not* a tradeoff. the whole point is that the Hurd architecture allows users to customize their environment *without* compromising the security of the system</dd> +<dt><ArneBab></dt><dd> antrik LibreMan: I think there we have one point: When you use Linux you are used to thinking of the Linux limits as the absolute limits.</dd> +<dt><LibreMan></dt><dd> antrik: the point is, wast majority of users do not need that ... AFAIK</dd> +<dt>-*- youpi is fed up with using sudo just to mount an iso image +<youpi></dt><dd> really</dd> +<dt><Tekk_></dt><dd> youpi: dbus</dd> +<dt><ArneBab></dt><dd> LibreMan: just wait for the first strong linux worm which spreads in a game and requires sudo for install… </dd> +<dt><youpi></dt><dd> (and it's just one of the strongest examples)</dd> +<dt><youpi></dt><dd> Tekk_: ??</dd> +<dt><antrik></dt><dd> LibreMan: ArneBab already gave you various exmples. including at least one that works out of the box</dd> +<dt><Tekk_></dt><dd> youpi: with dbus you don't need root permissions</dd> +<dt><youpi></dt><dd> Tekk_: and you can mount any iso?</dd> +<dt><antrik></dt><dd> (others would require some additional coding)</dd> +<dt><ArneBab></dt><dd> Tekk_: but something needs them.</dd> +<dt><Tekk_></dt><dd> youpi: oh, iso...</dd> +<dt><LibreMan></dt><dd> ArneBab: why would a game need sudo to install? :)</dd> +<dt><youpi></dt><dd> Tekk_: yes, iso</dd> +<dt><youpi></dt><dd> or $WHATEVER_FS</dd> +<dt><Tekk_></dt><dd> youpi: sec</dd> +<dt><ArneBab></dt><dd> LibreMan: yes, I would ask that, too. But the general Ubuntu user?</dd> +<dt><youpi></dt><dd> or sshfs, or ftpfs, etc.</dd> +<dt>-*- ArneBab had hoped you’d catch that :) +<LibreMan></dt><dd> the area where I can imagine hurd being better is virtualization</dd> +<dt><antrik></dt><dd> LibreMan: again, the "general Ubuntu user" won't directly see the benefits. but he will see them when developers use them to implement nice features that would be much harder to implement elsewhere</dd> +<dt><LibreMan></dt><dd> making everything cloudy is a big trend nowadays and hurd could provide additional flexibility there ... or no? I'm really just guessing based on what I read</dd> +<dt><antrik></dt><dd> it's a bad trend</dd> +<dt><ArneBab></dt><dd> LibreMan: it could give me more options when I have to work on another ones computer. After all it was conceived in the time of dumb terminals - which now comes back.</dd> +<dt><LibreMan></dt><dd> ArneBab: oh well, if we are talking about user stupidity then no OS is going to help ;)</dd> +<dt><antrik></dt><dd> I'm not sure whether the Hurd help with "making things cloudy", but it's not something I'd consider an advantage anyways :-)</dd> +<dt><ArneBab></dt><dd> LibreMan: The OS can reduce the impact of user stupidity (called DAU in german: „Dümmster anzunehmender User“ → “dumbest conceivable user”)</dd> +<dt><antrik></dt><dd> as for virtualization, indeed there is a *very* close relation between that and microkernel systems, which most people fail to see...</dd> +<dt><LibreMan></dt><dd> antrik: the "cloud" is coming if we like it or not ... it better run on FOSS if it comes ;)</dd> +<dt><Tekk_></dt><dd> you know, you gues have a huge advantage</dd> +<dt><Tekk_></dt><dd> guys*</dd> +<dt><Tekk_></dt><dd> people have waited forever and written you off</dd> +<dt><ArneBab></dt><dd> LibreMan: just think about the difference between a GNU/Linux distro and Windows XP where you were admin at all times.</dd> +<dt><Tekk_></dt><dd> and as duke nukem forever shows, that's a good thing ;P</dd> +<dt><antrik></dt><dd> in fact, the only fundamental difference is that a VM makes the subenvironment look more or less like a real machine, while in traditional microkernel systems different interfaces are used</dd> +<dt><ArneBab></dt><dd> LibreMan: A Hurd system would go one step further.</dd> +<dt><ArneBab></dt><dd> You’d even need a password more seldomly, reducing the incentive to just work as admin.</dd> +<dt><ArneBab></dt><dd> Reason: There are less things which can really badly impact the system.</dd> +<dt><antrik></dt><dd> LibreMan: when talking about "the cloud", people usually mean things that are fundamentally incompatible with the idea of free software</dd> +<dt><antrik></dt><dd> (you don't have control over the software running web services)</dd> +<dt><ArneBab></dt><dd> antrik: but the cloud just means “I’m on a different computer”. AGPLv3 is cool there :)</dd> +<dt><LibreMan></dt><dd> antrik: yeah, that's the common bussines practice but it doesn't have to be ... AGPL ;)</dd> +<dt><antrik></dt><dd> ArneBab: well, actually "the cloud" means something different to everyone ;-)</dd> +<dt><LibreMan></dt><dd> I do not have any problem with a cloud running AGPL software</dd> +<dt><ArneBab></dt><dd> antrik: well, yes :)</dd> +<dt><antrik></dt><dd> ArneBab: but generally it relies on using programs on foreign machines, controlled by someone else. AGPL doesn't change that</dd> +<dt><LibreMan></dt><dd> freedombox is going to be a "cloud" too</dd> +<dt><antrik></dt><dd> LibreMan: that's not what most people mean by "cloud"</dd> +<dt><LibreMan></dt><dd> at least I hope so</dd> +<dt><ArneBab></dt><dd> I don’t have control over my webserver. I can’t run a real Python there. Hurd could change that (though that will take a lot of coding: the conceptual options are there)</dd> +<dt><ArneBab></dt><dd> LibreMan: what could interest you: http://www.gnu.org/software/hurd/community/weblogs/ArneBab/niches_for_the_hurd.html</dd> +<dt><ArneBab></dt><dd> “Niches of the Hurd”</dd> +<dt><LibreMan></dt><dd> what I mean by "the cloud" is what Eben Moglen explaied it as ... the technology which make it possible to forget about the "iron" and move servers around seamlessly</dd> +<dt><LibreMan></dt><dd> ArneBab: thank.. going to look at it</dd> +<dt><antrik></dt><dd> LibreMan: that's actually more or less what used to be called "grid computing" before the cloud hype</dd> +<dt><LibreMan></dt><dd> antrik: well ... yes, essentially</dd> +<dt><antrik></dt><dd> LibreMan: but most people mean many other things too when talking about "clouds"</dd> +<dt><antrik></dt><dd> and anyways, you can't really forget about the iron. there is a middle layer which you don't have control of</dd> +<dt><LibreMan></dt><dd> antrik: sure, I would say that most people do not know what they mean :)</dd> +<dt><Tekk_></dt><dd> hmm</dd> +<dt><Tekk_></dt><dd> ArneBab: I can see a big place for virtualization in browsers</dd> +<dt><Tekk_></dt><dd> I mean, with everyone so worried about the code getting executed there, we just have GNUBrowse run in it's own little environment all closed off</dd> +<dt><ArneBab></dt><dd> Tekk_: me too: safe subenvironments.</dd> +<dt><LibreMan></dt><dd> the reason I follow Hurd is because i would LOVE to have viable GPLv3 OS as opposed to GPLv2 Linux</dd> +<dt><antrik></dt><dd> that's not a good reason</dd> +<dt><LibreMan></dt><dd> there are people hard at work subverting Linux</dd> +<dt><antrik></dt><dd> first of all, we'd have to get rid of all Linux code</dd> +<dt><LibreMan></dt><dd> locking it down ... and Linus doesn't seem to care</dd> +<dt><antrik></dt><dd> also, if that's all you care about, it would be less work to implement a simple monolithic kernel from scratch</dd> +<dt><LibreMan></dt><dd> antrik: so why is hurd a good idea if it's so much harder to develop?</dd> +<dt><-- azeem (~mbanck@p5DF41DDE.dip0.t-ipconnect.de) hat das Netzwerk verlassen (Ping timeout: 240 seconds) +<LibreMan></dt><dd> antrik: I thought that was the reason all along ... to develop GNU mopatible kernel</dd> +<dt><Tekk_></dt><dd> LibreMan: what do you mean GNU compatible?</dd> +<dt><LibreMan></dt><dd> Tekk_: the philosophy of GNU</dd> +<dt><Tekk_></dt><dd> ah, yes they've always needed a gnu kernel</dd> +<dt><LibreMan></dt><dd> the reason why it was created in a first place</dd> +<dt><ArneBab></dt><dd> LibreMan: I just added a missing part in the article: </dd> +<dt><ArneBab></dt><dd> “And the fact that a translator is just a simple standalone program means that these can be shared and tested much more easily, opening up completely new options for lowlevel hacking, because it massively lowers the barrier of entry.</dd> +<dt><ArneBab></dt><dd> ”</dd> +<dt><LibreMan></dt><dd> ok, next question :) why is it so hard to make Hurd work?</dd> +<dt><ArneBab></dt><dd> because we are so few people… </dd> +<dt><LibreMan></dt><dd> I mean, it's in developement for 20 years or so, no?</dd> +<dt><Tekk_></dt><dd> LibreMan: it's never been done before too</dd> +<dt><LibreMan></dt><dd> ArneBab: well, one person developed Linux</dd> +<dt><ArneBab></dt><dd> One person make Linux basically work</dd> +<dt><Tekk_></dt><dd> LibreMan: there has *never* been a full microkernel outside of research, which is what hurd plans to be</dd> +<dt><Tekk_></dt><dd> LibreMan: yeah, one person made linux kinda work in a year, then basically handed it off to everyone to help</dd> +<dt><LibreMan></dt><dd> but if it's so much complicated to develop, is it worth it?</dd> +<dt><Tekk_></dt><dd> LibreMan: and that was with a well trodden path that everyone knwos</dd> +<dt><ArneBab></dt><dd> But for the Hurd to basically work means it already provides far more options than waat Linux did.</dd> +<dt><ArneBab></dt><dd> That’s why the foundation is harder: It makes everything else easier.</dd> +<dt><ArneBab></dt><dd> But at the moment it works.</dd> +<dt><ArneBab></dt><dd> And I think I’ll just repeat that: The Hurd works. It is not feature complete, but all the really hard parts work.</dd> +<dt><ArneBab></dt><dd> Missing are many of the hard (but not really hard) parts, like adding drivers.</dd> +<dt><ArneBab></dt><dd> (and then there are ultra hard features which are possible but currently layed off, but now I get into beat-em-up speech :) )</dd> +<dt><LibreMan></dt><dd> I would define "works" as I can install it right now and run stable system ... I do not think it worls in those terms</dd> +<dt><ArneBab></dt><dd> LibreMan: that I’d define as production system</dd> +<dt><youpi></dt><dd> LibreMan: define "stable"</dd> +<dt><Tekk_></dt><dd> LibreMan: I'm pretty sure linux didn't "work" by your definition when linus passed it off</dd> +<dt><ArneBab></dt><dd> But I can start a Hurd right now and code in it.</dd> +<dt><LibreMan></dt><dd> ArneBab: yeah, that's waht "works" means for me :)</dd> +<dt><ArneBab></dt><dd> I can start emacs</dd> +<dt><youpi></dt><dd> LibreMan: I wouldn't even call my linux "stable"</dd> +<dt><youpi></dt><dd> as I just need to unplug my external USB hdd to make it crash...</dd> +<dt><youpi></dt><dd> while in a hurd system, it'd just crash the corresponding ext2fs daemon only</dd> +<dt><LibreMan></dt><dd> youpi: well yes :) but you can function on it pretty successfully ...</dd> +<dt><ArneBab></dt><dd> LibreMan: Frankly all I’m missing for a production environment are USB support and Audio.</dd> +<dt><youpi></dt><dd> you can on a hurd system too</dd> +<dt><ArneBab></dt><dd> (and it should work on an OLPC)</dd> +<dt><LibreMan></dt><dd> youpi: without USB and sound? :P</dd> +<dt><ArneBab></dt><dd> Both are driver issues. No problem in the kernel.</dd> +<dt><youpi></dt><dd> I seldomly use USB and sound actually</dd> +<dt><youpi></dt><dd> and never for my actual work</dd> +<dt><-- Tekk_ (~user@2002:474d:d1e9:0:21d:72ff:fe24:4c37) hat das Netzwerk verlassen (Remote host closed the connection) +--></dt><dd> Tekk_ (~user@2002:474d:d1e9:0:21f:3aff:fe54:7cc3) hat #hurd betreten</dd> +<dt><LibreMan></dt><dd> youpi: yeah, but how many users can ay that?</dd> +<dt><youpi></dt><dd> so what?</dd> +<dt><youpi></dt><dd> how many users can install linux?</dd> +<dt><youpi></dt><dd> does that make it unsuccessful?</dd> +<dt><LibreMan></dt><dd> many people actually</dd> +<dt><youpi></dt><dd> well, many people don't care about USB and sound either</dd> +<dt><youpi></dt><dd> depends what you mean by "many"</dd> +<dt><LibreMan></dt><dd> compared to how many do not mind having USB and sound ...</dd> +<dt><youpi></dt><dd> just like depends what you mean by "stable"</dd> +<dt><youpi></dt><dd> so _basically_ it works</dd> +<dt><youpi></dt><dd> not for all users on earth of course</dd> +<dt><youpi></dt><dd> not for all linux users of course</dd> +<dt><youpi></dt><dd> but for a lot of them already</dd> +<dt><ArneBab></dt><dd> LibreMan: USB and Sound are just driver issues.</dd> +<dt><ArneBab></dt><dd> They are not part of the core functionality.</dd> +<dt><Tekk_></dt><dd> LibreMan: usb keyboards and mice work though</dd> +<dt><LibreMan></dt><dd> ArneBab: sure, but that doesn't matter ... user doen't care about the technicalities ...</dd> +<dt><Tekk_></dt><dd> I think</dd> +<dt><ArneBab></dt><dd> LibreMan: But for the Hurd it means that it’s no general unsolved problem, but just an issue of too little coders to do the work.</dd> +<dt><LibreMan></dt><dd> if it's driver, kernel, microkernel whatever ... does it work or not, that's what it comes down to</dd> +<dt><ArneBab></dt><dd> s/too little/too few/</dd> +<dt><LibreMan></dt><dd> so it's a matter of attracting more people to work on it</dd> +<dt><ArneBab></dt><dd> LibreMan: yes</dd> +<dt><ArneBab></dt><dd> The Arch people helped a lot with that, because 2 distributions is not just 2× one distribution (in it’s outside effect)</dd> +<dt><ArneBab></dt><dd> But we need more people who do the easy work.</dd> +<dt><ArneBab></dt><dd> relatively easy… </dd> +<dt><ArneBab></dt><dd> porting the 10-15% packages which just have PATH_MAX issues.</dd> +<dt><LibreMan></dt><dd> so there would need to be sufficient motivations for them to join developement ... so far I do not see any different than Free Software ideals</dd> +<dt><ArneBab></dt><dd> for example</dd> +<dt><ArneBab></dt><dd> jupp</dd> +<dt><ArneBab></dt><dd> plus some cool options.</dd> +<dt><ArneBab></dt><dd> and experimenting in low-level</dd> +<dt><ArneBab></dt><dd> “ever wanted to write your own filesystem from scratch - and test it without wrecking your box?”</dd> +<dt><LibreMan></dt><dd> I do not understand why FSF does not do something similar to GSoC</dd> +<dt><ArneBab></dt><dd> LibreMan: I assume “too little money”…</dd> +<dt><Tekk_></dt><dd> LibreMan: hurd is in the gsoc</dd> +<dt><LibreMan></dt><dd> yeah, that would be the obvious answer :) and the right onw I guess</dd> +<dt><ArneBab></dt><dd> Besides: antrik, do you know how jkkenig fares?</dd> +<dt><Tekk_></dt><dd> LibreMan: gsoc is a per project thing, and most of them don't need the help</dd> +<dt><ArneBab></dt><dd> jkoenig</dd> +<dt><LibreMan></dt><dd> Tekk_: I know ... I just do not like that a company like Google needs to sponsor it and "we" are not selfsufficient</dd> +<dt><ArneBab></dt><dd> LibreMan: well, too few people are used to pay for what they like instead of for what requires payment.</dd> +<dt><ArneBab></dt><dd> But that is changing.</dd> +<dt><LibreMan></dt><dd> ArneBab: exactly ...</dd> +<dt><LibreMan></dt><dd> things like Flattr are trying to change that mentalty</dd> +<dt><ArneBab></dt><dd> Besides (stable): Hurd runs the Hurd wiki.</dd> +<dt><ArneBab></dt><dd> http://www.bddebian.com/~hurd-web/</dd> +<dt><LibreMan></dt><dd> I'm quite surprised I did not know about this http://www.fossfactory.org</dd> +<dt><LibreMan></dt><dd> I was planning to make such a website myself ...</dd> +<dt><LibreMan></dt><dd> I do not understand why it doesn't get more publicity ... the way Kickstarted does</dd> +<dt>-*- ArneBab goes lurker, sons here +<jkoenig></dt><dd> ArneBab, I have exams till friday, I should be more present after that</dd> +<dt><Tekk_></dt><dd> what does the hello world translator do? XD</dd> +<dt><ArneBab></dt><dd> Tekk_: content = hello</dd> +<dt><antrik></dt><dd> ArneBab: I have no idea about the status of GSoC</dd> +<dt><antrik></dt><dd> I haven't even read my mails for a couple of weeks; so you probably know more than me</dd> +<dt><antrik></dt><dd> Tekk_: provide a pseudo-file with "hello world" as contents</dd> +<dt><Tekk_></dt><dd> ah</dd> +<dt><antrik></dt><dd> BTW, gvfs is actually my favourite example of why the Hurd architecture makes sense</dd> +<dt><antrik></dt><dd> they implemented an extra GNOME-specific VFS layer, that is mostly redundant with the kernel one, adding complexity, overhead, and not integrated with the rest of the system -- and the only reason they need it is because the kernel VFS of traditional systems is too limited</dd> +<dt><antrik></dt><dd> with the Hurd's decentralized VFS, they could have implemented everything they need trivially right in the system VFS layer</dd> +<dt><antrik></dt><dd> the question is not really what features are possible with the Hurd architecture: given enough effort, any feature can be implemented with any architecture. it's the amount of effort that differs, making some things *feasible* that are not on other systems</dd> +<dt><Tekk_></dt><dd> see: windows ME</dd> +<dt><antrik></dt><dd> there is no reason for example why things like isolated subenvironments couldn't be implemented on Linux. (and it fact it's clearly moving in that direction, with the virtualisation hype) -- but it requires a shitload of kernel changes. while on Hurd all it needs is a little userspace programming</dd> +<dt><antrik></dt><dd> and every new feature added to Linux container solutions require further kernel hacking</dd> +<dt><antrik></dt><dd> or every new feature added to FUSE</dd> +<dt><antrik></dt><dd> and so on</dd> +</dl> + +[[!tag open_issue_documentation]] diff --git a/community/weblogs/ArneBab/what_we_need.mdwn b/community/weblogs/ArneBab/what_we_need.mdwn new file mode 100644 index 00000000..4511eb64 --- /dev/null +++ b/community/weblogs/ArneBab/what_we_need.mdwn @@ -0,0 +1,39 @@ +We created a list of the things we still need for using the Hurd for in our day-to-day activities (work or hobby). + +As soon as these issues are taken care of, the Hurd offers everything we need for fullfilling most of our computing needs on at least one of our devices: + +- USB (5): Arne, ms, Michael, Emilio, antrik²³ +- Wireless (5): Arne, ms, Carl Fredrik, Michael (netbook), antrik (notebook) +- Sound (4): ms, Carl Fredrik, Michael, antrik² + +- SATA (2): Michael, (Emilio) +- Tested for modern machines°¹ (2): Emilio, antrik (notebook) +- Stable Xorg° (2): Emilio, antrik +- PPPoE (2): Carl Fredrik, antrik² + +- Graphical Desktop (1): Emilio +- Full featured high-resultion console which doesn’t need X (1): antrik +- Switching between console and X° (1): antrik +- full-featured browser (i.e. Firefox)°⁵ (1): antrik +- NFS working for climm, w3m and git (1): antrik⁴ +- mplayer with win32codecs (1): antrik³ +- gnash or alternatives (1): antrik³ + +°: Very likely needed by more people, but not named as most pressing issue. +¹: It’s unclear on which processors the Hurd would have problems. Please report it if you have one! +→ [info](http://www.mail-archive.com/bug-hurd@gnu.org/msg19105.html) +²: Would be OK to use a router box instead. +³: Not critical but would be convenient. +⁴: Only while *not* using Hurd as the only machine. +⁵: [We’re close to that](http://www.mail-archive.com/bug-hurd@gnu.org/msg19177.html). + +So, if one of these issues seems to be interesting for you, or you think “I can do that easily”, +why not become a Hurd hacker and add your touch? :) + +You can reach us in the [[mailing_lists]] and in [[irc]]. + +The sourcecode is in our [[source_repositories]] (git). When you want to check sources relevant for you, [DDE](http://git.savannah.gnu.org/cgit/hurd/incubator.git/tree/?h=dde) might be a good place to start for wireless and sound. USB on the other hand might need work in [gnumach](http://git.savannah.gnu.org/cgit/hurd/gnumach.git/) ([[hacking_info|microkernel/mach/gnumach]]). + +Besides: “The great next stuff” is in the [incubator git repo](http://git.savannah.gnu.org/cgit/hurd/incubator.git/), including (among others) [clisp](http://git.savannah.gnu.org/cgit/hurd/incubator.git/tree/?h=clisp) (translators in lisp) and [nsmux](http://git.savannah.gnu.org/cgit/hurd/incubator.git/tree/?h=nsmux) (dynamically setting translators on files for one command by accessing `file,,translator`). + +Happy hacking! diff --git a/community/weblogs/antrik.mdwn b/community/weblogs/antrik.mdwn new file mode 100644 index 00000000..6db88dd9 --- /dev/null +++ b/community/weblogs/antrik.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!inline +pages="community/weblogs/antrik/* and !community/weblogs/antrik/*/*" +show=0 +actions=no +rootpage="community/weblogs/antrik" postformtext="Add a new entry named:"]] diff --git a/community/weblogs/antrik/plan9-and-the-hurd-major-differences.mdwn b/community/weblogs/antrik/plan9-and-the-hurd-major-differences.mdwn new file mode 100644 index 00000000..9e6143bf --- /dev/null +++ b/community/weblogs/antrik/plan9-and-the-hurd-major-differences.mdwn @@ -0,0 +1,50 @@ +[[!meta copyright="Copyright © 2009 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="Major differences between Plan9 and the Hurd"]] + +There are some similarities between the Hurd and Plan 9 regarding the file +system handling -- but there are also very fundamental differences which go +far beyond monolithic vs. microkernel design: + +- The Hurd is UNIX (POSIX) compatible + +- While (almost) all services are attached to the file system tree, not + all services actually export a file system interface! + + Personally, I advocate using FS-based interfaces as much as possible. + Yet, there are some cases where they get very awkward and/or + inefficient, and domain-specific interfaces simply make a lot more + sense. + + Also, some Hurd services are indeed used to implement the file systems + in the first place -- they work below the FS level, and obviously + can't use an FS interface! + +- File systems are completely decentralized -- clients always talk to + the FS servers directly, without any central VFS layer. (I don't think + that's the case in Plan 9?) + + This offers much more flexibility -- the way the FS interfaces + themselves work can be modified. Many things can be implemented as + normal translators, that would require special VFS support on other + systems. (Extended attributes, VFS-based union mounts, local + namespaces, firmlink, magic file name suffixes etc.) + +- The system design allows users and applications to change almost all + aspects of the system functionality in the local environment easily + and without affecting other parts of the system. + + (This is possible with Plan 9 to some extent; but the Hurd allows it + at a much lower level -- including stuff like the filesystem + interfaces, access control mechanisms, program execution and process + management, and so on.) + +I hope I didn't forget any major differences... diff --git a/community/weblogs/hook.mdwn b/community/weblogs/hook.mdwn new file mode 100644 index 00000000..e9e083dc --- /dev/null +++ b/community/weblogs/hook.mdwn @@ -0,0 +1,25 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +Well as [[weblogs/ArneBab]] asked me to, I made a blog here in the Hurd's community section. + +So I suppose it's time for me to introduce myself. I'm a lawyer student just short of my masters' (called "diploma" here in Slovenia) and a hacker by heart. I've been using GNU/Linux for over a decade now, started on Slackware and continued on Gentoo. I try to give back to the community by being an active member (posting bugs and whatnot), coordinating the local FSFE Fellowship group and lately also lending a hand to the Gentoo Licenses team. I keep a [website and blog](http://matija.suklje.name) of my own and occasionally even write some short sad piece of sloppy code. + +Small disclaimer about my coding abilities: + + 10 IANAC IAAL + +For those who wonder about what IANAC IAAL means — it's the oposite of IANAL IAAC and means "I Am Not A Coder, I Am A Lawyer" ;) + +[[!inline +pages="community/weblogs/hook/* and !community/weblogs/hook/*/*" +show=0 +actions=no +rootpage="community/weblogs/hook" postformtext="Add a new entry named:"]] diff --git a/community/weblogs/hook/Post.mdwn b/community/weblogs/hook/Post.mdwn new file mode 100644 index 00000000..904ff372 --- /dev/null +++ b/community/weblogs/hook/Post.mdwn @@ -0,0 +1,27 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +You might wonder why this post is not titled "First Post" or anything similar showing off my arrival in the Hurd community. + +Well, that's both easy and hard to explain — easy, because it doesn't take much words and hard because of their impact. + +The thing is it may well be my first and last post here. + +I am not making this decision lightly, because I care a lot for FOSS and although I'm new and not much of a coder (`GOTO 10`), I can see how important GNU Hurd is and needs more advocates and contributors. + +Sadly, as I stated [on my normal blog](http://matija.suklje.name/?q=node/205), to be more help to the FOSS community, I actually have to help less. I have to make the painful choice to select from many FOSS-related things I care about deeply only a few I'm really good at and discard the rest. And since law is my forte, that's where I'll help and leave coding to those who are better at it. + +That too is freedom and probably the biggest burden of it. + +I'm pretty sure most people here haven't had the time to get to know me yet, but I'll still miss you. And thank you guys for your outstanding work in making the system that gives the user the most freedom possible! **Please, keep up the work!** + + *hook out → just out (hopefully not forever)* + +P.S. `10 IANAC IAAL` diff --git a/community/weblogs/tschwinge.mdwn b/community/weblogs/tschwinge.mdwn new file mode 100644 index 00000000..fc0d2ace --- /dev/null +++ b/community/weblogs/tschwinge.mdwn @@ -0,0 +1,15 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!inline +pages="community/weblogs/tschwinge/* and !community/weblogs/tschwinge/*/*" +show=0 +actions=no +rootpage="community/weblogs/tschwinge" postformtext="Add a new entry named:"]] diff --git a/community/weblogs/tschwinge/2009-06-23_split_patch_and_git_rebase_--interactive.mdwn b/community/weblogs/tschwinge/2009-06-23_split_patch_and_git_rebase_--interactive.mdwn new file mode 100644 index 00000000..bd08060e --- /dev/null +++ b/community/weblogs/tschwinge/2009-06-23_split_patch_and_git_rebase_--interactive.mdwn @@ -0,0 +1,545 @@ +[[!meta copyright="Copyright © 2009 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="splitting a patch into three, and then some git rebase +--interactive"]] + +I was revisiting the issue of getting the Hurd's code base compiled with recent +versions of GCC. Specifically, there were a lot of duplicate symbols shown at +linking time, and all these were related to `inline` functions. Originally, in +2007, we had solved this problem already (or rather, shifted it) by using GCC's +`-fgnu89-inline` option, but as we [[!GNU_Savannah_patch 6840 desc="saw now"]], +that one obviously doesn't help anymore if third-party code is using the Hurd's +unfixed header files. + +So I was revisiting this issue. I was already prepared that this would take +some hours, with lots of editing, compiling cycles, plus some analyzing of the +binaries. So I made up a fresh repository for this work. + + $ mkdir hurd-ei + $ cd hurd-ei/ + $ git init + [...] + $ git remote add savannah git://git.savannah.gnu.org/hurd/hurd.git + $ git fetch + [...] + +Switch to a new topic-branch. + + $ git checkout -b master-ei savannah/master + Branch master-ei set up to track remote branch master from savannah. + Switched to a new branch 'master-ei' + +(*`ei`* is short for `extern inline`.) + +The first thing to do was to disable that `-fgnu89-inline` option, so I edited +`Makeconf` where it was added to `CFLAGS`. + +I started editing, compiling, editing, compiling, and so on. + +Finally, the tree was in a shape where everything was building fine and the +resulting libraries contained the symbols they should, etc. + +I committed the whole junk as one *big blob* commit, to store it in a safe +place (you never know with these Hurd machines...), and to continue working on +it the next day. + + $ git commit -a + +For the commit message, I already mostly assembled a `ChangeLog`-style log. +Then: + + $ git format-patch savannah/master.. + 0001-Bla.patch + +... and here is [[0001-Bla.patch.bz2]] (compressed). + + +The next day, a.k.a. today, in a different Git repository. + + $ git checkout -b master-fix_inline savannah/master + Branch master-fix_inline set up to track remote branch master from savannah. + Switched to a new branch 'master-fix_inline' + $ bunzip2 < ../some/where/0001-Bla.patch.bz2 | git am + Applying: Bla. + +The *big blob* is now on top of savannah/master (which was +`2772f5c6a6a51cf946fd95bf6ffe254273157a21`, by the way -- in case that you want +to reproduce this tutorial later, simply substitute `savannah/master` with +`2772...`). + +By then, I had come to the conclusion that the commit essentially was fine, but +should be split into two, and the `configure` hunk shouldn't be in there. So +be it. + +So, the `HEAD` of the active branch is our *big blob* commit that we want to +work on. Check with `git show HEAD`: + + $ git show HEAD + commit 93e97f3351337c349e2926f4041e61bc487ef9e6 + Author: Thomas Schwinge <tschwinge@gnu.org> + Date: Tue Jun 23 00:27:28 2009 +0200 + + Bla. + + * console-client/timer.h (fetch_jiffies): Use static inline instead of extern + inline. + * ext2fs/ext2fs.h (test_bit, set_bit, clear_bit, dino, global_block_modified) + (record_global_poke, sync_global_ptr, record_indir_poke, sync_global) + (alloc_sync): Likewise. + * libftpconn/priv.h (unexpected_reply): Likewise. + * term/term.h (qsize, qavail, clear_queue, dequeue_quote, dequeue) + (enqueue_internal, enqueue, enqueue_quote, unquote_char, char_quoted_p) + (queue_erase): Likewise. + * ufs/ufs.h (dino, indir_block, cg_locate, sync_disk_blocks, sync_dinode) + (swab_short, swab_long, swab_long_long): Likewise. + * term/munge.c (poutput): Use static inline instead of inline. + + * libdiskfs/diskfs.h: Apply inline optimization only ifdef + [__USE_EXTERN_INLINES]. Use __extern_inline instead of extern inline. + * libftpconn/ftpconn.h: Likewise. + * libpipe/pipe.h: Likewise. + * libpipe/pq.h: Likewise. + * libshouldbeinlibc/idvec.h: Likewise. + * libshouldbeinlibc/maptime.h: Likewise. + * libshouldbeinlibc/ugids.h: Likewise. + * libstore/store.h: Likewise. + * libthreads/rwlock.h: Likewise. + * libdiskfs/extern-inline.c: Adapt to these changes. + * libftpconn/xinl.c: Likewise. And don't #include "priv.h". + * libpipe/pipe-funcs.c: Likewise. + * libpipe/pq-funcs.c: Likewise. + * libshouldbeinlibc/maptime-funcs.c: Likewise. And remove superfluous + includes. + * libstore/xinl.c: Likewise. + * libthreads/rwlock.c: Likewise. + + * Makeconf (CFLAGS): Don't append $(gnu89-inline-CFLAGS). + * pfinet/Makefile (CFLAGS): Append $(gnu89-inline-CFLAGS). + + diff --git a/Makeconf b/Makeconf + index e9b2045..236f1ec 100644 + --- a/Makeconf + +++ b/Makeconf + @@ -65,7 +65,7 @@ INCLUDES += -I$(..)include -I$(top_srcdir)/include + CPPFLAGS += $(INCLUDES) \ + -D_GNU_SOURCE -D_IO_MTSAFE_IO -D_FILE_OFFSET_BITS=64 \ + $($*-CPPFLAGS) + -CFLAGS += -std=gnu99 $(gnu89-inline-CFLAGS) -Wall -g -O3 \ + +CFLAGS += -std=gnu99 -Wall -g -O3 \ + [...] + +We want to undo this one commit, but preserve its changes in the working +directory. + + $ git reset HEAD^ + Makeconf: locally modified + configure: locally modified + console-client/timer.h: locally modified + ext2fs/ext2fs.h: locally modified + libdiskfs/diskfs.h: locally modified + libdiskfs/extern-inline.c: locally modified + libftpconn/ftpconn.h: locally modified + libftpconn/priv.h: locally modified + libftpconn/xinl.c: locally modified + libpipe/pipe-funcs.c: locally modified + libpipe/pipe.h: locally modified + libpipe/pq-funcs.c: locally modified + libpipe/pq.h: locally modified + libshouldbeinlibc/idvec.h: locally modified + libshouldbeinlibc/maptime-funcs.c: locally modified + libshouldbeinlibc/maptime.h: locally modified + libshouldbeinlibc/ugids.h: locally modified + libstore/store.h: locally modified + libstore/xinl.c: locally modified + libthreads/rwlock.c: locally modified + libthreads/rwlock.h: locally modified + pfinet/Makefile: locally modified + term/munge.c: locally modified + term/term.h: locally modified + ufs/ufs.h: locally modified + +Now, `HEAD` points to the commit before the previous `HEAD`, i.e. `HEAD^`. +Again, check with `git show HEAD`: + + $ git show HEAD + commit 2772f5c6a6a51cf946fd95bf6ffe254273157a21 + Author: Samuel Thibault <samuel.thibault@ens-lyon.org> + Date: Thu Apr 2 23:06:37 2009 +0000 + + 2009-04-03 Samuel Thibault <samuel.thibault@ens-lyon.org> + + * exec.c (prepare): Call PREPARE_STREAM earlier to permit calling + finish_mapping on E even after errors, as is already done in do_exec. + + diff --git a/exec/ChangeLog b/exec/ChangeLog + index 5a0ad1d..a9300bf 100644 + --- a/exec/ChangeLog + +++ b/exec/ChangeLog + @@ -1,3 +1,8 @@ + +2009-04-03 Samuel Thibault <samuel.thibault@ens-lyon.org> + + + + * exec.c (prepare): Call PREPARE_STREAM earlier to permit calling + + finish_mapping on E even after errors, as is already done in do_exec. + + + 2008-06-10 Samuel Thibault <samuel.thibault@ens-lyon.org> + + * elfcore.c (TIME_VALUE_TO_TIMESPEC): Completely implement instead of + diff --git a/exec/exec.c b/exec/exec.c + index 05dc883..cb3d741 100644 + --- a/exec/exec.c + +++ b/exec/exec.c + @@ -726,6 +726,9 @@ prepare (file_t file, struct execdata *e) + + e->interp.section = NULL; + + + /* Initialize E's stdio stream. */ + + prepare_stream (e); + [...] + +Luckily, Git saves the previous (i.e. before the `git reset`) `HEAD` reference +as `ORIG_HEAD`. Have a look at it with `git show ORIG_HEAD` -- it contains the +*big blob* commit, including the preliminary commit message -- just what HEAD +was before: + + $ git show ORIG_HEAD + commit 93e97f3351337c349e2926f4041e61bc487ef9e6 + Author: Thomas Schwinge <tschwinge@gnu.org> + Date: Tue Jun 23 00:27:28 2009 +0200 + + Bla. + + * console-client/timer.h (fetch_jiffies): Use static inline instead of extern + inline. + [...] + + diff --git a/Makeconf b/Makeconf + index e9b2045..236f1ec 100644 + --- a/Makeconf + +++ b/Makeconf + @@ -65,7 +65,7 @@ INCLUDES += -I$(..)include -I$(top_srcdir)/include + CPPFLAGS += $(INCLUDES) \ + -D_GNU_SOURCE -D_IO_MTSAFE_IO -D_FILE_OFFSET_BITS=64 \ + $($*-CPPFLAGS) + -CFLAGS += -std=gnu99 $(gnu89-inline-CFLAGS) -Wall -g -O3 \ + +CFLAGS += -std=gnu99 -Wall -g -O3 \ + [...] + +OK, now let's pick the files that we want to have in the first of the +envisioned two commits: these are the *static inline instead of extern inline* +and *apply inline optimization only...* sections. + + $ git add console-client/timer.h ext2fs/ext2fs.h [...] libthreads/rwlock.c + +Oh, we forgot something: now that we're preparing this stuff to go into the +*master* repository, update the copyright years. Edit, edit, edit, and then, +again: + + $ git add console-client/timer.h ext2fs/ext2fs.h [...] libthreads/rwlock.c + +Now Git's staging area contains the changes that we want to commit (and the +working directory contains the rest of the *big blob*). Commit these `add`ed +files, and use *big blob*'s commit message as a template for the new one, as it +already contains most of what we want (don't forget to chop off the unneeded +parts). + + $ git commit -c ORIG_HEAD + Waiting for Emacs... + [master-fix_inline 51c15bc] Use static inline where appropriate. + 6 files changed, 50 insertions(+), 51 deletions(-) + $ git show HEAD + commit c6c9d7a69dea26e04bba7010582e7bcd612e710c + Author: Thomas Schwinge <tschwinge@gnu.org> + Date: Tue Jun 23 00:27:28 2009 +0200 + + Use static inline where appropriate and use glibc's __extern_inline machinery. + + * console-client/timer.h (fetch_jiffies): Use static inline instead of extern + inline. + * ext2fs/ext2fs.h (test_bit, set_bit, clear_bit, dino, global_block_modified) + (record_global_poke, sync_global_ptr, record_indir_poke, sync_global) + (alloc_sync): Likewise. + * libftpconn/priv.h (unexpected_reply): Likewise. + * term/term.h (qsize, qavail, clear_queue, dequeue_quote, dequeue) + (enqueue_internal, enqueue, enqueue_quote, unquote_char, char_quoted_p) + (queue_erase): Likewise. + * ufs/ufs.h (dino, indir_block, cg_locate, sync_disk_blocks, sync_dinode) + (swab_short, swab_long, swab_long_long): Likewise. + * term/munge.c (poutput): Use static inline instead of inline. + + * libdiskfs/diskfs.h: Apply inline optimization only ifdef + [__USE_EXTERN_INLINES]. Use __extern_inline instead of extern inline. + * libftpconn/ftpconn.h: Likewise. + * libpipe/pipe.h: Likewise. + * libpipe/pq.h: Likewise. + * libshouldbeinlibc/idvec.h: Likewise. + * libshouldbeinlibc/maptime.h: Likewise. + * libshouldbeinlibc/ugids.h: Likewise. + * libstore/store.h: Likewise. + * libthreads/rwlock.h: Likewise. + * libdiskfs/extern-inline.c: Adapt to these changes. + * libftpconn/xinl.c: Likewise. And don't #include "priv.h". + * libpipe/pipe-funcs.c: Likewise. + * libpipe/pq-funcs.c: Likewise. + * libshouldbeinlibc/maptime-funcs.c: Likewise. And remove superfluous + includes. + * libstore/xinl.c: Likewise. + * libthreads/rwlock.c: Likewise. + + diff --git a/console-client/timer.h b/console-client/timer.h + index 4204192..5e64e97 100644 + --- a/console-client/timer.h + +++ b/console-client/timer.h + @@ -1,5 +1,7 @@ + /* timer.h - Interface to a timer module for Mach. + - Copyright (C) 1995,96,2000,02 Free Software Foundation, Inc. + + + + Copyright (C) 1995, 1996, 2000, 2002, 2009 Free Software Foundation, Inc. + + + Written by Michael I. Bushnell, p/BSG and Marcus Brinkmann. + + This file is part of the GNU Hurd. + @@ -54,7 +56,7 @@ int timer_remove (struct timer_list *timer); + /* Change the expiration time of the timer TIMER to EXPIRES. */ + void timer_change (struct timer_list *timer, long long expires); + + -extern inline long long + +static inline long long + [...] + +As you can see, `HEAD` now points to the new commit on top of the current +branch. (`ORIG_HEAD` doesn't change.) + +On to the next, and last one, only two changes should be left: the `Makeconf` +and `pfinet/Makefile` ones. + + $ git status + # On branch master-fix_inline + # Your branch is ahead of 'savannah/master' by 1 commit. + # + # Changed but not updated: + # (use "git add <file>..." to update what will be committed) + # (use "git checkout -- <file>..." to discard changes in working directory) + # + # modified: Makeconf + # modified: configure + # modified: pfinet/Makefile + # + # Untracked files: + # (use "git add <file>..." to include in what will be committed) + # + # 0001-Bla.patch + # autom4te.cache/ + # hurd_extern_inline_fix.patch?file_id=18191 + no changes added to commit (use "git add" and/or "git commit -a") + +Alright, there is as well still the `configure` hunk that we want to get rid +of. But first for the real second commit, after editing for again adding the +copyright year update: + + $ git add Makeconf pfinet/Makefile + $ git commit -c ORIG_HEAD + Waiting for Emacs... + [master-fix_inline 6a967d1] We're now C99 inline safe -- apart from the Linux code in pfinet. + 2 files changed, 6 insertions(+), 3 deletions(-) + +Check that we're in a clean state now: + + $ git status + # On branch master-fix_inline + # Your branch is ahead of 'savannah/master' by 2 commits. + # + # Changed but not updated: + # (use "git add <file>..." to update what will be committed) + # (use "git checkout -- <file>..." to discard changes in working directory) + # + # modified: configure + # + # Untracked files: + # (use "git add <file>..." to include in what will be committed) + # + # 0001-Bla.patch + # autom4te.cache/ + # hurd_extern_inline_fix.patch?file_id=18191 + no changes added to commit (use "git add" and/or "git commit -a") + +Oops, we forgot something... + + $ git checkout -- configure + +Now, our tree is clean again. (Check with `git status`.) + +By now, we came to the conclusion that the first of the two commits should have +been further split into two separate ones. Of course, essentially we would do +the same splitting again that we've done just now -- but how to easily modify +the first commit, now that we have another one on top of it? + +Alright, `git rebase --interactive` to the rescue -- let's interactively +*`rebase`* the last two commits, to modify them as wanted. + + $ git rebase --interactive HEAD~2 + Waiting for Emacs... + +Emacs wants us to tell which commits we want to keep as they are (`pick`), +which should be merged into others (`squash`), and which we want to `edit`. In +our scenario, we want to `edit` the first one and `pick` the second one. +Change the file thusly and close it. + + Stopped at 5becbb5... Use static inline where appropriate and use... + You can amend the commit now, with + + git commit --amend + + Once you are satisfied with your changes, run + + git rebase --continue + +We want to undo this first commit to split it into two. Again, use `git reset` +for that, while preserving the commit's changes in the working directory. + + $ git reset HEAD^ + console-client/timer.h: locally modified + [...] + +Pick the set of files that we want to have in the first of the envisioned two +commits: the *static inline instead of extern inline* section, and commit them, +again using the previous commit message as a template for the new one: + + $ git add console-client/timer.h ext2fs/ext2fs.h [...] term/munge.c + $ git commit -c ORIG_HEAD + Waiting for Emacs... + [detached HEAD 51c15bc] Use static inline where appropriate. + 6 files changed, 50 insertions(+), 51 deletions(-) + +Next part: *apply inline optimization only...*. Again, `git add` those files +that shall be part of the next commit, i.e. all remaining ones. As before, use +the previous commit message as a template. + + $ git add libdiskfs/diskfs.h [...] libthreads/rwlock.c + $ git commit -c ORIG_HEAD + Waiting for Emacs... + [detached HEAD 8ac30ea] [__USE_EXTERN_INLINES]: Use glibc's __extern_inline machinery. + 16 files changed, 508 insertions(+), 356 deletions(-) + +Now we're done with splitting that commit into two. (Check with `git status` +that we didn't forget anything.) What's missing is getting back the other +commit on top of the two now-split ones: + + $ git rebase --continue + Successfully rebased and updated refs/heads/master-fix_inline. + +Here we go. The other commit has been applied on top of the two new ones. + +Due to time-honored tradition, I always double-check what I have just +committed, before distributing it to the world: + + $ git log --reverse -p -C --cc savannah/master.. + +... and promptly, I recognize some changes that shouldn't be in there: when +using it on some files, Emacs' `copyright-fix-years`, aside from indeed fixing +the list of copyright years, and adding the current year, also changed *GPL +... version 2* into *version 3*, which would be nice, but which we can't do for +the moment. The error is present only in the first and second commit. If it +were in only in the third (the last) one, simply editing the files, and then +using `git commit --amend` would be the solution. But again there is the +problem about how to modify the first (`HEAD~2`) and second (`HEAD~1`, or +`HEAD^`) commit now that there is another one on top of it. By now, we know +the solution: + + $ git rebase --interactive HEAD~3 + Waiting for Emacs... + +This time, we need to `edit` the first and second commits, and `pick` the third +one. + + Stopped at ffd215b... Use static inline where appropriate. + You can amend the commit now, with + + git commit --amend + + Once you are satisfied with your changes, run + + git rebase --continue + +`git show` (which defaults to showing `HEAD`, by the way) can again be used to +have a look at the current `HEAD` (which is the first of the three commits), +and then we revert the unwanted changes in the editor, resulting with the +following changed files: + + $ git status + # Not currently on any branch. + # Changed but not updated: + # (use "git add <file>..." to update what will be committed) + # (use "git checkout -- <file>..." to discard changes in working directory) + # + # modified: ext2fs/ext2fs.h + # modified: libftpconn/priv.h + # modified: term/munge.c + # modified: term/term.h + # modified: ufs/ufs.h + # + # Untracked files: + # (use "git add <file>..." to include in what will be committed) + # + # 0001-Bla.patch + # autom4te.cache/ + # hurd_extern_inline_fix.patch?file_id=18191 + no changes added to commit (use "git add" and/or "git commit -a") + +Then, we can -- as `git rebase` suggested above -- *amend* the existing `HEAD` +commit with these changes (`--amend` and `--all`), reusing `HEAD`'s commit +message without spawning an editor (`-C HEAD`): + + $ git commit --amend -C HEAD --all + [detached HEAD c6c9d7a] Use static inline where appropriate. + 6 files changed, 45 insertions(+), 46 deletions(-) + +Continue with the next commit: + + $ git rebase --continue + Stopped at 8ac30ea... [__USE_EXTERN_INLINES]: Use glibc's __extern_inline machinery. + You can amend the commit now, with + + git commit --amend + + Once you are satisfied with your changes, run + + git rebase --continue + +Again, have a look at the commit (`git show`), revert the unwanted changes, +*amend* `HEAD`, and continue to the next commit: + + $ git commit --amend -C HEAD --all + [detached HEAD 9990dc6] [__USE_EXTERN_INLINES]: Use glibc's __extern_inline machinery. + 16 files changed, 500 insertions(+), 348 deletions(-) + $ git rebase --continue + Stopped at 6a967d1... We're now C99 inline safe -- apart from the Linux code in pfinet. + You can amend the commit now, with + + git commit --amend + + Once you are satisfied with your changes, run + + git rebase --continue + +Two files are left to be edited (`git show`, etc., again), and finally: + + $ git commit --amend -C HEAD --all + [detached HEAD 241c605] We're now C99 inline safe -- apart from the Linux code in pfinet. + 2 files changed, 5 insertions(+), 2 deletions(-) + $ git rebase --continue + Successfully rebased and updated refs/heads/master-fix_inline. + +That's it. `git log --reverse -p -C --cc savannah/master..` now looks as nice +as can be. + + +Of course, this is only a small insight of what is possible to do with `git +rebase` and friends -- see the manual for further explanations. diff --git a/community/weblogs/tschwinge/2009-06-23_split_patch_and_git_rebase_--interactive/0001-Bla.patch.bz2 b/community/weblogs/tschwinge/2009-06-23_split_patch_and_git_rebase_--interactive/0001-Bla.patch.bz2 Binary files differnew file mode 100644 index 00000000..4a682c86 --- /dev/null +++ b/community/weblogs/tschwinge/2009-06-23_split_patch_and_git_rebase_--interactive/0001-Bla.patch.bz2 diff --git a/community/weblogs/tschwinge/2009-06-24_importing_from_gnu_arch_into_git.mdwn b/community/weblogs/tschwinge/2009-06-24_importing_from_gnu_arch_into_git.mdwn new file mode 100644 index 00000000..f397e75b --- /dev/null +++ b/community/weblogs/tschwinge/2009-06-24_importing_from_gnu_arch_into_git.mdwn @@ -0,0 +1,104 @@ +[[!meta copyright="Copyright © 2009 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="converting from GNU arch to Git -- without direct repository +access"]] + +I wanted to import an old GNU arch repository into Git, but only had HTTP +access via ArchZoom. I spent quite some time to try teaching `git archimport` +to use HTTP access to that repository, but this didn't work out. Too bad -- +but at least, using ArchZoom, I was able to get the individual revisions' +tarballs: + + $ ls -1 *.tar.gz + bpf--devel--0.0--base-0.tar.gz + bpf--devel--0.0--patch-1.tar.gz + bpf--devel--0.0--patch-10.tar.gz + bpf--devel--0.0--patch-11.tar.gz + bpf--devel--0.0--patch-12.tar.gz + bpf--devel--0.0--patch-2.tar.gz + bpf--devel--0.0--patch-3.tar.gz + [...] + bpf--devel--0.0--patch-9.tar.gz + bpf--release--0.1--base-0.tar.gz + bpf--release--0.1--patch-1.tar.gz + bpf--release--0.1--patch-2.tar.gz + [...] + bpf--release--0.1--patch-8.tar.gz + +I unpacked these: + + $ for f in *.tar.gz; do tar -xz < "$f" || echo >&2 "$f" failed; done + +The last revision's tree apparently contains all previous revisions' commit +information (author, date, message), so use that: + + $ cp -a ↩ + bpf--release--0.1--patch-8/{arch}/bpf/bpf--devel/bpf--devel--0.0/info@hurdfr.org--hurdfr/patch-log ↩ + d-patch-log + $ cp -a ↩ + bpf--release--0.1--patch-8/{arch}/bpf/bpf--release/bpf--release--0.1/info@hurdfr.org--hurdfr/patch-log ↩ + r-patch-log + +... and extract the information that we need: + + $ base=bpf--devel--0.0-- && ↩ + for f in d-patch-log/*; do ↩ + grep < "$f" ^Creator: | head -n 1 ↩ + | { read j c && ↩ + echo "$c" | sed s%' <.*'%% ↩ + > "$base""$(basename "$f")".author_name && ↩ + echo "$c" | sed -e 's%.*<%%' -e 's%>.*%%' ↩ + > "$base""$(basename "$f")".author_email; } && ↩ + grep < "$f" ^Standard-date: | head -n 1 | { read j d && echo "$d" ↩ + > "$base""$(basename "$f")".author_date; } && ↩ + { grep < "$f" ^Summary: | head -n 1 | { read j m && echo "$m"; } && ↩ + echo && sed < "$f" '1,/^$/d'; } ↩ + > "$base""$(basename "$f")".log ↩ + || echo >&2 "$f" failed; ↩ + done + $ base=bpf--release--0.1-- && ↩ + for f in r-patch-log/*; [...] + +(Of course, I could have used something more elaborate than shell scripting...) + +Remove the GNU arch stuff that we don't need anymore: + + $ find bpf--*/ -type d \( -name {arch} -o -name .arch-ids \) -print0 ↩ + | xargs -0 rm -r + +The `base-0` revisions are actually either empty (the `devel` one) or +equivalent to the previous revision (the `release` one), so remove these: + + $ rm -rf bpf--devel--0.0--base-0 bpf--release--0.1--base-0 + +Finally, import all the other ones: + + $ mkdir g && ( cd g/ && git init ) + $ for d in bpf--d*-? bpf--d*-?? bpf--r*; do ↩ + test -d "$d"/ || continue && ↩ + ( cd g/ && ↩ + rsync -a --delete --exclude=/.git ../"$d"/ ./ && ↩ + git add . && ↩ + GIT_AUTHOR_NAME="$(cat ../"$d".author_name)" ↩ + GIT_AUTHOR_EMAIL="$(cat ../"$d".author_email)" ↩ + GIT_AUTHOR_DATE="$(cat ../"$d".author_date)" ↩ + git commit -F ../"$d".log -a ); ↩ + done + +Voilà! + + +**Update 2009-06-25:** + +Half a day later, [[HurdFr]] published a `git archimport`-converted repository +-- which was *identical* to my hand-crafted one (apart from having +`git-archimport-id:` tags in the commit messages, and the first (empty) commit +not being stripped off). :-) |