summaryrefslogtreecommitdiff
path: root/community/gsoc/2012/virt/discussion.mdwn
diff options
context:
space:
mode:
Diffstat (limited to 'community/gsoc/2012/virt/discussion.mdwn')
-rw-r--r--community/gsoc/2012/virt/discussion.mdwn389
1 files changed, 389 insertions, 0 deletions
diff --git a/community/gsoc/2012/virt/discussion.mdwn b/community/gsoc/2012/virt/discussion.mdwn
new file mode 100644
index 00000000..e0085322
--- /dev/null
+++ b/community/gsoc/2012/virt/discussion.mdwn
@@ -0,0 +1,389 @@
+[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+
+# IRC, freenode, #hurd, 2012-07-19
+
+ <nowhere_man> well, I really actively started last week, so I'm ironing my
+ various use cases and above all I'm taking my barings in Hurd's code
+ <nowhere_man> I'm currently reading boot/ and pfinet/
+ <braunr> sorry for asking but
+ <braunr> can you describe brielfy what you mean to achieve
+ <braunr> i know it sounds weird but the project description is a bit vague
+ for me
+ <nowhere_man> OK
+ <nowhere_man> the main goal is to be able to easily spawn a subhurd that's
+ connected in some way to its host
+ <braunr> ok
+ <nowhere_man> mainly connected by network, possibly sharing resources like
+ FS
+ <braunr> is it similar in spirit with something like linux containers ?
+ <nowhere_man> IIRC about them, yes
+ <braunr> ok
+ <braunr> that will do for me then
+ <tschwinge> Yes, so not complete virtualization, but instaed limitied to
+ several components.
+ <braunr> lxc with more runtime features to increase/decrease the level of
+ isolation
+ <nowhere_man> at first it would be static, at creation time only
+ <braunr> ok, i clearly understand the proposal now :)
+ <braunr> what kind of help could you need in the near future ?
+ <braunr> (except permanent access to youpi's brain?)
+ <tschwinge> Yes, that's my question, too -- what can we do to "get this
+ thing going".
+ <nowhere_man> by monday or tuesday I should be clear on what I understand
+ or not in the code
+ <nowhere_man> I'm still a bit up to my elbows in it
+ <nowhere_man> at that point I'll be happy to be able to pop a lot of
+ questions about it
+ <braunr> so you'll be ready for the next meeting
+ <nowhere_man> yeah
+ <tschwinge> Please do as soon as there are questions that you cannot
+ resolve in a reasonably short amount of time.
+ <tschwinge> So often a quick hint from someone else already helps to ge
+ un-stuck.
+ <nowhere_man> OK
+ <tschwinge> There is no problem with asking for help given this huge and
+ convoluted code-base, where often design decisions are not obvious, too.
+ <nowhere_man> I will
+ <tschwinge> Good. :-)
+ <antrik> nowhere_man: hm... what you said so far doesn't sound any
+ different than the work zhengda already did on boot years ago...
+ <antrik> (although none of it ever got upstream IIRC :-( )
+ <nowhere_man> antrik: wasn't aware of it, is there some code published?
+ <tschwinge> There are bits and pieces, but certainly there is enough work
+ left to be done, to put it all together.
+ <antrik> yes, his git repository should be up somewhere. it's quite
+ convoluted though, as he worked on several things, and also wasn't very
+ experienced with revision control in the beginning
+ <tschwinge> nowhere_man:
+ http://www.gnu.org/software/hurd/community/gsoc/2008.html
+ <tschwinge> nowhere_man: http://www.gnu.org/software/hurd/user/zhengda.html
+ <tschwinge> Second section of the latter one.
+ <antrik> well, my understanding of the proposal (and more or less what I
+ was driving at in the project idea, which is rather vague admittedly) is
+ something lighter than a real subhurd... rather some kind of thin
+ subenvironment that doesn't actually boot a complete system instance with
+ various daemons etc.
+ <tschwinge> nowhere_man: It is certainly valid for you to use pre-existing
+ code/patches, by the way.
+ <antrik> BTW, regarding the "full subhurd" thing, the missing pieces are
+ mostly virtual device implementations
+ <antrik> (that and some tough bug(s) remaining in zhengda's modified
+ boot...)
+ <nowhere_man> cool, I'll take a look
+ <antrik> in any case, getting a picture of the work zhengda did is, is
+ definitely the first thing to do :-)
+ <tschwinge> nowhere_man: I'll also try to locate some bits and stuff from
+ his verious repositories (I just fond a Subverision one; will convert to
+ Git).
+ <antrik> tschwinge: I'm pretty sure zhengda's git repository was converted
+ from the SVN one...
+ <tschwinge> antrik: Thanks for reminding us about this -- I failed to
+ remember all that.
+ <antrik> (which was in turn converted from CVS...)
+ <tschwinge> antrik: OK, will have a lot.
+ <tschwinge> Yeah, found a CVS tree, too. ;-)
+ <antrik> BTW, zhengda's work more exactly was about subhurd without root
+ privileges. but that lays a lot of the groundwork for all kinds of more
+ flexible subhurd usage
+ <antrik> (but it's still quite a different thing that thing
+ subenvironments, so don't get confused...)
+ <antrik> err... thin subenvironments
+
+
+# IRC, freenode, #hurd, 2012-07-27
+
+ <nowhere_man> bddebian: I'm actually not progressing much while reading the
+ source, I'm jumping all over the place to grasp the various types and
+ functions used where I start
+ <nowhere_man> would there be a few starting points that could help me?
+ <tschwinge> nowhere_man: So what exactly is your status; what are you
+ doing, what do you need help with? We surely can provide help, but need
+ to know where.
+ <nowhere_man> I'm starting from the source of boot/ and pfinet/ and as soon
+ as I encounter something that I don't understand, I find its definition
+ <nowhere_man> I'm kind of doing a depth-first search of what I need to
+ understand in the source code
+ <nowhere_man> I'm wondering if there are a few places in the source code
+ that I should start reading before anything else
+ <nowhere_man> well, I'll have to go in a few minutes
+ <nowhere_man> I'll continue my DFS ;-)
+
+
+# IRC, freenode, #hurd, 2012-08-02
+
+ <nowhereman> well, I made a leap forward in understanding the code, when I
+ stopped my DFS
+ <nowhereman> in hindsight, I'd say my way of approaching the code was
+ probably one of the worst possible
+ <braunr> oh
+ <tschwinge> OK, so at least you learned something, which is good.
+ <tschwinge> So, what's the new approach? And what are you working on at
+ the moment
+ <tschwinge> ?
+ <nowhereman> I just remembered SICP, the idea of wishful thinking when you
+ code, and didn't bother with the fine details behind what I'm reading
+ <nowhereman> like, I don't really get what happens when a Mach port is
+ allocated, but I know approximately what a Mach port is
+ <tschwinge> So originally you worked on investigating all that, every line
+ of code?
+ <nowhereman> almost, yeah
+ <braunr> nowhereman: again, feel free to ask
+ <tschwinge> Yes indeed -- that's too complex for a single person to tackle
+ at one time.
+ <braunr> and quickly
+ <braunr> don't loose time
+ <tschwinge> Not even braunr and I have looked up all these things.
+ (Speaking for Richard here, but I'm quite sure he'll agree. Perhaps he
+ has in fact looked up all the Mach things, though.)
+ <tschwinge> nowhereman: ufc?
+ <nowhereman> BTW, last week I wanted to push my description of how the tool
+ could be used, the use cases
+ <nowhereman> ufs
+ <nowhereman> but flubber is not online
+ <tschwinge> nowhereman: Oh, why ufs specifically?
+ <braunr> don't waste time on ufs
+ <braunr> really
+ <tschwinge> nowhereman: Yes, flubber is down. But you can push directly to
+ the Savannah repository.
+ <tschwinge> nowhereman: Please immediatelly tell us if you're stuck on
+ something, like flubber not being available.
+ <tschwinge> We may not be able to help immediatelly, but we're the at least
+ aware of issues.
+ <braunr> and we may be able to help immediately :)
+ <tschwinge> As we're not sitting in a lab next to each other, we can't tell
+ otherwise what's going on.
+ <tschwinge> We may in fact even be able to tell you immediatelly to use
+ Savannah instead of flubber, indeed.
+ <tschwinge> nowhereman: So, back to ufs -- which you don't specifically
+ need to look at, I think -- ext2fs is what everyone uses. But even there
+ you shouldn't really need to know many details/internals.
+ <nowhereman> OK, I was looking into it has it appears in hurd.boot
+ <tschwinge> Ah, OK. Yeah, that's just an example/template, and should use
+ ext2fs nowadays.
+ <nowhereman> in fact, as far as FS are concerned, I suppose I will merely
+ need to know how to pass a port to the host's FS to some proxy FS in the
+ subhurd
+ <nowhereman> mmmh, Savannah only mentions a hurd.git
+ <tschwinge> Exactly that is the abstraction level you need, yes.
+ <nowhereman> I'm looking at http://savannah.gnu.org/git/?group=hurd
+ <tschwinge> Yeah, that's a known shortcoming -- look here instead:
+ http://git.savannah.gnu.org/cgit/hurd
+ <tschwinge> Here is some more up-to-date stuff on subhurds:
+ http://www.gnu.org/software/hurd/hurd/subhurd.html
+ <tschwinge> nowhereman: You know how to tell git to add a new remote to
+ your web pages checkout and such stuff?
+ <nowhereman> yeah, no problem with that
+ <braunr> have you prepared any question to ask us ?
+ <nowhereman> the only I have now is if you can tell me where to look in the
+ code about passing Mach ports
+ <braunr> you don't pass ports, you pass rights
+ <braunr> http://www.gnu.org/software/hurd/gnumach-doc/index.html is the
+ best location to have a look at
+ <braunr>
+ http://www.gnu.org/software/hurd/gnumach-doc/Exchanging-Port-Rights.html#Exchanging-Port-Rights
+ <braunr> i suppose the mig doc will help too, as you may be using a higher
+ level interface to exchange rights
+ <braunr> be careful about user references on port rights
+ <braunr> deallocate releases a reference, it doesn't immediately destroy a
+ resource
+ <braunr> portinfo -v can help monitoring a task's rights
+ <braunr> nowhereman: so what are you planning to do now ?
+ <braunr> during the next week
+ <nowhereman> documenting what I understand from the boot process and where
+ things can be changed to fit my various use cases
+ <braunr> do you expect that to take the whole week ?
+ <nowhereman> and doing some first modifications to servers for the simplest
+ cases
+ <braunr> ok
+ <braunr> well i hope you're able to really start working on it soon, and
+ won't face weird issues in the meantime
+ <braunr> i'm a bit disappointed that you don't have more questions
+ <braunr> my feeling is you either did understand everything (except passing
+ port rights), or you didn't attempt to seriously understand the code
+ <braunr> or you don't dare ask questions
+ <braunr> this is something that must change
+ <braunr> or these meetings won't be as useful as they could be
+ <tschwinge> Yes. But also please don't wait for the meetings, but ask
+ questions throughout the week, too.
+
+
+# IRC, freenode, #hurd, 2012-08-09
+
+ <nowhere_man> hey, does anyone knows the network device interface well?
+ <nowhere_man> I don't get it by reading net_io.c/h in gnumach
+ <braunr> nowhere_man: ask your question
+ <braunr> nowhere_man: http://www.sceen.net/~rbraun/pcap-hurd.c <- this may
+ help
+ <nowhere_man> I don't see what the entry point is
+ <nowhere_man> I finally understood that I actually don't need to touch
+ pfinet for gsoc project
+ <nowhere_man> but I should do a replacement network device instead
+ <nowhere_man> is the net_io_init function called at start?
+ <braunr> what entry point ?
+ <braunr> and you should perhaps have a look at the eth-multiplexer by
+ zhengda
+ <braunr> yes net_io_init is called at startup
+ <braunr> nowhere_man: did you find your answers about networking ?
+ <nowhere_man> no, I'm still digging in mach's code
+ <braunr> nowhere_man: well keep asking :/
+ <braunr> you left conversation without notice :/
+ <braunr> nowhere_man: and why mach ?
+ <nowhere_man> I thought hardware devices are there
+ <tschwinge> nowhere_man: You wanted to push your documentation one/two
+ weeks ago. Why has that not yet happened?
+ <youpi> nowhere_man: they used to be there, they are now in netdde, but in
+ both case it's just a matter of the same RPC interface
+ <nowhere_man> tschwinge: I spent very few time this week on gsoc, and
+ completely forgot about the push on savannah
+ <braunr> nowhere_man: i told you to look at the work by zhengda concerning
+ eth-multiplexer, did you do that ?
+ <tschwinge> nowhere_man: You realize GSoC is meant to be a full-time job?
+ <tschwinge> Or, next to full-time?
+ <braunr> it's full-time normally
+ <braunr> the payment is justified by that
+ <youpi> nowhere_man: most RPC operations you need to know about network can
+ be seen at work in pfinet/ethernet.c, wherever "ether_port" appears
+ <youpi> i.e. device_open, set_filter, write, set/get_status
+ <braunr> again, http://www.sceen.net/~rbraun/pcap-hurd.c should guide you
+ pretty well
+ <braunr> since it's the very least necessary to use that interface
+ <tschwinge> nowhere_man: How, roughly but realistically, are your plans to
+ continue this task?
+ <tschwinge> nowhere_man: What has been blocking you this week so you
+ couldn't work on your task?
+ <nowhere_man> tschwinge: mostly a previous work that was supposed to end at
+ the beginning of the summer and only went online now, for which I'm
+ basically sysadmin
+ <braunr> 21:25 < tschwinge> nowhere_man: How, roughly but realistically,
+ are your plans to continue this task?
+ <braunr> this question is really more interesting actually
+ <nowhere_man> right now, I want to write a netword device that just sends
+ its frames by IPC
+ <braunr> why ?
+ <nowhere_man> as I never wrote any program using Mach's IPC, that seems the
+ easiest to get them right
+ <braunr> you won't have time
+ <braunr> 21:22 < braunr> nowhere_man: i told you to look at the work by
+ zhengda concerning eth-multiplexer, did you do that ?
+ <nowhere_man> braunr: not yet, no
+ <braunr> well that's your best chance to make some progress
+ <nowhere_man> braunr: is writing the virtal network device that hard?
+ <braunr> basically, it allows "bridgind" the pfinet instances of various
+ subhurds
+ <braunr> the virtual network device you want *is* eth-multiplexer
+ <tschwinge> nowhere_man: GSoC is nearly over. That's why I'm asking how
+ this task is going to continue. I'm sorry but I reckon you have not
+ spend anywhere near the amount of hours that are meant to be spent on it.
+ <braunr> and from what antrik told me, yes it's hard, and moreover, why
+ rewrite it if it already exists and you're late
+ <braunr> i agree
+ <nowhere_man> tschwinge: I know, I've started way too late because of my
+ second round of exams
+ <tschwinge> nowhere_man: OK, that's how you started. But how is it going
+ to continue...
+ <nowhere_man> tschwinge: in short, I write a prototype that just starts a
+ subhurd, and when that works correctly I add the network
+ <tschwinge> nowhere_man: I mean from an organizational point of view.
+ <nowhere_man> well, between now and the beginning of september, I'll work
+ full-time on this
+ <nowhere_man> up until september 8th
+
+
+# IRC, freenode, #hurd, 2012-08-09
+
+ <antrik> nowhere_man: you do *not* have to do a replacement network
+ device. zhengda did that years ago.
+ <antrik> nowhere_man: also note that zhengda also implemented the support
+ for *using* the virtual network device (in fact any replacement devices
+ -- except that no others actually exist yet) in boot
+ <youpi> which is already in, actually, isn't it?
+ <antrik> youpi: hm, yes... it was the patch that zhengda posted on the list
+ once, but later updated, and at some later point you merged the outdated
+ variant from the list...
+ <youpi> outdated?
+ <youpi> ah, but he never posted the updated one, and it got lost in git
+ repos, right?
+ <youpi> (what was updated actually?)
+ <antrik> he changed the option name and description later for more
+ clarity. don't remember whether there were other changes
+ <antrik> -f, --device=device_name=device_file
+ <antrik> Specify a device file used by subhurd
+ and its
+ <antrik> virtual name.
+ <antrik> that's the one from the Debian package
+ <antrik> -m, --device-map=DEBICENAME=DEVICEFILE
+ <antrik> Map the device in subhurd to the
+ device in the
+ <antrik> main Hurd.
+ <antrik> that's the one I have locally built from his tree
+ <youpi> so you actually have access to his tree?
+ <antrik> uhm... I used to... it was on flubber
+
+
+# IRC, freenode, #hurd, 2012-08-18
+
+ <nowhere_man> so, this week I discovered how fun it is to work on a
+ non-mainstream OS
+ <nowhere_man> I hoped to start coding the tool itself, put together the
+ skeleton, but every Lisp implementation I tried had problems
+ <braunr> ah you want to write it in lisp ?
+ <nowhere_man> ECL, that I had ported a few years ago, actually FTBFS since
+ <nowhere_man> I hoped to be able, it would be easier for me
+ <nowhere_man> and when I tried Scheme, I started with Guile (it's GNU's own
+ Scheme implementation, after all)
+ <nowhere_man> and when I execute the FFI functions, to access functions in
+ libmachdec
+ <nowhere_man> I get SIGILL
+ <braunr> i can't advise you about anything lisp related
+ <braunr> the most reliable thing you'll find on the hurd is C
+ <nowhere_man> I tried to debug that, but running Guile in GDB gets me a
+ SIGSEV
+ <nowhere_man> I'll try to make ECL to build again
+ <braunr> this seems like a waste of time to me
+ <braunr> avoid spending time on anything that isn't directly related to
+ your goal if you still hope to finish it
+ <nowhere_man> I'm ten times more comfortable coding in Lisp
+ <braunr> it doesn't matter, you're late
+ <nowhere_man> yeah, I know, so taking the time to correct that problem
+ won't change the fact that I won't finish in time
+ <nowhere_man> so I'll finish anyway, and in Lisp
+ <braunr> and if you lack something else, like some mach/hurd specific lisp
+ bindings, you'll have to spend more time on that
+ <braunr> ok
+ <nowhere_man> do you know if someone had a SIGILL situation on Hurd in the
+ past?
+ <nowhere_man> I'm wondering if that's a known kind of issue
+ <braunr> there are lots of issues
+ <braunr> especially when it comes to other languages and runtime
+ environments
+ <nowhere_man> but is it like MAX_PATH_LEN, something that is known to
+ happen when porting something on Hurd?
+ <braunr> i'm not sure how comparable it is
+ <braunr> i'd say it's often before of the conformance issues of the hurd
+ <braunr> because*
+ <nowhere_man> like missing bits of POSIX ?
+ <braunr> or simple wrong for some corner cases
+ <braunr> simply*
+ <bubu^> nowhere_man, I was able to run guile on my hurd image through qemu
+ <bubu^> but I didn't make any complexe programms to check if everything
+ works fine
+ <nowhere_man> yeah, it runs fine
+ <nowhere_man> FFI functions get you a SIGILL
+ <nowhere_man>
+ http://www.gnu.org/software/guile/manual/html_node/Dynamic-FFI.html
+ <nowhere_man> the define-module form at the beginning triggers the signal
+ <antrik> nowhere_man: what do you want to implement in Lisp?
+ <antrik> BTW, the guy working on Lisp bindings a couple of years ago used
+ Clisp
+ <antrik> it was working back then
+ <nowhere_man> antrik: the program that sets up a subhurd
+ <nowhere_man> I always forget about clisp, I'll try it right away