path: root/community/gsoc/2012/virt/discussion.mdwn

[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]]

[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]


# IRC, freenode, #hurd, 2012-07-19

    <nowhere_man> well, I really actively started last week, so I'm ironing my
      various use cases and above all I'm taking my barings in Hurd's code
    <nowhere_man> I'm currently reading boot/ and pfinet/
    <braunr> sorry for asking but
    <braunr> can you describe brielfy what you mean to achieve
    <braunr> i know it sounds weird but the project description is a bit vague
      for me
    <nowhere_man> OK
    <nowhere_man> the main goal is to be able to easily spawn a subhurd that's
      connected in some way to its host
    <braunr> ok
    <nowhere_man> mainly connected by network, possibly sharing resources like
      FS
    <braunr> is it similar in spirit with something like linux containers ?
    <nowhere_man> IIRC about them, yes
    <braunr> ok
    <braunr> that will do for me then
    <tschwinge> Yes, so not complete virtualization, but instaed limitied to
      several components.
    <braunr> lxc with more runtime features to increase/decrease the level of
      isolation
    <nowhere_man> at first it would be static, at creation time only
    <braunr> ok, i clearly understand the proposal now :)
    <braunr> what kind of help could you need in the near future ?
    <braunr> (except permanent access to youpi's brain?)
    <tschwinge> Yes, that's my question, too -- what can we do to "get this
      thing going".
    <nowhere_man> by monday or tuesday I should be clear on what I understand
      or not in the code
    <nowhere_man> I'm still a bit up to my elbows in it
    <nowhere_man> at that point I'll be happy to be able to pop a lot of
      questions about it
    <braunr> so you'll be ready for the next meeting
    <nowhere_man> yeah
    <tschwinge> Please do as soon as there are questions that you cannot
      resolve in a reasonably short amount of time.
    <tschwinge> So often a quick hint from someone else already helps to ge
      un-stuck.
    <nowhere_man> OK
    <tschwinge> There is no problem with asking for help given this huge and
      convoluted code-base, where often design decisions are not obvious, too.
    <nowhere_man> I will
    <tschwinge> Good.  :-)
    <antrik> nowhere_man: hm... what you said so far doesn't sound any
      different than the work zhengda already did on boot years ago...
    <antrik> (although none of it ever got upstream IIRC :-( )
    <nowhere_man> antrik: wasn't aware of it, is there some code published?
    <tschwinge> There are bits and pieces, but certainly there is enough work
      left to be done, to put it all together.
    <antrik> yes, his git repository should be up somewhere. it's quite
      convoluted though, as he worked on several things, and also wasn't very
      experienced with revision control in the beginning
    <tschwinge> nowhere_man:
      http://www.gnu.org/software/hurd/community/gsoc/2008.html
    <tschwinge> nowhere_man: http://www.gnu.org/software/hurd/user/zhengda.html
    <tschwinge> Second section of the latter one.
    <antrik> well, my understanding of the proposal (and more or less what I
      was driving at in the project idea, which is rather vague admittedly) is
      something lighter than a real subhurd... rather some kind of thin
      subenvironment that doesn't actually boot a complete system instance with
      various daemons etc.
    <tschwinge> nowhere_man: It is certainly valid for you to use pre-existing
      code/patches, by the way.
    <antrik> BTW, regarding the "full subhurd" thing, the missing pieces are
      mostly virtual device implementations
    <antrik> (that and some tough bug(s) remaining in zhengda's modified
      boot...)
    <nowhere_man> cool, I'll take a look
    <antrik> in any case, getting a picture of the work zhengda did is, is
      definitely the first thing to do :-)
    <tschwinge> nowhere_man: I'll also try to locate some bits and stuff from
      his verious repositories (I just fond a Subverision one; will convert to
      Git).
    <antrik> tschwinge: I'm pretty sure zhengda's git repository was converted
      from the SVN one...
    <tschwinge> antrik: Thanks for reminding us about this -- I failed to
      remember all that.
    <antrik> (which was in turn converted from CVS...)
    <tschwinge> antrik: OK, will have a lot.
    <tschwinge> Yeah, found a CVS tree, too.  ;-)
    <antrik> BTW, zhengda's work more exactly was about subhurd without root
      privileges. but that lays a lot of the groundwork for all kinds of more
      flexible subhurd usage
    <antrik> (but it's still quite a different thing that thing
      subenvironments, so don't get confused...)
    <antrik> err... thin subenvironments


# IRC, freenode, #hurd, 2012-07-27

    <nowhere_man> bddebian: I'm actually not progressing much while reading the
      source, I'm jumping all over the place to grasp the various types and
      functions used where I start
    <nowhere_man> would there be a few starting points that could help me?
    <tschwinge> nowhere_man: So what exactly is your status; what are you
      doing, what do you need help with?  We surely can provide help, but need
      to know where.
    <nowhere_man> I'm starting from the source of boot/ and pfinet/ and as soon
      as I encounter something that I don't understand, I find its definition
    <nowhere_man> I'm kind of doing a depth-first search of what I need to
      understand in the source code
    <nowhere_man> I'm wondering if there are a few places in the source code
      that I should start reading before anything else
    <nowhere_man> well, I'll have to go in a few minutes
    <nowhere_man> I'll continue my DFS ;-)


# IRC, freenode, #hurd, 2012-08-02

    <nowhereman> well, I made a leap forward in understanding the code, when I
      stopped my DFS
    <nowhereman> in hindsight, I'd say my way of approaching the code was
      probably one of the worst possible
    <braunr> oh
    <tschwinge> OK, so at least you learned something, which is good.
    <tschwinge> So, what's the new approach?  And what are you working on at
      the moment
    <tschwinge> ?
    <nowhereman> I just remembered SICP, the idea of wishful thinking when you
      code, and didn't bother with the fine details behind what I'm reading
    <nowhereman> like, I don't really get what happens when a Mach port is
      allocated, but I know approximately what a Mach port is
    <tschwinge> So originally you worked on investigating all that, every line
      of code?
    <nowhereman> almost, yeah
    <braunr> nowhereman: again, feel free to ask
    <tschwinge> Yes indeed -- that's too complex for a single person to tackle
      at one time.
    <braunr> and quickly
    <braunr> don't loose time
    <tschwinge> Not even braunr and I have looked up all these things.
      (Speaking for Richard here, but I'm quite sure he'll agree.  Perhaps he
      has in fact looked up all the Mach things, though.)
    <tschwinge> nowhereman: ufc?
    <nowhereman> BTW, last week I wanted to push my description of how the tool
      could be used, the use cases
    <nowhereman> ufs
    <nowhereman> but flubber is not online
    <tschwinge> nowhereman: Oh, why ufs specifically?
    <braunr> don't waste time on ufs
    <braunr> really
    <tschwinge> nowhereman: Yes, flubber is down.  But you can push directly to
      the Savannah repository.
    <tschwinge> nowhereman: Please immediatelly tell us if you're stuck on
      something, like flubber not being available.
    <tschwinge> We may not be able to help immediatelly, but we're the at least
      aware of issues.
    <braunr> and we may be able to help immediately :)
    <tschwinge> As we're not sitting in a lab next to each other, we can't tell
      otherwise what's going on.
    <tschwinge> We may in fact even be able to tell you immediatelly to use
      Savannah instead of flubber, indeed.
    <tschwinge> nowhereman: So, back to ufs -- which you don't specifically
      need to look at, I think -- ext2fs is what everyone uses.  But even there
      you shouldn't really need to know many details/internals.
    <nowhereman> OK, I was looking into it has it appears in hurd.boot
    <tschwinge> Ah, OK.  Yeah, that's just an example/template, and should use
      ext2fs nowadays.
    <nowhereman> in fact, as far as FS are concerned, I suppose I will merely
      need to know how to pass a port to the host's FS to some proxy FS in the
      subhurd
    <nowhereman> mmmh, Savannah only mentions a hurd.git
    <tschwinge> Exactly that is the abstraction level you need, yes.
    <nowhereman> I'm looking at http://savannah.gnu.org/git/?group=hurd
    <tschwinge> Yeah, that's a known shortcoming -- look here instead:
      https://git.savannah.gnu.org/cgit/hurd
    <tschwinge> Here is some more up-to-date stuff on subhurds:
      http://www.gnu.org/software/hurd/hurd/subhurd.html
    <tschwinge> nowhereman: You know how to tell git to add a new remote to
      your web pages checkout and such stuff?
    <nowhereman> yeah, no problem with that
    <braunr> have you prepared any question to ask us ?
    <nowhereman> the only I have now is if you can tell me where to look in the
      code about passing Mach ports
    <braunr> you don't pass ports, you pass rights
    <braunr> http://www.gnu.org/software/hurd/gnumach-doc/index.html is the
      best location to have a look at
    <braunr>
      http://www.gnu.org/software/hurd/gnumach-doc/Exchanging-Port-Rights.html#Exchanging-Port-Rights
    <braunr> i suppose the mig doc will help too, as you may be using a higher
      level interface to exchange rights
    <braunr> be careful about user references on port rights
    <braunr> deallocate releases a reference, it doesn't immediately destroy a
      resource
    <braunr> portinfo -v can help monitoring a task's rights
    <braunr> nowhereman: so what are you planning to do now ?
    <braunr> during the next week
    <nowhereman> documenting what I understand from the boot process and where
      things can be changed to fit my various use cases
    <braunr> do you expect that to take the whole week ?
    <nowhereman> and doing some first modifications to servers for the simplest
      cases
    <braunr> ok
    <braunr> well i hope you're able to really start working on it soon, and
      won't face weird issues in the meantime
    <braunr> i'm a bit disappointed that you don't have more questions
    <braunr> my feeling is you either did understand everything (except passing
      port rights), or you didn't attempt to seriously understand the code
    <braunr> or you don't dare ask questions
    <braunr> this is something that must change
    <braunr> or these meetings won't be as useful as they could be
    <tschwinge> Yes.  But also please don't wait for the meetings, but ask
      questions throughout the week, too.


# IRC, freenode, #hurd, 2012-08-09

    <nowhere_man> hey, does anyone knows the network device interface well?
    <nowhere_man> I don't get it by reading net_io.c/h in gnumach
    <braunr> nowhere_man: ask your question
    <braunr> nowhere_man: http://www.sceen.net/~rbraun/pcap-hurd.c <- this may
      help
    <nowhere_man> I don't see what the entry point is
    <nowhere_man> I finally understood that I actually don't need to touch
      pfinet for gsoc project
    <nowhere_man> but I should do a replacement network device instead
    <nowhere_man> is the net_io_init function called at start?
    <braunr> what entry point ?
    <braunr> and you should perhaps have a look at the eth-multiplexer by
      zhengda
    <braunr> yes net_io_init is called at startup
    <braunr> nowhere_man: did you find your answers about networking ?
    <nowhere_man> no, I'm still digging in mach's code
    <braunr> nowhere_man: well keep asking :/
    <braunr> you left conversation without notice :/
    <braunr> nowhere_man: and why mach ?
    <nowhere_man> I thought hardware devices are there
    <tschwinge> nowhere_man: You wanted to push your documentation one/two
      weeks ago.  Why has that not yet happened?
    <youpi> nowhere_man: they used to be there, they are now in netdde, but in
      both case it's just a matter of the same RPC interface
    <nowhere_man> tschwinge: I spent very few time this week on gsoc, and
      completely forgot about the push on savannah
    <braunr> nowhere_man: i told you to look at the work by zhengda concerning
      eth-multiplexer, did you do that ?
    <tschwinge> nowhere_man: You realize GSoC is meant to be a full-time job?
    <tschwinge> Or, next to full-time?
    <braunr> it's full-time normally
    <braunr> the payment is justified by that
    <youpi> nowhere_man: most RPC operations you need to know about network can
      be seen at work in pfinet/ethernet.c, wherever "ether_port" appears
    <youpi> i.e. device_open, set_filter, write, set/get_status
    <braunr> again, http://www.sceen.net/~rbraun/pcap-hurd.c should guide you
      pretty well
    <braunr> since it's the very least necessary to use that interface
    <tschwinge> nowhere_man: How, roughly but realistically, are your plans to
      continue this task?
    <tschwinge> nowhere_man: What has been blocking you this week so you
      couldn't work on your task?
    <nowhere_man> tschwinge: mostly a previous work that was supposed to end at
      the beginning of the summer and only went online now, for which I'm
      basically sysadmin
    <braunr> 21:25 < tschwinge> nowhere_man: How, roughly but realistically,
      are your plans to continue this task?
    <braunr> this question is really more interesting actually
    <nowhere_man> right now, I want to write a netword device that just sends
      its frames by IPC
    <braunr> why ?
    <nowhere_man> as I never wrote any program using Mach's IPC, that seems the
      easiest to get them right
    <braunr> you won't have time
    <braunr> 21:22 < braunr> nowhere_man: i told you to look at the work by
      zhengda concerning eth-multiplexer, did you do that ?
    <nowhere_man> braunr: not yet, no
    <braunr> well that's your best chance to make some progress
    <nowhere_man> braunr: is writing the virtal network device that hard?
    <braunr> basically, it allows "bridgind" the pfinet instances of various
      subhurds
    <braunr> the virtual network device you want *is* eth-multiplexer
    <tschwinge> nowhere_man: GSoC is nearly over.  That's why I'm asking how
      this task is going to continue.  I'm sorry but I reckon you have not
      spend anywhere near the amount of hours that are meant to be spent on it.
    <braunr> and from what antrik told me, yes it's hard, and moreover, why
      rewrite it if it already exists and you're late
    <braunr> i agree
    <nowhere_man> tschwinge: I know, I've started way too late because of my
      second round of exams
    <tschwinge> nowhere_man: OK, that's how you started.  But how is it going
      to continue...
    <nowhere_man> tschwinge: in short, I write a prototype that just starts a
      subhurd, and when that works correctly I add the network
    <tschwinge> nowhere_man: I mean from an organizational point of view.
    <nowhere_man> well, between now and the beginning of september, I'll work
      full-time on this
    <nowhere_man> up until september 8th


# IRC, freenode, #hurd, 2012-08-09

    <antrik> nowhere_man: you do *not* have to do a replacement network
      device. zhengda did that years ago.
    <antrik> nowhere_man: also note that zhengda also implemented the support
      for *using* the virtual network device (in fact any replacement devices
      -- except that no others actually exist yet) in boot
    <youpi> which is already in, actually, isn't it?
    <antrik> youpi: hm, yes... it was the patch that zhengda posted on the list
      once, but later updated, and at some later point you merged the outdated
      variant from the list...
    <youpi> outdated?
    <youpi> ah, but he never posted the updated one, and it got lost in git
      repos, right?
    <youpi> (what was updated actually?)
    <antrik> he changed the option name and description later for more
      clarity. don't remember whether there were other changes
    <antrik>   -f, --device=device_name=device_file
    <antrik>                              Specify a device file used by subhurd
      and its
    <antrik>                              virtual name.
    <antrik> that's the one from the Debian package
    <antrik>   -m, --device-map=DEBICENAME=DEVICEFILE
    <antrik>                              Map the device in subhurd to the
      device in the
    <antrik>                              main Hurd.
    <antrik> that's the one I have locally built from his tree
    <youpi> so you actually have access to his tree?
    <antrik> uhm... I used to... it was on flubber


# IRC, freenode, #hurd, 2012-08-18

    <nowhere_man> so, this week I discovered how fun it is to work on a
      non-mainstream OS
    <nowhere_man> I hoped to start coding the tool itself, put together the
      skeleton, but every Lisp implementation I tried had problems
    <braunr> ah you want to write it in lisp ?
    <nowhere_man> ECL, that I had ported a few years ago, actually FTBFS since
    <nowhere_man> I hoped to be able, it would be easier for me
    <nowhere_man> and when I tried Scheme, I started with Guile (it's GNU's own
      Scheme implementation, after all)
    <nowhere_man> and when I execute the FFI functions, to access functions in
      libmachdec
    <nowhere_man> I get SIGILL
    <braunr> i can't advise you about anything lisp related
    <braunr> the most reliable thing you'll find on the hurd is C
    <nowhere_man> I tried to debug that, but running Guile in GDB gets me a
      SIGSEV
    <nowhere_man> I'll try to make ECL to build again
    <braunr> this seems like a waste of time to me
    <braunr> avoid spending time on anything that isn't directly related to
      your goal if you still hope to finish it
    <nowhere_man> I'm ten times more comfortable coding in Lisp
    <braunr> it doesn't matter, you're late
    <nowhere_man> yeah, I know, so taking the time to correct that problem
      won't change the fact that I won't finish in time
    <nowhere_man> so I'll finish anyway, and in Lisp
    <braunr> and if you lack something else, like some mach/hurd specific lisp
      bindings, you'll have to spend more time on that
    <braunr> ok
    <nowhere_man> do you know if someone had a SIGILL situation on Hurd in the
      past?
    <nowhere_man> I'm wondering if that's a known kind of issue
    <braunr> there are lots of issues
    <braunr> especially when it comes to other languages and runtime
      environments
    <nowhere_man> but is it like MAX_PATH_LEN, something that is known to
      happen when porting something on Hurd?
    <braunr> i'm not sure how comparable it is
    <braunr> i'd say it's often before of the conformance issues of the hurd
    <braunr> because*
    <nowhere_man> like missing bits of POSIX ?
    <braunr> or simple wrong for some corner cases
    <braunr> simply*
    <bubu^> nowhere_man, I was able to run guile on my hurd image through qemu
    <bubu^> but I didn't make any complexe programms to check if everything
      works fine
    <nowhere_man> yeah, it runs fine
    <nowhere_man> FFI functions get you a SIGILL
    <nowhere_man>
      http://www.gnu.org/software/guile/manual/html_node/Dynamic-FFI.html
    <nowhere_man> the define-module form at the beginning triggers the signal
    <antrik> nowhere_man: what do you want to implement in Lisp?
    <antrik> BTW, the guy working on Lisp bindings a couple of years ago used
      Clisp
    <antrik> it was working back then
    <nowhere_man> antrik: the program that sets up a subhurd
    <nowhere_man> I always forget about clisp, I'll try it right away