summaryrefslogtreecommitdiff
path: root/community/gsoc/2012/virt/discussion.mdwn
blob: 11c33e52f71e7449ef232714637378ece9d03396 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]]

[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]


# IRC, freenode, #hurd, 2012-07-19

    <nowhere_man> well, I really actively started last week, so I'm ironing my
      various use cases and above all I'm taking my barings in Hurd's code
    <nowhere_man> I'm currently reading boot/ and pfinet/
    <braunr> sorry for asking but
    <braunr> can you describe brielfy what you mean to achieve
    <braunr> i know it sounds weird but the project description is a bit vague
      for me
    <nowhere_man> OK
    <nowhere_man> the main goal is to be able to easily spawn a subhurd that's
      connected in some way to its host
    <braunr> ok
    <nowhere_man> mainly connected by network, possibly sharing resources like
      FS
    <braunr> is it similar in spirit with something like linux containers ?
    <nowhere_man> IIRC about them, yes
    <braunr> ok
    <braunr> that will do for me then
    <tschwinge> Yes, so not complete virtualization, but instaed limitied to
      several components.
    <braunr> lxc with more runtime features to increase/decrease the level of
      isolation
    <nowhere_man> at first it would be static, at creation time only
    <braunr> ok, i clearly understand the proposal now :)
    <braunr> what kind of help could you need in the near future ?
    <braunr> (except permanent access to youpi's brain?)
    <tschwinge> Yes, that's my question, too -- what can we do to "get this
      thing going".
    <nowhere_man> by monday or tuesday I should be clear on what I understand
      or not in the code
    <nowhere_man> I'm still a bit up to my elbows in it
    <nowhere_man> at that point I'll be happy to be able to pop a lot of
      questions about it
    <braunr> so you'll be ready for the next meeting
    <nowhere_man> yeah
    <tschwinge> Please do as soon as there are questions that you cannot
      resolve in a reasonably short amount of time.
    <tschwinge> So often a quick hint from someone else already helps to ge
      un-stuck.
    <nowhere_man> OK
    <tschwinge> There is no problem with asking for help given this huge and
      convoluted code-base, where often design decisions are not obvious, too.
    <nowhere_man> I will
    <tschwinge> Good.  :-)
    <antrik> nowhere_man: hm... what you said so far doesn't sound any
      different than the work zhengda already did on boot years ago...
    <antrik> (although none of it ever got upstream IIRC :-( )
    <nowhere_man> antrik: wasn't aware of it, is there some code published?
    <tschwinge> There are bits and pieces, but certainly there is enough work
      left to be done, to put it all together.
    <antrik> yes, his git repository should be up somewhere. it's quite
      convoluted though, as he worked on several things, and also wasn't very
      experienced with revision control in the beginning
    <tschwinge> nowhere_man:
      http://www.gnu.org/software/hurd/community/gsoc/2008.html
    <tschwinge> nowhere_man: http://www.gnu.org/software/hurd/user/zhengda.html
    <tschwinge> Second section of the latter one.
    <antrik> well, my understanding of the proposal (and more or less what I
      was driving at in the project idea, which is rather vague admittedly) is
      something lighter than a real subhurd... rather some kind of thin
      subenvironment that doesn't actually boot a complete system instance with
      various daemons etc.
    <tschwinge> nowhere_man: It is certainly valid for you to use pre-existing
      code/patches, by the way.
    <antrik> BTW, regarding the "full subhurd" thing, the missing pieces are
      mostly virtual device implementations
    <antrik> (that and some tough bug(s) remaining in zhengda's modified
      boot...)
    <nowhere_man> cool, I'll take a look
    <antrik> in any case, getting a picture of the work zhengda did is, is
      definitely the first thing to do :-)
    <tschwinge> nowhere_man: I'll also try to locate some bits and stuff from
      his verious repositories (I just fond a Subverision one; will convert to
      Git).
    <antrik> tschwinge: I'm pretty sure zhengda's git repository was converted
      from the SVN one...
    <tschwinge> antrik: Thanks for reminding us about this -- I failed to
      remember all that.
    <antrik> (which was in turn converted from CVS...)
    <tschwinge> antrik: OK, will have a lot.
    <tschwinge> Yeah, found a CVS tree, too.  ;-)
    <antrik> BTW, zhengda's work more exactly was about subhurd without root
      privileges. but that lays a lot of the groundwork for all kinds of more
      flexible subhurd usage
    <antrik> (but it's still quite a different thing that thing
      subenvironments, so don't get confused...)
    <antrik> err... thin subenvironments


# IRC, freenode, #hurd, 2012-07-27

    <nowhere_man> bddebian: I'm actually not progressing much while reading the
      source, I'm jumping all over the place to grasp the various types and
      functions used where I start
    <nowhere_man> would there be a few starting points that could help me?
    <tschwinge> nowhere_man: So what exactly is your status; what are you
      doing, what do you need help with?  We surely can provide help, but need
      to know where.
    <nowhere_man> I'm starting from the source of boot/ and pfinet/ and as soon
      as I encounter something that I don't understand, I find its definition
    <nowhere_man> I'm kind of doing a depth-first search of what I need to
      understand in the source code
    <nowhere_man> I'm wondering if there are a few places in the source code
      that I should start reading before anything else
    <nowhere_man> well, I'll have to go in a few minutes
    <nowhere_man> I'll continue my DFS ;-)


# IRC, freenode, #hurd, 2012-08-02

    <nowhereman> well, I made a leap forward in understanding the code, when I
      stopped my DFS
    <nowhereman> in hindsight, I'd say my way of approaching the code was
      probably one of the worst possible
    <braunr> oh
    <tschwinge> OK, so at least you learned something, which is good.
    <tschwinge> So, what's the new approach?  And what are you working on at
      the moment
    <tschwinge> ?
    <nowhereman> I just remembered SICP, the idea of wishful thinking when you
      code, and didn't bother with the fine details behind what I'm reading
    <nowhereman> like, I don't really get what happens when a Mach port is
      allocated, but I know approximately what a Mach port is
    <tschwinge> So originally you worked on investigating all that, every line
      of code?
    <nowhereman> almost, yeah
    <braunr> nowhereman: again, feel free to ask
    <tschwinge> Yes indeed -- that's too complex for a single person to tackle
      at one time.
    <braunr> and quickly
    <braunr> don't loose time
    <tschwinge> Not even braunr and I have looked up all these things.
      (Speaking for Richard here, but I'm quite sure he'll agree.  Perhaps he
      has in fact looked up all the Mach things, though.)
    <tschwinge> nowhereman: ufc?
    <nowhereman> BTW, last week I wanted to push my description of how the tool
      could be used, the use cases
    <nowhereman> ufs
    <nowhereman> but flubber is not online
    <tschwinge> nowhereman: Oh, why ufs specifically?
    <braunr> don't waste time on ufs
    <braunr> really
    <tschwinge> nowhereman: Yes, flubber is down.  But you can push directly to
      the Savannah repository.
    <tschwinge> nowhereman: Please immediatelly tell us if you're stuck on
      something, like flubber not being available.
    <tschwinge> We may not be able to help immediatelly, but we're the at least
      aware of issues.
    <braunr> and we may be able to help immediately :)
    <tschwinge> As we're not sitting in a lab next to each other, we can't tell
      otherwise what's going on.
    <tschwinge> We may in fact even be able to tell you immediatelly to use
      Savannah instead of flubber, indeed.
    <tschwinge> nowhereman: So, back to ufs -- which you don't specifically
      need to look at, I think -- ext2fs is what everyone uses.  But even there
      you shouldn't really need to know many details/internals.
    <nowhereman> OK, I was looking into it has it appears in hurd.boot
    <tschwinge> Ah, OK.  Yeah, that's just an example/template, and should use
      ext2fs nowadays.
    <nowhereman> in fact, as far as FS are concerned, I suppose I will merely
      need to know how to pass a port to the host's FS to some proxy FS in the
      subhurd
    <nowhereman> mmmh, Savannah only mentions a hurd.git
    <tschwinge> Exactly that is the abstraction level you need, yes.
    <nowhereman> I'm looking at http://savannah.gnu.org/git/?group=hurd
    <tschwinge> Yeah, that's a known shortcoming -- look here instead:
      https://git.savannah.gnu.org/cgit/hurd
    <tschwinge> Here is some more up-to-date stuff on subhurds:
      http://www.gnu.org/software/hurd/hurd/subhurd.html
    <tschwinge> nowhereman: You know how to tell git to add a new remote to
      your web pages checkout and such stuff?
    <nowhereman> yeah, no problem with that
    <braunr> have you prepared any question to ask us ?
    <nowhereman> the only I have now is if you can tell me where to look in the
      code about passing Mach ports
    <braunr> you don't pass ports, you pass rights
    <braunr> http://www.gnu.org/software/hurd/gnumach-doc/index.html is the
      best location to have a look at
    <braunr>
      http://www.gnu.org/software/hurd/gnumach-doc/Exchanging-Port-Rights.html#Exchanging-Port-Rights
    <braunr> i suppose the mig doc will help too, as you may be using a higher
      level interface to exchange rights
    <braunr> be careful about user references on port rights
    <braunr> deallocate releases a reference, it doesn't immediately destroy a
      resource
    <braunr> portinfo -v can help monitoring a task's rights
    <braunr> nowhereman: so what are you planning to do now ?
    <braunr> during the next week
    <nowhereman> documenting what I understand from the boot process and where
      things can be changed to fit my various use cases
    <braunr> do you expect that to take the whole week ?
    <nowhereman> and doing some first modifications to servers for the simplest
      cases
    <braunr> ok
    <braunr> well i hope you're able to really start working on it soon, and
      won't face weird issues in the meantime
    <braunr> i'm a bit disappointed that you don't have more questions
    <braunr> my feeling is you either did understand everything (except passing
      port rights), or you didn't attempt to seriously understand the code
    <braunr> or you don't dare ask questions
    <braunr> this is something that must change
    <braunr> or these meetings won't be as useful as they could be
    <tschwinge> Yes.  But also please don't wait for the meetings, but ask
      questions throughout the week, too.


# IRC, freenode, #hurd, 2012-08-09

    <nowhere_man> hey, does anyone knows the network device interface well?
    <nowhere_man> I don't get it by reading net_io.c/h in gnumach
    <braunr> nowhere_man: ask your question
    <braunr> nowhere_man: http://www.sceen.net/~rbraun/pcap-hurd.c <- this may
      help
    <nowhere_man> I don't see what the entry point is
    <nowhere_man> I finally understood that I actually don't need to touch
      pfinet for gsoc project
    <nowhere_man> but I should do a replacement network device instead
    <nowhere_man> is the net_io_init function called at start?
    <braunr> what entry point ?
    <braunr> and you should perhaps have a look at the eth-multiplexer by
      zhengda
    <braunr> yes net_io_init is called at startup
    <braunr> nowhere_man: did you find your answers about networking ?
    <nowhere_man> no, I'm still digging in mach's code
    <braunr> nowhere_man: well keep asking :/
    <braunr> you left conversation without notice :/
    <braunr> nowhere_man: and why mach ?
    <nowhere_man> I thought hardware devices are there
    <tschwinge> nowhere_man: You wanted to push your documentation one/two
      weeks ago.  Why has that not yet happened?
    <youpi> nowhere_man: they used to be there, they are now in netdde, but in
      both case it's just a matter of the same RPC interface
    <nowhere_man> tschwinge: I spent very few time this week on gsoc, and
      completely forgot about the push on savannah
    <braunr> nowhere_man: i told you to look at the work by zhengda concerning
      eth-multiplexer, did you do that ?
    <tschwinge> nowhere_man: You realize GSoC is meant to be a full-time job?
    <tschwinge> Or, next to full-time?
    <braunr> it's full-time normally
    <braunr> the payment is justified by that
    <youpi> nowhere_man: most RPC operations you need to know about network can
      be seen at work in pfinet/ethernet.c, wherever "ether_port" appears
    <youpi> i.e. device_open, set_filter, write, set/get_status
    <braunr> again, http://www.sceen.net/~rbraun/pcap-hurd.c should guide you
      pretty well
    <braunr> since it's the very least necessary to use that interface
    <tschwinge> nowhere_man: How, roughly but realistically, are your plans to
      continue this task?
    <tschwinge> nowhere_man: What has been blocking you this week so you
      couldn't work on your task?
    <nowhere_man> tschwinge: mostly a previous work that was supposed to end at
      the beginning of the summer and only went online now, for which I'm
      basically sysadmin
    <braunr> 21:25 < tschwinge> nowhere_man: How, roughly but realistically,
      are your plans to continue this task?
    <braunr> this question is really more interesting actually
    <nowhere_man> right now, I want to write a netword device that just sends
      its frames by IPC
    <braunr> why ?
    <nowhere_man> as I never wrote any program using Mach's IPC, that seems the
      easiest to get them right
    <braunr> you won't have time
    <braunr> 21:22 < braunr> nowhere_man: i told you to look at the work by
      zhengda concerning eth-multiplexer, did you do that ?
    <nowhere_man> braunr: not yet, no
    <braunr> well that's your best chance to make some progress
    <nowhere_man> braunr: is writing the virtal network device that hard?
    <braunr> basically, it allows "bridgind" the pfinet instances of various
      subhurds
    <braunr> the virtual network device you want *is* eth-multiplexer
    <tschwinge> nowhere_man: GSoC is nearly over.  That's why I'm asking how
      this task is going to continue.  I'm sorry but I reckon you have not
      spend anywhere near the amount of hours that are meant to be spent on it.
    <braunr> and from what antrik told me, yes it's hard, and moreover, why
      rewrite it if it already exists and you're late
    <braunr> i agree
    <nowhere_man> tschwinge: I know, I've started way too late because of my
      second round of exams
    <tschwinge> nowhere_man: OK, that's how you started.  But how is it going
      to continue...
    <nowhere_man> tschwinge: in short, I write a prototype that just starts a
      subhurd, and when that works correctly I add the network
    <tschwinge> nowhere_man: I mean from an organizational point of view.
    <nowhere_man> well, between now and the beginning of september, I'll work
      full-time on this
    <nowhere_man> up until september 8th


# IRC, freenode, #hurd, 2012-08-09

    <antrik> nowhere_man: you do *not* have to do a replacement network
      device. zhengda did that years ago.
    <antrik> nowhere_man: also note that zhengda also implemented the support
      for *using* the virtual network device (in fact any replacement devices
      -- except that no others actually exist yet) in boot
    <youpi> which is already in, actually, isn't it?
    <antrik> youpi: hm, yes... it was the patch that zhengda posted on the list
      once, but later updated, and at some later point you merged the outdated
      variant from the list...
    <youpi> outdated?
    <youpi> ah, but he never posted the updated one, and it got lost in git
      repos, right?
    <youpi> (what was updated actually?)
    <antrik> he changed the option name and description later for more
      clarity. don't remember whether there were other changes
    <antrik>   -f, --device=device_name=device_file
    <antrik>                              Specify a device file used by subhurd
      and its
    <antrik>                              virtual name.
    <antrik> that's the one from the Debian package
    <antrik>   -m, --device-map=DEBICENAME=DEVICEFILE
    <antrik>                              Map the device in subhurd to the
      device in the
    <antrik>                              main Hurd.
    <antrik> that's the one I have locally built from his tree
    <youpi> so you actually have access to his tree?
    <antrik> uhm... I used to... it was on flubber


# IRC, freenode, #hurd, 2012-08-18

    <nowhere_man> so, this week I discovered how fun it is to work on a
      non-mainstream OS
    <nowhere_man> I hoped to start coding the tool itself, put together the
      skeleton, but every Lisp implementation I tried had problems
    <braunr> ah you want to write it in lisp ?
    <nowhere_man> ECL, that I had ported a few years ago, actually FTBFS since
    <nowhere_man> I hoped to be able, it would be easier for me
    <nowhere_man> and when I tried Scheme, I started with Guile (it's GNU's own
      Scheme implementation, after all)
    <nowhere_man> and when I execute the FFI functions, to access functions in
      libmachdec
    <nowhere_man> I get SIGILL
    <braunr> i can't advise you about anything lisp related
    <braunr> the most reliable thing you'll find on the hurd is C
    <nowhere_man> I tried to debug that, but running Guile in GDB gets me a
      SIGSEV
    <nowhere_man> I'll try to make ECL to build again
    <braunr> this seems like a waste of time to me
    <braunr> avoid spending time on anything that isn't directly related to
      your goal if you still hope to finish it
    <nowhere_man> I'm ten times more comfortable coding in Lisp
    <braunr> it doesn't matter, you're late
    <nowhere_man> yeah, I know, so taking the time to correct that problem
      won't change the fact that I won't finish in time
    <nowhere_man> so I'll finish anyway, and in Lisp
    <braunr> and if you lack something else, like some mach/hurd specific lisp
      bindings, you'll have to spend more time on that
    <braunr> ok
    <nowhere_man> do you know if someone had a SIGILL situation on Hurd in the
      past?
    <nowhere_man> I'm wondering if that's a known kind of issue
    <braunr> there are lots of issues
    <braunr> especially when it comes to other languages and runtime
      environments
    <nowhere_man> but is it like MAX_PATH_LEN, something that is known to
      happen when porting something on Hurd?
    <braunr> i'm not sure how comparable it is
    <braunr> i'd say it's often before of the conformance issues of the hurd
    <braunr> because*
    <nowhere_man> like missing bits of POSIX ?
    <braunr> or simple wrong for some corner cases
    <braunr> simply*
    <bubu^> nowhere_man, I was able to run guile on my hurd image through qemu
    <bubu^> but I didn't make any complexe programms to check if everything
      works fine
    <nowhere_man> yeah, it runs fine
    <nowhere_man> FFI functions get you a SIGILL
    <nowhere_man>
      http://www.gnu.org/software/guile/manual/html_node/Dynamic-FFI.html
    <nowhere_man> the define-module form at the beginning triggers the signal
    <antrik> nowhere_man: what do you want to implement in Lisp?
    <antrik> BTW, the guy working on Lisp bindings a couple of years ago used
      Clisp
    <antrik> it was working back then
    <nowhere_man> antrik: the program that sets up a subhurd
    <nowhere_man> I always forget about clisp, I'll try it right away