diff options
author | Thomas Schwinge <thomas@codesourcery.com> | 2013-07-21 15:35:02 -0400 |
---|---|---|
committer | Thomas Schwinge <thomas@codesourcery.com> | 2013-07-21 15:35:02 -0400 |
commit | 9933cec0a18ae2a3d752f269d1bb12c19f51199d (patch) | |
tree | cc30f2d56b87d3896e460a58b76e964231c0d578 /community | |
parent | 65efe654a9cb0b682efa9bf21065469a2e9147f4 (diff) |
IRC.
Diffstat (limited to 'community')
-rw-r--r-- | community/gsoc/2013/hacklu.mdwn | 617 | ||||
-rw-r--r-- | community/gsoc/2013/nlightnfotis.mdwn | 450 | ||||
-rw-r--r-- | community/gsoc/project_ideas/mtab.mdwn | 98 | ||||
-rw-r--r-- | community/gsoc/project_ideas/mtab/discussion.mdwn | 907 |
4 files changed, 1976 insertions, 96 deletions
diff --git a/community/gsoc/2013/hacklu.mdwn b/community/gsoc/2013/hacklu.mdwn new file mode 100644 index 00000000..d0185c60 --- /dev/null +++ b/community/gsoc/2013/hacklu.mdwn @@ -0,0 +1,617 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!toc]] + + +# IRC, freenode, #hurd, 2013-06-23 + + <hacklu> braunr: sorry for the late reply. Honestly to say, the school + works had taken most of my time these days. I haven't got any + siginificant progress now. I am trying to write a little debugger demo on + Hurd. + <hacklu> braunr: things goes more hard than I think, these are some + differences between ptrace() on Hurd and Linux. I am trying to solve + this. + + +# IRC, freenode, #hurd, 2013-06-24 + + <hacklu> this is my weekly report + http://hacklu.com/blog/gsoc-weekly-report1-117/. + <hacklu> and I have two main questions when I read the gdb source code. + <hacklu> 1/What is the S_exception_raise_request()? 2/what is the role of + ptrace in gdb port on Hurd? + <youpi> hacklu: where did you see S_exception_raise_request? + <hacklu> in gdb/gnu-nat.c + <youpi> ah, in gdb + <hacklu> yeah. and I have read the <The hurd hacking guide>. is says the S_ + start means server stub. + <youpi> yes + <youpi> what happens is that gnu_wait keeps calling mach_msg + <youpi> to get a message + <youpi> then it passes that message to the various stubs servers + <youpi> see just below, it calls exc_server, among others + <youpi> and that's exc_server which ends up calling + S_exception_raise_request, if the message is an exception_raise request + <youpi> exc_server is a mere multiplexer, actually + <tschwinge> S_exception_raise_request is the implementation of the request + part (so one half of a typical RPC) of the Mach exception interface. + <tschwinge> See gdb/exc_request.defs in GDB and include/mach/exc.defs in + Mach. + <hacklu> youpi: how gnu_wait pass one message to exc_server? in which + function? + <youpi> in gnu_wait() + <youpi> && !exc_server (&msg.hdr, &reply.hdr) + <hacklu> oh, I see this. + <hacklu> firstly I think it is a type check simply. + <youpi> see the comment: "handle what we got" + <tschwinge> The Hurd's proc server also is involved in the exception + passing protocol (see its source code). + <hacklu> tschwinge: I will check the source code later. is the exception + take place in this way: 1. the inferior call ptrace(TRACE_ME). 2.the gdb + call task_set_exception_port. 3. mach send a notification to the + exception port set before. 4. gdb take some action. + <tschwinge> hacklu: Yes, that's it, roughly. The idea is that GDB replaces + a process' standard exception port, and replaces it "with itself", so + that when the process that is debugged receives and exception (by Mach + sending a exception_raise RPC), GDB can then catch that and act + accordingly. + <tschwinge> hacklu: As for your other questions, about ptrace: As you can + see in [glibc]/sysdeps/mach/hurd/ptrace.c, ptrace on Hurd is simply a + wrapper around vm_read/write and more interfaces. + <tschwinge> hacklu: As the GDB port for Hurd is specific to Hurd by + definition, you can also directly use these calls in GDB for Hurd. + <tschwinge> ..., as it is currently done. + <hacklu> and in detail, the part 3 mach send a notification to the + excetption port is like this: gnu_wait get the message in mach_msg, and + then pass it to exc_serer by exc_server(),then exc_server call + S_exception_raise_request()? ? + <hacklu> tschwinge: yeah, I have see the ptrace.c. I was wonder about + nobody use ptrace in Hurd except TRACEME... + <tschwinge> hacklu: Right about »and in detail, [...]«. + <tschwinge> hacklu: It would be very good (and required for your + understanding anyway), if you could write up a list of things that + happens when a process (both under the control of GDB as well as without + GDB) is sent an exception (due to a breakpoint instruction, for example). + <tschwinge> Let me look something up. + <hacklu> tschwinge: what's the function of exc_server? if I can get the + notification in mach_msg(). + <youpi> to multiplex the message + <youpi> i.e. decoding it, etc. up to calling the S_ function with the + proper parameters + <youpi> exc_server being automatically generated, that saves a lot of code + <tschwinge> That is generated by MIG from the gdb/exc_request.defs file. + <tschwinge> You'll find the generated file in the GDB build directory. + <hacklu> I have wrote down the filenames. after this I will check that. + <tschwinge> hacklu: I suggest you also have a look at the Mach 3 Kernel + Principles book, + <http://www.gnu.org/software/hurd/microkernel/mach/documentation.html>. + <tschwinge> This also has some explanation of the thread/task's exception + mechanism. + <tschwinge> And of course, explains the RPC mechanism, which the exception + mechanism is built upon. + <tschwinge> And then, really make a step-by-step list of what happens; this + should help to better visualize what's going on. + <hacklu> ok. later I will update this list on my blog. + <tschwinge> hacklu: I cannot tell off-hand why GDB on Hurd is using + ptrace(PTRACE_TRACEME) instead of doing these calls manually. I will + have to look that up, too. + <hacklu> tschwinge: thanks. + <tschwinge> hacklu: Anyway -- you're asking sensible questions, so it seems + you're making progress/are on track. :-) + <hacklu> tschwinge: there is something harder than I had thought, I haven't + got any meaningful progress. sorry for this. + <tschwinge> hacklu: That is fine, and was expected. :-) (Also, you're + still busy with university.) + <hacklu> I will show more time and enthusiasm on this. + <tschwinge> hacklu: Oh, and one thing that may be confusing: as you may + have noticed, the names of the same RPC functions are sometimes slightly + different if different *.defs files. What is important is the subsystem + number, such as 2400 in [GDB]/gdb/exc_request.defs (and then incremented + per each routine/simpleroutine/skip directive). + <tschwinge> hacklu: Just for completeness, [hurd]/hurd/subsystems has a + list of RPC subsystems we're using. + <tschwinge> And the name given to routine 2400, for example, is just a + "friendly name" that is then used locally in the code where the *.defs + file has been processed by MIG. + <tschwinge> What a clumsy explanation of mine. But you'll get the idea, I + think. ;-) + <tschwinge> hacklu: And don't worry about your progress -- you're making a + lot of progress already (even if it doesn't look like it, because you're + not writing code), but the time spent on understanding these complex + issues (such as the RPC mechanism) definitely counts as progress, too. + <hacklu> tschwinge: not clearly to got it as I am not sensitive with the + MIG's grammer. But I know, the exc is the routine 2400's alias name? + <tschwinge> hacklu: I'd like to have you spend enough time to understand + these fundamental concepts now, and then switch to "hacking mode" (write + code) later, instead of now writing code but not understanding the + concepts behind it. + <hacklu> I have wrote a bit code to validate my understanding when I read + the soruce code. But the code not run. http://pastebin.com/r3wC5hUp + <tschwinge> The subsystem directive [...]. As well, let me just point you + to the documentation: + <http://www.gnu.org/software/hurd/microkernel/mach/mig/documentation.html>, + MIG - THE MACH INTERFACE GENERATOR, chapter 2.1 Subsystem identification. + <tschwinge> hacklu: Yes, writing such code for testing also is a good + approach. I will have to look at that in more detail, too. + * tschwinge guesses hacklu is probably laughing when seeing the years these + documents were written in (1989, etc.). ;-) + <hacklu> mach_msg make no sense in my code, and the process just hang. kill + -9 can't stop is either. + <braunr> hacklu: do you understand why kill -KILL might not work now ? + <hacklu> braunr: no, but I found I can use gdb to attach to that process, + then quit in gdb, the process quit too. + <hacklu> maybe that process was waiting a resume. + <braunr> something like that yes + <braunr> iirc it's related to a small design choice in the proc server + <braunr> something that put processes in an uninterruptible state when + being debugged + <hacklu> iirc ? + <braunr> if i recall cl=orrectly + <braunr> correctly* + <hacklu> like D status in linux? + <braunr> or T + <braunr> there has been a lot of improvements regarding signal handling in + linux over time so it's not really comparable now + <braunr> but that's the idea + <hacklu> in ps, i see the process STAT is THumx + <braunr> did you see that every process on the hurd has at least two + threads ? + <hacklu> no, but I have see that in hurd, the exception handler can't live + in the same context with the victim. so there must be at least two + threads. I think + <braunr> hacklu: yes + <braunr> that thread also handles regular signals + <braunr> in addition to mach exceptions + <braunr> (there are two levels of multiplexing in servers, first locating + the subsystem, then the server function) + <braunr> hacklu: if what i wrote is confusing, don't hesitate to ask for + clarifications (i really don't intend to make things confusing) + <hacklu> braunr: I don't know what you say about the "multiplexing in + servers". For instance, is it means how to pass message from mach_msg to + exc_server in gnu_wait()? + <braunr> hacklu: i said that the "message thread" handles both mach + exceptions and unix signals + <braunr> hacklu: these are two different interfaces (and in mach terms, + subsystems) + <braunr> hacklu: see hurd/msg.defs for the msg interface (which handles + unix signals) + <braunr> hacklu: to handle multiple interfaces in the same thread, servers + need to first find the right subsystem + <braunr> this is done by subsequently calling all demux functions until one + returns true + <braunr> (finding the right server function is done by these demux + functions) + <braunr> hacklu: see hurd/msgportdemux.c in glibc to see how it's done + there + <braunr> it's short actually, i'll past it here : + <braunr> return (_S_exc_server (inp, outp) || + <braunr> _S_msg_server (inp, outp)); + <braunr> hacklu: did that help ? + <hacklu> braunr: a bit more confusing. one "message thread" handles + exceptions and signals, means the message thread need to recive message + from two port. then pass the message to the right server which handle the + message. the server also should pick the right subsystem from a lot of + subsystems to handle the msg. is this ? + <braunr> the message thread is a server thread + <braunr> (which means every normal process is actually also a server, + receiving exceptions and signals) + <braunr> there may be only two ports, or more, it doesn't matter much, the + port set abstraction takes care of that + <hacklu> so the message thread directly pass the msg to the right + subsystem? + <braunr> not directly as you can see + <braunr> it tries them all until one is able to handle the incoming message + <braunr> i'm not sure it will help you with gdb, but it's important to + understand for a better general understanding of the system + <braunr> ugly sentence + <hacklu> ah, I see. like this in gnu-nat.c if(!notify_server(&msg.hdr, + &reply.hdr) && !exc_server(&msg.hdr...) + <braunr> yes + <hacklu> the thread just ask one by one. + <braunr> be careful about the wording + <braunr> the thread doesn't "send requests" + <braunr> it runs functions + <braunr> (one might be tempted to think there are other worker threads + waiting for a "main thread" to handle demultiplexing messages) + <hacklu> I got it. + <hacklu> the notify_server function is just run in the same context in + "message thread",and there is no RPC here. + <braunr> yes + <hacklu> and the notify_server code is generater by mig automatically. + <braunr> yes + + +# IRC, freenode, #hurd, 2013-06-29 + +[[!tag open_issue_documentation]] + + <hacklu> I just failed to build the demo on + this. http://walfield.org/pub/people/neal/papers/hurd-misc/ipc-hello.c + <hacklu> or, example in machsys.doc called simp_ipc.c + <pinotree> we don't use cthreads anymore, but pthreads + <hacklu> pinotree: em.. and I also failed to find the <servers/env_mgr.h> + in example of <A programmer's guide to MACH system call> + <pinotree> that i don't know + <hacklu> maybe the code in that book out-of-date + <teythoon> hacklu: mig and mach ipc documentation is quite dated + unfortunately, and so are many examples floating around the net + + <hacklu> btw, I have one more question. when I read <Mach 3 kernel + interface>. I find this state: When an exception occurs in a thread, the + thread sends an exception message to + <hacklu> its exception port, blocking in the kernel waiting for the receipt + of a reply. It is + <hacklu> assumed that some task is listening to this + <hacklu> port, using the exc_serverfunction to decode the messages and + then call the + <hacklu> linked in catch_exception_raise. It is the job of + catch_exception_raiseto handle the exception and decide the course of + action for thread. + <hacklu> that says, it assumed another task to recieve the msg send to one + thread's exception port. why another task? + <hacklu> I remmebered, there are at least two threads in one task, one is + handle the exception stuffs. + <braunr> there are various reasons + <braunr> first is, the thread causing the exception is usually not waiting + for a message + <braunr> next, it probably doesn't have all the info needed to resolve the + exception + <braunr> (depending on the system design) + <braunr> and yes, the second thread in every hurd process is the msg + thread, handling both mach exceptions and hurd signals + <hacklu> but in this state, I can't find any thing with the so called msg + thread + <braunr> ? + <hacklu> if exist a task to do the work, why we need this thread? + <braunr> this thread is the "task" + <hacklu> ? + <braunr> the msg thread is the thread handling exceptions for the other + threads in one task + <braunr> wording is important here + <braunr> a task is a collection of resources + <braunr> so i'm only talking about threads really + <braunr> 14:11 < hacklu> assumed that some task is listening to this + <braunr> this is wrong + <braunr> a task can't listen + <braunr> only a thread can + <hacklu> in you words, the two thread is in the same task? + <braunr> yes + <braunr> 14:32 < braunr> and yes, the second thread in every hurd process + is the msg thread, handling both mach exceptions and hurd signals + <braunr> process == task here + <hacklu> yeah, I always think the two thread stay in one task. but I found + that state in <mach 3 kernel interface>. so I confuzed + <hacklu> s/confuzed/confused + <braunr> statement you mean + <hacklu> if two thread stay in the same task. and the main thread throw a + exception, the other thread to handle it? + <braunr> depends on how it's configured + <braunr> the thread receiving the exceptions might not be in the same task + at all + <braunr> on the hurd, only the second thread of a task receives exception + <braunr> s + <hacklu> I just wonder how can the second thread catch the exception from + its containning task + <braunr> forget about tasks + <braunr> tasks are resource containers + <braunr> they don't generate or catch exceptions + <braunr> only threads do + <braunr> for each thread, there is an exception port + <braunr> that is, one receive right, and potentially many send rights + <braunr> the kernel uses a send right to send exceptions + <braunr> the msg thread waits for messages on the receive right + <braunr> that's all + <hacklu> ok. if I divide zero in main thread, the kernel will send a msg to + the main thread's exception port. and then, the second thread(in the same + task) is waiting on that port. so he get the msg. is it right? + <braunr> don't focus on main versus msg thread + <braunr> it applies to all other threads + <braunr> as well + <braunr> otherwise, you're right + <hacklu> ok, just s/main/first + <braunr> no + <braunr> main *and* all others except msg + <hacklu> main *and* all others except msg ? + <braunr> the msg thread gets exception messages for all other threads in + its task + <braunr> (at least, that's how the hurd configures things) + <hacklu> got it. + <hacklu> if the msg thread throw exception either, who server for himself? + <braunr> i'm not sure but i guess it's simply forbidden + <hacklu> i used gdb to attach a little progrom which just contains a divide + zero. and I only found the msg thread is in the glibc. + <braunr> yes + <hacklu> where is the msg thread located in. + <braunr> it's created by glibc + <hacklu> is it glibc/hurd/catch-exc.c? + <braunr> that's the exception handling code, yes + <hacklu> there are some differences between the code and the state in <mach + 3 system interface>. + <braunr> state or statement ? + <hacklu> staement + <braunr> which one ? + <hacklu> http://pastebin.com/ZTBrUAsV + When an exception occurs in a thread, the thread sends an exception + message to + its exception port, blocking in the kernel waiting for the receipt of a + reply. It is + assumed that some task is listening (most likely with mach_msg_server) + to this + port, using the exc_serverfunction to decode the messages and then + call the + linked in catch_exception_raise. It is the job of + catch_exception_raiseto handle the exception and decide the course of + action for thread. The state of the + blocked thread can be examined with thread_get_state. + <braunr> what difference ? + <hacklu> in the code, I can't find things like exc_server,mach_msg_server + <braunr> uh + <braunr> ok it's a little tangled + <braunr> but not that much + <braunr> you found the exception handling code, and now you're looking for + what calls it + <braunr> simple + <braunr> see _hurdsig_fault_init + <hacklu> from that statemnet I thought there are another _task_ do the + exception things for all of the systems thread before you have told me + the task means the msg thread. + <braunr> again + <braunr> 14:47 < braunr> forget about tasks + <braunr> 14:47 < braunr> tasks are resource containers + <braunr> 14:47 < braunr> they don't generate or catch exceptions + <braunr> 14:47 < braunr> only threads do + <hacklu> yeah, I think that document need update. + <braunr> no + <braunr> it's a common misnomer + <braunr> once you're used to mach concepts, the statement is obvious + <hacklu> braunr: so I need read more :) + <hacklu> _hurdsig_fault_init send exceptions for the signal thread to the + proc server? + <hacklu> why come about _proc_ server? + <braunr> no it gives the proc server a send right for signals + <braunr> exceptions are a mach thing, signals are a hurd thing + <braunr> the important part is + <braunr> err = __thread_set_special_port (_hurd_msgport_thread, + <braunr> THREAD_EXCEPTION_PORT, sigexc); + <hacklu> this one set the exception port? + <braunr> yes + <braunr> hm wait + <braunr> actually no, wrong part :) + <braunr> this sets the excpetion port for the msg thread (which i will call + the signal thread as mentioned in glibc) + <hacklu> but the comment above this line, Direct signal thread exceptions + to the proc server means what? + <braunr> that the proc server handles exceptions on the signal thread + <hacklu> the term signal thread equals the term msg thread? + <braunr> yes + <hacklu> so, the proc server handles the exceptions throwed by the msg + thread? + <braunr> looks that way + <hacklu> feels a little strange. + <braunr> why ? + <braunr> this thread isn't supposed to cause exceptions + <braunr> if it does, something is deeply wrong, and something must clean + that task up + <braunr> and the proc server seems to be the most appropriate place from + where to do it + <hacklu> why need a special server to just work the msg thread? I don't + think that thread will throw exception frequentlly + <braunr> what does frequency have to do with anything here ? + <braunr> ok the appropriate code is _hurdsig_init + <braunr> the port for receiving exceptions is _hurd_msgport + <braunr> the body of the signal thread is _hurd_msgport_receive + <hacklu> aha, in the _hurd_msgport_receive I have finally found the + while(1) loop mach_msg_server(). + <hacklu> so the code is conform with the documents. + <hacklu> braunr: [21:18] <braunr> what does frequency have to do with + anything here ? yes, I have totally understood your words now. thank you + very much. + <braunr> :) + + +# IRC, freenode, #hurd, 2013-07-01 + + <hacklu> hi. this is my weekly + report. http://hacklu.com/blog/gsoc-weekly-report2-124/ welcome to any + comment + <hacklu> teythoon: I only get clear about the rpc stuff. seems a lot behind + my plan + <youpi> good progress :) + <hacklu> I have wrote the details of the exception handle which was asked + by tschwing_ last week. Am I all right in my post? + <youpi> hacklu: as far as I understand signals, yes :) + <hacklu> youpi: thanks for god, I am on the right way finally... :) + <hacklu> the mig book says simpleroutine is the one use to implement asyn + RPCs which doesn't expect an reply. But I have found a place to pass an + reply port to the RPC interface which has been declared as simpleroutine + <youpi> hacklu: probably the simpleroutine hardcodes a reply port? + + <youpi> hacklu: about _hurd_internal_post_signal, this is the hairiest part + of GNU/Hurd, signal handling + <youpi> simply because it's the hairiest part of POSIX :) + <youpi> you probably want to just understand that it implements the + POSIXity of signal delivering + <youpi> i.e. deliver/kill/suspend the process as appropriate + <youpi> I don't think you'll need to dive more + <hacklu> aha. + <hacklu> it will save a lot of time. + <hacklu> it seems like the wait_for_inferior() in gdb. which also has too + many lines and too many goto + <youpi> hacklu: btw, which simpleroutine were you talking about ? + <hacklu> I forget where it is, I am finding it now. + <youpi> which version of gdb are you looking the source of? + <youpi> (in mine, wait_for_inferior is only 45 lines long) + <hacklu> I dont know how to pick the verison, I just use the git + version. maybe I give a wrong name. + <youpi> ok + <hacklu> youpi:I remembered, my experience comes from here + http://www.aosabook.org/en/gdb.html. (All of this activity is managed by + wait_for_inferior. Originally this was a simple loop, waiting for the + target to stop and then deciding what to do about it, but as ports to + various systems needed special handling, it grew to a thousand lines, + with goto statements criss-crossing it for poorly understood + <hacklu> reasons.) + <hacklu> youpi: the simpleroutine is gdb/gdb/exc_request.defs + <youpi> so there is indeed an explicit reply port + <hacklu> but simpleroutine is for no-reply use. why use reply port here? + <youpi> AIUI, it's simply a way to make the request asynchronous, but still + permit an answer + <hacklu> ok, I will read the mig book carefully. + <braunr> hacklu: as youpi says + <braunr> a routine can be broken into two simpleroutines + <braunr> that's why some interfaces have interface.defs, + interface_request.defs and interface_reply.defs files + <braunr> nlightnfotis: in mach terminology, a right *is* a capability + <braunr> the only thing mach doesn't easily provide is a way to revoke them + individually + <nlightnfotis> braunr: Right. And ports are associated with the process + server and the kernel right? I mean, from what I have understood, if a + process wants to send a signal to another one, it has to do so via the + ports to that process held by the process server + <nlightnfotis> and it has to establish its identity before doing so, so + that it can be checked if it has the right to send to that port. + <braunr> yes + <nlightnfotis> do process own any ports? or are all their ports associated + with the process server? + <nlightnfotis> *processes + <braunr> mach ports were intended for a lot of different uses + <braunr> but in the hurd, they mostly act as object references + <braunr> the process owning the receive right (one at most per port) + implements the object + <braunr> processes owning send rights invoke methods on the object + <braunr> use portinfo to find out about the rights in a task + <braunr> (process is the unix terminology, task is the mach terminologyà + <braunr> ) + <braunr> i use them almost interchangeably + <nlightnfotis> ahh yes, I remember about the last bit. And mach tasks have + a 1 to 1 association with user level processes (the ones associated with + the process server) + <braunr> the proc server is a bit special because it has to know about all + processes + <braunr> yes + +In context of [[open_issues/libpthread/t/fix_have_kernel_resources]]: + + <braunr> hacklu: if you ever find out about either glibc or the proc server + creating one receive right for each thread, please let me know + + +# IRC, freenode, #hurd, 2013-07-07 + + <hacklu> how fork() goes? + <pinotree> see sysdeps/mach/hurd/fork.c in glibc' sources + <hacklu> when the father has two thread( main thread and the signal thead), + if the father call fork, then the child inmediatelly call exev() to + change the excute file. how many thread in the children? + <hacklu> For instance, the new execute file also have two thread. + <hacklu> will the exev() destroyed two threads and then create two new? + <hacklu> s/exev()/excv() + <hacklu> s/exev()/exec() :) + + <hacklu> what libhurduser-2.13.so does? + <hacklu> where can I find this source? + <pinotree> contains all the client stubs for hurd-specific RPCs + <pinotree> it is generated and built automatically within the glibc build + process + + <hacklu> and what is the "proc" server? + <pinotree> what handles in user spaces the processes + <hacklu> so if I call proc_wait_request(), I will go into the + S_proc_wait_reply? + <hacklu> thanks, I have found that. + + +# IRC, freenode, #hurd, 2013-07-08 + + <hacklu> hi, this is my weekly + report. http://hacklu.com/blog/gsoc-weekly-report3-137/ + <hacklu> this week I have met a lot of obstacles. And I am quite desired to + participate in this meeting. + <tschwinge> hacklu: So from your report, the short version is: you've been + able to figure out how the things work that you were looking at (good!), + and now there are some new open questions that you're working on now. + <tschwinge> hacklu: That sounds good. We can of course try to help with + your open questions, if you're stuck figuring them out on your own. + <hacklu> tschwinge: the most question is: what is the proc server? why need + to call proc_get_reqeust() before the mach_msg()? + <hacklu> and Is there exist any specific running sequence between father + and child task after fork()? And I found the inferior always call the + trace_me() in the same time(the trace me printf always in the same line + of the output log). which I have post in my report. + <tschwinge> hacklu: The fork man-page can provide a high-level answer to + your Q3: »The child process is created with a single thread—the one that + called fork(). The entire virtual address space of the parent is + replicated in the child, including the states of mutexes, condition + variables, and other pthreads objects [...]« + <tschwinge> hacklu: What happens in GNU Hurd is that the signal thread is + also "cloned" (additionally to the thread which called fork), but then it + (the signal thread) is re-started from the beginning. (So this is very + much equivalent to creating a new signal thread.) + <tschwinge> hacklu: Then, upon exec, a new memory image is created/loaded, + replacing the previous one. [glibc]/sysdeps/mach/hurd/execve.c. What + actually happens with the existing thread (in particular, the signal + thread) I don't know off-hand. Then answer is probably found in + [glibc]/hurd/hurdexec.c -- and perhaps some code of the exec server + ([hurd]/exec/). + <hacklu> I have checked the status of my regiter mail to FSF. it says it + had arrived in USA. + <tschwinge> hacklu: OK, good. + <tschwinge> hacklu: This is some basic information about the observer_* + functions is GDB: + http://sourceware.org/gdb/current/onlinedocs/gdbint/Algorithms.html#index-notifications-about-changes-in-internals-57 + »3.10 Observing changes in gdb internals«. + <hacklu> tschwinge: not too clear. I will think this latter. and what is + the proc server? + <teythoon> hacklu: /hurd/proc, maps unix processes to mach threads afaiui + <hacklu> teythoon: question is, the mach_msg() will never return unless I + called proc_wait_request() first. + <teythoon> hacklu: sorry, I've no idea ;) + <hacklu> teythoon: :) + <tschwinge> hacklu: I will have to look into that myself, too; don't know + the answer off-hand. + <tschwinge> hacklu: In your blog you write proc_get_request -- but such a + functions doesn't seems to exist? + <hacklu> tschwinge: s/proc_get_request/proc_wait_request called in + gun_wait() [gnu-nat.c] + <tschwinge> hacklu: Perhaps the wait man-page's description of WUNTRACED + gives a clue: »also return if a child has stopped [...]«. But it also to + me is not yet clear, how this relates to the mach_mag call, and how the + proc server exactly is involved in it. + <tschwinge> I'm reading various source code files. + <tschwinge> At least, I don't undestand why it is required for an exception + to be forwarded. + <hacklu> if I need to read the proc server source code? + <tschwinge> I can see how it to become relevant for the case that GDB has + to be informed that the debugee has exited normally. + <tschwinge> hacklu: Yeah, probably you should spend some time with that, as + it will likely help to get a clearer picture of the situation, and is + relevant for other interactions in GDB, too. + <tschwinge> hacklu: By the way, if you find that pieces of the GDB source + code (especially the Hurd files of it) are insufficiently documented, + it's a very good idea, once you have figured out something, to add more + source code comments to the existing code. Or writed these down + separately, if that is easier. + <hacklu> which is the proc server? hurd/exec ? + <hacklu> that ok, I already comment things on my notes. + <tschwinge> hacklu: [Hurd]/proc/ + <tschwinge> hacklu: And [Hurd]/hurd/process*.defs + <hacklu> got it + <tschwinge> hacklu: I'll have to experiment a bit with your HDebugger + example, but I'm out of time right now, sorry. Will continue later. + <hacklu> tschwinge: yep, the HDebugger has a problem, if you put the + sleep() after the printf in the just_print(), thing will hang. + <hacklu> tschwinge: and I am a little curious about how do you find my + code? I dont't remember I have mentioned that :) + <hacklu> tschwinge: I have post my gihub link in the last week report, I + found that. + <tschwinge> hacklu: That's how I found it, yes. + <hacklu> tschwinge: :) diff --git a/community/gsoc/2013/nlightnfotis.mdwn b/community/gsoc/2013/nlightnfotis.mdwn new file mode 100644 index 00000000..43f9b14c --- /dev/null +++ b/community/gsoc/2013/nlightnfotis.mdwn @@ -0,0 +1,450 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!toc]] + + +# IRC, freenode, #hurd, 2013-06-29 + + <teythoon> so, how is your golang port going? + <nlightnfotis> I just started working on it. I had been reading + documentation so far. Maybe over reading as people told me when I asked + for their feedback + <nlightnfotis> but I will report on what I have done (technically tomorrow, + and post it in the mailing list too. + + <nlightnfotis> Hey guys, what could possibly cause the following error + message when executing a program in the Hurd? "./dumper: Could not open + note: (system server) error with unknown subsystem" + <nlightnfotis> My program is one that opens a file and dumps it into stdout + <nlightnfotis> pinotree: the code I am using is the one present here + http://www.gnu.org/software/hurd/hacking-guide/hhg.html under paragraph + 6.1 + <nlightnfotis> I investigated it a bit but can not find a lead. I seem to + have all the rights to open the file that I want to dump to stdout + <pinotree> what if you reset errno to 0 just after all the declarations in + main, before the instructions? + <nlightnfotis> will check this out and get back to you. + <pinotree> sure :) + <nlightnfotis> pinotree: Now it suggests that it can't get the number of + readable files, which the source suggests that is normal behavior. + Thanks for your assistance. + + +# IRC, freenode, #hurd, 2013-07-01 + + <nlightnfotis> youpi: from my part I can report that I have started working + with the code, and doing as Thomas suggested. I was about to write my + report yesterday, but I am facing some build errors on the HURD, which I + would like to investigate further before I write my report. + <nlightnfotis> that's why I decided to write it later in the day. + <youpi> I don't think you have to wait + <youpi> you can simply write in your report that you are having build + errors + <nlightnfotis> ok. I will have it written and delivered later in the day. + <nlightnfotis> braunr: that's cool. I think my reading has paid for + itself. And you may be pleased to know that I have gotten my hands dirty + with the code. I was about to write report yesterday, but some build + errors with the gcc (that I am investigating atm) are holding me + off. Will have that written later in the day. + <braunr> don't hesitate to ask help about build errors + <braunr> don't wait too much + <braunr> you need to progress on what matters, and not be blocked by + secondary problems + <nlightnfotis> I will see myself asking for help rather sooner than later, + but I would like to investigate it myself, and attempt to solve the + issues that occur to me before resort to bugging you guys. + <braunr> sure + <braunr> just not too long + <braunr> too long being a day or so + <nlightnfotis> these were my build_results on the hurd + <nlightnfotis> they were linker errors + <nlightnfotis> + https://gist.github.com/NlightNFotis/5896188#file-build_results + <nlightnfotis> I am trying to build gcc on a linux 32 bit environment. It + also has some issues but not linker errors + <nlightnfotis> will resolve them to see if the linker errors are + reproducible on linux + <braunr> oh, lex stuff + <braunr> should be easy enough + + +# IRC, freenode, #hurd, 2013-07-05 + + <nlightnfotis> I have not made much progress, but I see myself working with + it. + <nlightnfotis> I have managed to build gcc go on Linux + <nlightnfotis> but Hurd seems to have some issues + <nlightnfotis> it seems to randomly crash + <teythoon> the build process? + <nlightnfotis> not quite randomly it seems to be though + <nlightnfotis> yeah + <nlightnfotis> I have noticed that there is a pattern + <nlightnfotis> it does crash after some time + <teythoon> ^^ + <nlightnfotis> but it doesn't crash at specific files + <braunr> define crash + <nlightnfotis> at some times it may crash during compiling insn-emit.c + <braunr> (hello guys) + <teythoon> hi braunr :) + <nlightnfotis> braunr: hey there! It does seem to keep on compiling this + file for a very long time (I have let it do so for 10, 20, 30 minutes) + but the result is the same + <nlightnfotis> and it does so for different files for different build + options + <braunr> ok so it doesn't crash + <braunr> it just doesn't complete + <braunr> is the virtual machine eating 100% cpu during that time ? + <nlightnfotis> I can still type at the terminal, but I can't send a term + signal + <nlightnfotis> I can report that QEMU does hold 100% of one core at that + time, (like it keeps processing) but there is no output on the terminal + <braunr> ok + <nlightnfotis> of course I can type at the terminal + <nlightnfotis> but nothing happens + <braunr> any idea of the size of the files involved ? + <nlightnfotis> I am checking it out right now + <nlightnfotis> before this goes any further, let me report on my + investigation + <braunr> i expect that to be our classic writeback thread storm issue + <nlightnfotis> initially, I thought it might be that it run out of memory + <nlightnfotis> even though I know that compilation is not memory intensive, + rather, cpu intensive + <nlightnfotis> anyway I increased the size of ram available to the vm + <nlightnfotis> from 1024 mb to 1536 + <nlightnfotis> that didn't seem to have any effect. The "crash" still + happens at the same time, at the same files + <braunr> use freeze + <braunr> not crash + <braunr> crash is very misleading here + <nlightnfotis> freeze it is then. + <nlightnfotis> anyway + <nlightnfotis> then it striked me that it might be that the hard disk size + (3gb) might be too small (considering the gcc git repo is 1gb+) + <nlightnfotis> so I resized the qemu image to 8gb of hdd size + <nlightnfotis> the new size is acknowledged by the vm + <pinotree> for gcc in debug mode? might still not be enough + <nlightnfotis> but still it has no effect - it seems to follow its freezing + patterns + <pinotree> giving your work, i'd have not less than 15-20 + <braunr> i'd use 32 + <pinotree> *given + <braunr> but that's because i like power of twos + <nlightnfotis> pinotree: thanks for the advice. Right now I was gonna + increase the swap size + <nlightnfotis> according to vmstat in the hurd + <nlightnfotis> swap size is 173 mb + <nlightnfotis> don't know if it does have an impact + <braunr> it may but before rushing + <braunr> if you need swap, you're doomed anyway + <braunr> consider swap highly unreliable on the hurd + <braunr> please show the output of df -h on the file system you're using to + build + <braunr> ideally, i'd recommend using separate / and /home file systems + <braunr> it really improves reliability + <nlightnfotis> I don't think it swaps to be honest; however that's + something that my mentor thomas had suggested (increasing swap size) so I + am gonna try it at some time. + <pinotree> or have a separate file system in a subdi and work on it + <braunr> yes, /home or whatever suits you + <braunr> just not / + <nlightnfotis> braunr: pinotree: thanks both for your advice. Will do now, + and report on the results. + <braunr> that's not all + <braunr> 11:17 < braunr> please show the output of df -h on the file system + you're using to build + <nlightnfotis> braunr: I am on it. Oh and btw, everytime I am forced to + close the vm (due to the freezes) when I restart it ext2 reports that the + file system was not cleanly unmounted and does some repair to some + files. I am trying to find an explanation for that, but I can think of + many things + <braunr> well obviously + <pinotree> ext2 has no journaling + <braunr> the file system was not cleanly unmounted since you restarted it + with a cold reset + <nlightnfotis> braunr: df -h comes out with this: "df: cannot read table of + mounted file systems" + <pinotree> also, even if you manage to always shut down correctly, when + fsck runs because of the maximum mount count it'd find errors anyway (so + we have some bug) + <braunr> nlightnfotis: df -h /path/to/build/dir + <braunr> pinotree: not really bugs but it could be cleaned up + <nlightnfotis> filesystem: - Size 2.8G Used 2.8G Avail 0 Use% 100% Mounted + on / + <nlightnfotis> wow + <braunr> nlightnfotis: see + <nlightnfotis> that seems to explain many things + <teythoon> ^^ + <nlightnfotis> thanks for that braunr! + <braunr> you resized the disk, but not the partition and the file system + <pinotree> braunr: well, if something in ext2 (or its libs) leaves issues + in the fs, i'd call that a bug :> + <nlightnfotis> yeah, that was utterly stupid of me + <braunr> pinotree: they're not issues + <braunr> nlightnfotis: be careful, mach needs a reboot every time you + change a partition table + <teythoon> nlightnfotis: important thing is that you found the issue :) + <braunr> then only, you can use resize2fs + <teythoon> braunr: weird, I thought mach nowadays can reload the partition + tables? + <teythoon> braunr: doesn't d-i need that? + <braunr> maybe a recent change i forgot + <braunr> or maybe fdisk still reports the error although it's fine + <braunr> in doubt, rebooting is still safe :p + <teythoon> or maybe youpi hacked it into d-is gnumach + <braunr> i doubt it would be there for the installer only :) + <braunr> if it's there, it's there + <braunr> i just don't know it + <nlightnfotis> braunr: teythoon: and everyone else that helped me. Thanks + you all guys. This was something that was driving me crazy. Will do all + that you suggested and report back on my status + + +# IRC, freenode, #hurd, 2013-07-08 + + <nlightnfotis> tschwinge, I have managed to overcome most of the obstacles + I had initially faced with my project + <nlightnfotis> but I still had some build errors, that's why I have not + reported yet. Wanna try to see if I can resolve them today, and write my + report in the afternoon. + <tschwinge> nlightnfotis: So, from a quick look into the IRC backlog, it + was a "simple" out of disk space problem? %-) That happens. + <tschwinge> nlightnfotis: And yes, GCC needs a lot of disk space. + <tschwinge> nlightnfotis: What kind of build errors are you seeing now? + <nlightnfotis> tschwinge, yeah I felt stupid at the time, but it didn't + actually strike me that the file system didn't see the extra space. Also + it took me some time to figure out that in order to mount the new + partition, I only had to edit /etc/fstab + <nlightnfotis> always tried to mount it with the ext2 translator + <nlightnfotis> and the translator kept dying + <nlightnfotis> but it's all figured out now + <nlightnfotis> the latest build errors I am seeing are these + <teythoon> nlightnfotis: o_O you used fstab and it worked? + <nlightnfotis> yeah + <teythoon> nlightnfotis: that's unexpected from my perspective... + <nlightnfotis> I only had to add the new partition into fstab + <nlightnfotis> teythoon: I can pastebin my fstab if you wanna take a look + at it + <nlightnfotis> tschwinge: these were my latest build errors + https://www.dropbox.com/s/b0pssdnfa22ajbp/build_results + <teythoon> nlightnfotis: I'm pretty sure that mount -a isn't done on hurd + w/o pinos runsystem.sysv + <teythoon> weird + <nlightnfotis> tschwinge: I have also tried to build gcc with "make -w" + which from what I know supresses the errors that stopped compilation + <nlightnfotis> but the weird thing is that gcc nearly took forever to build + <teythoon> nlightnfotis: could you do a showtrans /your/mountpoint? + <nlightnfotis> teythoon: /hurd/ext2fs /dev/hd0s3 + <teythoon> nlightnfotis: ok, so you've set a passive translator and an + active is started on demand + <nlightnfotis> it must be a passive translator + <teythoon> nlightnfotis: this is the hurd way of doing things, fstab is + unrelated + <nlightnfotis> it seems to persist during reboots + <teythoon> yes, exactly + <nlightnfotis> teythoon: my fstab if you wanna take a look + http://pastebin.com/ef94JPhG + <nlightnfotis> after I added /dev/hd0s3 to fstab along with its mountpoint, + and restarting the hurd, only then I did manage to use that partition + <nlightnfotis> before doing so I tried pretty much anything involving + mounting the partition and setting the ext2fs translator for it, but it + kept dying + <nlightnfotis> of course it was a ext2 filesystem + <youpi> err, perhaps adding to fstab simply triggered an fsck at reboot? + <teythoon> nlightnfotis: might have been that you needed to reboot mach so + that it picks up the new partition table + <teythoon> youpi: I thought this was fixed, the partition reloading I mean? + <youpi> that is needed, yes + <youpi> let me check + <nlightnfotis> youpi: it could be, though, to be honest, my hurd system + does an fsck all the time at boot + <teythoon> how do you manage to do that w/o rebooting for d-i? + <youpi> (I don't remember whether device busy is detected) + <youpi> teythoon: by making all translators go away, iirc + <teythoon> nlightnfotis: btw, you have ~/gcc_new as mountpoint in your + fstab, pretty sure that this cannot work, the path has to be absolute and + no ~ expansion is done + <nlightnfotis> tbh it does work, and it's weird + <teythoon> nlightnfotis: it works b/c of the passive translator you set, + not b/c of the fstab entry + <nlightnfotis> teythoon: should I change it? + <teythoon> probably, yes + <tschwinge> Well, that is probably not used anywhere. + <teythoon> tschwinge: not yet but soon ;) + <tschwinge> Isn't /etc/fstab only consulted for fsck. + <youpi> atm yes + <tschwinge> Anyway, it is definitely a very good idea to have a partition + separate from the rootfs for doing actual work. + <tschwinge> I think I described that in one of the first GSoC coodridation + emails. In the long one. + <nlightnfotis> teythoon: Oh it struck me now! Is it because tilde expansion + is only happening in bash, but /etc/fstab is read before bash is + initialized? + <tschwinge> nlightnfotis: Instead of fumbling around with partitioning of + disk images, it may be easier in your KVM/QEMU setup to simply add a new + disk using -hdb [file] (or similar). + <tschwinge> nlightnfotis: Basically, yes. + <youpi> nlightnfotis: fstab is not related with bash in any way + <nlightnfotis> anyway, it shouldn't matter now, it seems to be working, and + I wouldn't like fiddling around with it and messing it up now. I will + continue with resolving the gcc issues. + <tschwinge> But /etc/fstab has its very own "language" (layout), so tilde + expansion will never be done there. + <tschwinge> nlightnfotis: df -h ~/gcc_new/ + <nlightnfotis> tschwinge: size 24G Used: 4.2G Avail 18G + <tschwinge> OK, that's fine. + <tschwinge> As you can see on + <http://darnassus.sceen.net/~hurd-web/open_issues/gcc/#index4h1>, GCC + will easily need some GiB. + <nlightnfotis> tschwinge: I have some questions about GCC: out of curiosity + how much time does it take to compile it on your machine? Because + yesterday I tried a -w (suppress warnings) build and it seemed to take + forever + <nlightnfotis> mind you the vm has 1536 ram available (I have read + somewhere that it can utilise such an amount) and the vm is KVM enabled + <youpi> without disabling g++, it can easily take hours + <tschwinge> nlightnfotis: The build error is unexpected, because I had + addressed that issue in a recent patch. :-) + <tschwinge> nlightnfotis: This is wrong: »checking whether setcontext + clobbers TLS variables... [...] yes«. Please check your sources, that + they correspond to the current version of the upstream + tschwinge/t/hurd/go branch. + <tschwinge> nlightnfotis: Quoting from that wiki page: »This takes up + around 3.5 GiB, and needs roughly 3.5 h on kepler.SCHWINGE and 15 h on + coulomb.SCHWINGE.« The latter is my Hurd machine. + <tschwinge> That's however with Java and Ada enabled, and a full + three-stages bootstrap. + <youpi> ah, right, there's java & ada too + <nlightnfotis> tschwinge: git branch (in the repo): master, + *tschwinge/t/hurd/go + <youpi> in debian they are built separately + <tschwinge> What I asked you to do is configure »--disable-bootstrap + --enable-languages=go«. + <tschwinge> So that should be a lot quicker. + <nlightnfotis> tschwinge: oh yes, everytime I have tried to compile gcc I + have done with these configurations + <tschwinge> But still a few hours perhaps. + <nlightnfotis> that's what I did yesterday too. + <tschwinge> OK, good. :-) + <tschwinge> A bootstrap build is a good way to check the just-built GCC for + sanity, but we expect that it is fine, as we concentrate on the GCC Go + port. + <nlightnfotis> the only "extra" configuration yesterday was my "-w" flag to + make, because those errors were actually triggered by -Werror + <tschwinge> Let me read up what make -w does. ;-) + <nlightnfotis> ah, yes, d/w I have read and understood what the bootstrap + build is. Seems like we don't need it atm + <nlightnfotis> afaik it suppresses all warnings + <pinotree> youpi: gcj no more + <nlightnfotis> the way gcc builds, it does convert (some) warnings to + errors + <tschwinge> Hmm. -w, --print-directory Print a message containing the + working directory before and after other processing. + <pinotree> youpi: doko folded gcj and gdc into gcc-4.8 to "workaround" + Built-Using + <tschwinge> nlightnfotis: Ah, that'S configure --enable-werror or something + like that. + <youpi> pinotree: right + <nlightnfotis> yep, and -w suppresses it + <nlightnfotis> (from what I have understood) + <tschwinge> nlightnfotis: Are you thinking about make -k? + <tschwinge> Yeah, I guess. + <nlightnfotis> let me see what -k does + <pinotree> youpi: (just to make builds even more lightweight, eh</irony>) + <nlightnfotis> yeah, -k should do too, I shall try it + <tschwinge> But: if gcc -Werror fails, even with make -k, the build will + not be able to come to a successful end, because that one complation + artefact that failed will be missing. + <nlightnfotis> so I shall try again with -w (supressed warnings) + <tschwinge> Configureing with --disable-werror (or similar) will "help" if + -Werror is the default, and the build fails due to that. + <nlightnfotis> from what I have understood these "errors" are not something + critical: it's only that function prototypes for these functions are + missing + <nlightnfotis> I have seen the code there, and even "default" gcc generated + prototypes (from the first usage of the function) should do, so I can't + understand why it might be a serious problem if I tell gcc to skip that + point + <tschwinge> nlightnfotis: Ah, now I see. You don't mean make -w, but + rather gcc -w: »-w Inhibit all warning messages.« + <tschwinge> But really, there shouldn't be such warnings/errors that make + the build fail. + <nlightnfotis> yeah + <tschwinge> nlightnfotis: In your GCC sources directory, what does this + tell: git rev-parse HEAD + <tschwinge> And, is the checkout clean: git status + <tschwinge> The latter will take some time. + <nlightnfotis> git status takes an awful amount of time + <nlightnfotis> last I checked + <nlightnfotis> but git rev-parse HEAD + <nlightnfotis> produces this result: + <nlightnfotis> 91840dfb3942a8d241cc4f0e573e5a9956011532 + <tschwinge> OK, that's correct. So probably some of the checked out files + are not in a pristine state? + <nlightnfotis> I shall run a git clean and see. If that doesn't work too, + maybe I shall reclone the repository? + <nlightnfotis> there's nothing foreign to the repo that I have added, only + lib gmp, lib mpc and lib mpfr (and they are in their own folders inside + my gcc working directory) + <tschwinge> nlightnfotis: You shouldn't need to do the latter if you + instead run: apt-get build-dep gcc-4.8 + <nlightnfotis> I remember having done that inside the Hurd, but it always + resulted in an error from what I can recall + <nlightnfotis> let me check this out + <nlightnfotis> yes + <tschwinge> nlightnfotis: Whenever you use Git on Hurd, pass the --quiet + flag, to avoid the rare but possible corruption issue described on + <http://darnassus.sceen.net/~hurd-web/open_issues/git_duplicated_content/> + and <http://darnassus.sceen.net/~hurd-web/open_issues/git-core-2/>. + <nlightnfotis> tschwinge: Forgive me for that. I will set up an alias + immediately. + <tschwinge> nlightnfotis: I don't know if an alias is possible, because -- + I think -- you'll need to do things like: git fetch --quiet + <tschwinge> So pass --quiet to subcommands. + <nlightnfotis> oh. ok. + <tschwinge> nlightnfotis: What you can also do, is shut down your Hurd VM, + and mount the disk image on GNU/Linux (mount with offset to get the right + partition), and then run a diff -ru against a Git clone done on + GNU/Linux, and see whether there are any unexpected differences outside + of the .git/ directory. + <nlightnfotis> sounds like a plan. I will check this out today then :) + <nlightnfotis> tschwinge: if all else fails, then recloning the repo with + --quiet passed should work, right? + <tschwinge> Yes, that's probably the most straight-forward check to do. + <tschwinge> Heh, yes to both these questions. :-) + <tschwinge> nlightnfotis: Oh, you don't even have to re-clone, but rather + re-check-out the branch. + <nlightnfotis> I was thinking of recloning just to bring the whole + repository to a pristine state + <tschwinge> So something like (inside the source directory): rm -rf ./* + (remove any files, but leave .* in place, in particular the .git/ + directory), followd by git checkout -f HEAD --quiet + <tschwinge> nlightnfotis: But before doing that, please do the diff first, + so that we know (hopefully) where the erroneous build results were coming + from. + <nlightnfotis> considering the Copyright assignment files, I have sent them + from day 1 (that is the 20th of June). I have not heard anything about + those documents to date (sadly) + <nlightnfotis> what's worst is that although I have a reference number to + track those documents, their (greek postal office) tracking service sucks + so badly, that one day it's offline, the next it suggests it can't find + the object in their database, the next it says it is still in the local + post office + <nlightnfotis> let me check it out now + <nlightnfotis> still nothing from their online service + <nlightnfotis> let me call them + <nlightnfotis> tschwinge: I called the post office regarding the copyright + papers. They told me that the same day (the 20th of June) it left from + Herakleion, Crete to Athens and the same day it must have left the + country heading towards the US. They also told me it takes about 1 week + for it to arrive. + <tschwinge> nlightnfotis: OK, so probably waiting at the FSF office to be + processed. Let's allow for some more time. After all, this is not + critical for your progress. diff --git a/community/gsoc/project_ideas/mtab.mdwn b/community/gsoc/project_ideas/mtab.mdwn index d6f04385..0c0bb87b 100644 --- a/community/gsoc/project_ideas/mtab.mdwn +++ b/community/gsoc/project_ideas/mtab.mdwn @@ -11,6 +11,8 @@ License|/fdl]]."]]"""]] [[!meta title="mtab"]] +[[!tag open_issue_hurd]] + In traditional monolithic system, the kernel keeps track of all mounts; the information is available through `/proc/mounts` (on Linux at least), and in a very similar form in `/etc/mtab`. @@ -73,99 +75,3 @@ Possible mentors: Olaf Buddenhagen (antrik), Carl Fredrik Hammar (cfhammar) Exercise: Make some improvement to any of the existing Hurd translators. Especially those in [hurdextras](http://www.nongnu.org/hurdextras/) are often quite rudimentary, and it shouldn't be hard to find something to improve. - - -# Related Discussion - -## IRC, freenode, #hurd, 2013-04-17 - - <kuldeepdhaka> thinking how to get the listing. traversing would be - ineffecient, trying to come up with something better - <braunr> what listing ? - <braunr> and traversing what ? - <kuldeepdhaka> mtab - <braunr> well i assumed so - <braunr> be more precise please - <kuldeepdhaka> when the translator is done initalized <translation - info> are written to /etc/mtab <translation info> will be provided - by the translator, and when some one want to read the info just read it - this way if their is some credentials like ftp sites pass username can be - masked by the translator - <kuldeepdhaka> if some trans dont want to list them, no need to write to - file | while unmounting (sorry i couldnt find the right word) , it - will pass the mount node address | <translation info> will have special - structure to remove/add mounts example "a /mount-to /mount-from" = add - , "r /mount-to" = remove here "/mount-to" will be unique for every - mount - <kuldeepdhaka> this have a draw back , we would have to trust trans for the - listed data | also "/mount-to" + "/mount-from" could be used a - combination for making sure that other trans unable remove others trans - mount data - <kuldeepdhaka> sorry but "also "/mount-to" + "/mount-from" could be used a - combination for making sure that other trans unable remove others trans - mount data" this is a bad idea if we had to print the whole thing - <kuldeepdhaka> braunr, whats ur opinion? - <pinotree> you don't need a mtab to "unmount" things on hurd - <braunr> kuldeepdhaka: hum, have you read the project idea ? - <braunr> - http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/mtab/ - <braunr> A more promising approach is to have mtab exported by a special - translator, which gathers the necessary information on demand. This could - work by traversing the tree of translators, asking each one for mount - points attached to it. - <kuldeepdhaka> pinotree, not to unmount, i mean is to remove the - <translation data> - <braunr> for a first implementation, i'd suggest a recursive traversal of - root-owned translators - <kuldeepdhaka> braunr, hum, but it did stated it as inefficient - <braunr> where ? - <kuldeepdhaka> para 5 , line 3 - <kuldeepdhaka> and line 6 - <braunr> no - <braunr> traversing "all" nodes would be inefficient - <braunr> translators which host the nodes of other translators could - maintain a simple list of active translators - <braunr> ext2fs, e.g. (if that's not already the case) could keep the list - of the translators it started - <braunr> we can already see that list with pstree for example - <braunr> but this new list would only retain those relevant for mtab - <braunr> i.e. root-owned ones - <pinotree> i would not limit to those though - <braunr> and then filter on their type (e.g. file system ones) - <braunr> pinotree: why ? - <pinotree> this way you could have proper per-user /proc/$pid/mounts info - <braunr> we could also very easily have a denial of service - <kuldeepdhaka> but how will the mount point and source point will be - listed? - <braunr> they're returned by the translator - <kuldeepdhaka> k - <braunr> you ask /, it returns its store and its options, and asks its - children recursively - <braunr> a /home translator would return its store and its options - <braunr> etc.. - <braunr> each translator would build the complete path before returning it - <braunr> sort of, it's very basic - <braunr> but that would be a very hurdish way to do it - <kuldeepdhaka> shall /etc/mtab should be made seek-able and what should be - the filesize? content are generated on demand so, it could arise problem - (fsize:0 , seek-able:no), ur opinions? - <braunr> kuldeepdhaka: it should have all the properties of a regular file - <braunr> the filesize would be determined after it's generated - <braunr> being empty doesn't imply it's not seekable - <kuldeepdhaka> content is generated on demand so, could cause problem while - seeking and filesize, shall i still program as regular file? - <kuldeepdhaka> in two different read, it could generate different content, - though same seek pos is used... - <braunr> what ? - <braunr> the content is generated on open - <kuldeepdhaka> ooh, ok - - -## IRC, freenode, #hurd, 2013-06-04 - - <safinaskar> how to see list of all connected translators? - <braunr> you can't directly - <braunr> you can use ps to list processes and guess which are translators - <braunr> (e.g. everything starting with /hurd/) - <braunr> a recursive call to obtain such a list would be useful - <braunr> similar to what's needed to implement /proc/mounts diff --git a/community/gsoc/project_ideas/mtab/discussion.mdwn b/community/gsoc/project_ideas/mtab/discussion.mdwn new file mode 100644 index 00000000..0e322c11 --- /dev/null +++ b/community/gsoc/project_ideas/mtab/discussion.mdwn @@ -0,0 +1,907 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +# IRC, freenode, #hurd, 2013-04-17 + + <kuldeepdhaka> thinking how to get the listing. traversing would be + ineffecient, trying to come up with something better + <braunr> what listing ? + <braunr> and traversing what ? + <kuldeepdhaka> mtab + <braunr> well i assumed so + <braunr> be more precise please + <kuldeepdhaka> when the translator is done initalized <translation + info> are written to /etc/mtab <translation info> will be provided + by the translator, and when some one want to read the info just read it + this way if their is some credentials like ftp sites pass username can be + masked by the translator + <kuldeepdhaka> if some trans dont want to list them, no need to write to + file | while unmounting (sorry i couldnt find the right word) , it + will pass the mount node address | <translation info> will have special + structure to remove/add mounts example "a /mount-to /mount-from" = add + , "r /mount-to" = remove here "/mount-to" will be unique for every + mount + <kuldeepdhaka> this have a draw back , we would have to trust trans for the + listed data | also "/mount-to" + "/mount-from" could be used a + combination for making sure that other trans unable remove others trans + mount data + <kuldeepdhaka> sorry but "also "/mount-to" + "/mount-from" could be used a + combination for making sure that other trans unable remove others trans + mount data" this is a bad idea if we had to print the whole thing + <kuldeepdhaka> braunr, whats ur opinion? + <pinotree> you don't need a mtab to "unmount" things on hurd + <braunr> kuldeepdhaka: hum, have you read the project idea ? + <braunr> + http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/mtab/ + <braunr> A more promising approach is to have mtab exported by a special + translator, which gathers the necessary information on demand. This could + work by traversing the tree of translators, asking each one for mount + points attached to it. + <kuldeepdhaka> pinotree, not to unmount, i mean is to remove the + <translation data> + <braunr> for a first implementation, i'd suggest a recursive traversal of + root-owned translators + <kuldeepdhaka> braunr, hum, but it did stated it as inefficient + <braunr> where ? + <kuldeepdhaka> para 5 , line 3 + <kuldeepdhaka> and line 6 + <braunr> no + <braunr> traversing "all" nodes would be inefficient + <braunr> translators which host the nodes of other translators could + maintain a simple list of active translators + <braunr> ext2fs, e.g. (if that's not already the case) could keep the list + of the translators it started + <braunr> we can already see that list with pstree for example + <braunr> but this new list would only retain those relevant for mtab + <braunr> i.e. root-owned ones + <pinotree> i would not limit to those though + <braunr> and then filter on their type (e.g. file system ones) + <braunr> pinotree: why ? + <pinotree> this way you could have proper per-user /proc/$pid/mounts info + <braunr> we could also very easily have a denial of service + <kuldeepdhaka> but how will the mount point and source point will be + listed? + <braunr> they're returned by the translator + <kuldeepdhaka> k + <braunr> you ask /, it returns its store and its options, and asks its + children recursively + <braunr> a /home translator would return its store and its options + <braunr> etc.. + <braunr> each translator would build the complete path before returning it + <braunr> sort of, it's very basic + <braunr> but that would be a very hurdish way to do it + <kuldeepdhaka> shall /etc/mtab should be made seek-able and what should be + the filesize? content are generated on demand so, it could arise problem + (fsize:0 , seek-able:no), ur opinions? + <braunr> kuldeepdhaka: it should have all the properties of a regular file + <braunr> the filesize would be determined after it's generated + <braunr> being empty doesn't imply it's not seekable + <kuldeepdhaka> content is generated on demand so, could cause problem while + seeking and filesize, shall i still program as regular file? + <kuldeepdhaka> in two different read, it could generate different content, + though same seek pos is used... + <braunr> what ? + <braunr> the content is generated on open + <kuldeepdhaka> ooh, ok + + +# IRC, freenode, #hurd, 2013-06-04 + + <safinaskar> how to see list of all connected translators? + <braunr> you can't directly + <braunr> you can use ps to list processes and guess which are translators + <braunr> (e.g. everything starting with /hurd/) + <braunr> a recursive call to obtain such a list would be useful + <braunr> similar to what's needed to implement /proc/mounts + + +# IRC, freenode, #hurd, 2013-06-25 + +In context of [[microkernel/mach/mig/documentation/structured_data]]. + + <teythoon> should I go for an iterator like interface instead? + <teythoon> btw, what's the expected roundtrip time? + <braunr> don't think that way + <braunr> consider the round trip delay as varying + <teythoon> y, is it that bad? + <braunr> no + <braunr> but the less there is the better + <braunr> we think the same with system calls even if they're faster + <braunr> the delay itself isn't the real issue + <braunr> look at how proc provides information + <braunr> (in procfs for example) + + +## IRC, freenode, #hurd, 2013-06-26 + + <teythoon> so tell me about the more hurdish way of dealing with that issue + <teythoon> creating a specialized translator for this? + <braunr> 11:45 < pinotree> there's also + http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/mtab/ + about that topic + <braunr> you need to avoid thinking with centralization in mind + <braunr> the hurd is a distributed system in practice + <braunr> i think proc is the only centralized component in there + <teythoon> braunr: would having an mtab translator and having fs + translators register to thae be acceptable? + <teythoon> that* + <braunr> teythoon: why do you want to centralize it ? + <braunr> translators already register themselves when they get attached to + a node + <braunr> we don't want an additional registration + <braunr> have you read the link we gave you ? + <teythoon> I did and I got the message, but isn't the concept of + /proc/mounts or a mtab file already a centralized one? + <braunr> that doesn't mean the implementation has to be + <braunr> and no, i don't think it's centralized actually + <braunr> it's just a file + <braunr> you can build a file from many sources + <teythoon> or if we do it your way recursing on fs translators *but* + restricting this to root owned translators also suffering from + centralization wrt to the root user? I mean the concept of all mounted + filesystems does not apply cleanly to the hurd + <braunr> i don't understand + <braunr> restricting to the root user doesn't mean it's centralized + <braunr> trust has nothing to do with being centralized + <teythoon> I guess I'm not used to thinking this way + <braunr> teythoon: i guess that's the main reason why so few developers + work on the hurd + <teythoon> also the way fs notification is done is also centralized, that + could also be done recursively + <braunr> what doyou call fs notification ? + <teythoon> and the information I need could just be stuffed into the same + mechanism + <teythoon> fs translators being notified of system shutdown + <braunr> right + <braunr> that gets a bit complicated because the kernel is also a + centralized component + <braunr> it knows every memory object and their pagers + <braunr> it manages all virtual memory + <braunr> there are two different issues here + <braunr> syncing memory and shutting down file systems + <braunr> the latter could be done recursively, yes + <braunr> i wonder if the former could be delegated to external pagers as + well + <braunr> teythoon: but that's not the focus of your work aiui, it would + take much time + <teythoon> sure, but missing an mtab file or better yet /proc/mounts could + be an issue for me, at least a cosmetic one, if not a functional one + <braunr> i understand + <teythoon> and hacking up a quick solution for that seemed like a good + exercise + <braunr> i suggest you discuss it with your mentors + <braunr> they might agree to a temporary centralized solution + <braunr> although i don't think it's much simpler than the recursive one + <teythoon> braunr: would that be implemented in libdiskfs and friends? + <braunr> teythoon: i'm not sure, it might be a generic fs operation + <braunr> libnetfs etc.. are also mount points + <teythoon> so where would it go if it was generic? + <braunr> libfshelp perhaps + <teythoon> translator startup is handled in start-translator-long.c, so in + case a startup is successful, I'd add it to a list? + <braunr> i'd say so, yes + <teythoon> would that cover all cases, passive and active translators? + <braunr> that's another question + <braunr> do we consider passive translators as mounted ? + <teythoon> ah, that was not what i meant + <braunr> i know + <braunr> but it's related + <teythoon> start b/c of accessing a passive one vs. starting an active one + using settrans + <braunr> start_translator_xxx only spawn active translators + <braunr> it's the same + <teythoon> ok + <braunr> the definition of a passive translator is that it starts the + active translator on access + <teythoon> yeah I can see how that wouldn't be hard to implement + <braunr> i think we want to include passive translators in the mount table + <braunr> so registration must happen before starting the active one + <teythoon> so it's a) keeping a list of active translators and b) add an + interface to query fs translators for this list and c) an interface to + query mtab style information? + <braunr> keeping a list of all translators attached + <braunr> and yes + <braunr> well + <braunr> a is easy + <braunr> b is the real work + <braunr> c would be procfs using b + <teythoon> oh? I thought recursing on the translators and querying info + would be separate operations? + <braunr> why so ? + <braunr> the point is querying recursively :) + <braunr> and when i say recursively, it's only a logical view + <teythoon> ok, yes, it can be implemented this way, so we construct the + list while recursing on the translators + <braunr> i think it would be better to implement it the way looking up a + node is done + <teythoon> in a loop, using a stack? + <braunr> iteratively + <braunr> a translator would provide information about itself (if + supported), and referrences to translators locally registered to it + <teythoon> could you point me to the node lookup? + <teythoon> ah, yes + <braunr> eg., you ask /, it tells you it's on /dev/hd0, read-write, with + options, and send rights to /home, /proc, etc.. + <braunr> well rights, references + <braunr> it could be the path itself + <teythoon> rights as in a port to the translators? + <braunr> i think the path would be better but i'm not sure + <braunr> it would also allow you to check the permissions of the node + before querying + <teythoon> path would be nicer in the presence of stacked translators + <braunr> and obviously you'd have the path right away, no need to provide + it in the reply + <teythoon> true + + <teythoon> braunr: if we want to list passive translators (and I agree, we + should), it isn't sufficient to touch libfshelp, as setting a passive + translator is not handled there, only the startup + <braunr> teythoon: doesn't mean you can't add something there that other + libraries will use + <braunr> so yes, not sufficient + + +## IRC, freenode, #hurd, 2013-06-29 + + <teythoon> braunr: diskfs_S_fsys_set_options uses diskfs_node_iterate to + recurse on active translators if do_children is given + <teythoon> braunr: I wonder how fast that is in practice + <teythoon> braunr: if it's fast enough, there might not even be a need for + a new function in fsys.defs + <teythoon> and no need to keep a list of translators for that reason + <teythoon> braunr: if it's not fast enough, then diskfs_S_fsys_set_options + could use the list to speed this up + <braunr> teythoon: on all nodes ? + <teythoon> braunr: i believe so, yes, see libdiskfs/fsys-options.c + <braunr> teythoon: well, if it really is all node, you clearly don't want + that + + +## IRC, freenode, #hurd, 2013-07-01 + + <teythoon> I've ment to ask, the shiny new fsys_get_translators interface, + should it return the options for the queried translator or not? + <braunr> i don't think it should + <teythoon> ok + <braunr> let's walk through why it shouldn't + <teythoon> may I assume that the last argument returned by fsys_get_options + is the "source"? + <braunr> how would you know these options ? + <braunr> the source ? + <teythoon> I wouldn't actually + <braunr> yes, you wouldn't + <braunr> you'd have to ask the translators for that + <braunr> so the only thing you can do is point to them + <teythoon> well, the device to include in the mtab file + <braunr> and the client asks + <braunr> i don't know fsys_get_options tbh + <teythoon> well, both tmpfs and ext2fs print an appropriate value for + "device" as last argument + <braunr> looks like a bad interface to me + <braunr> options should be options + <braunr> there should be a specific call for the device + <braunr> but if everyone agrees with the options order, you can do it that + way for now i guess + <teythoon> one that could be used to recreate the "mount" using either + mount or settrans + <braunr> just comment it where appropriate + <teythoon> I thought that'd be the point? + <braunr> ? + <teythoon> % fsysopts tmp + <teythoon> /hurd/tmpfs --writable --no-inherit-dir-group --no-sync 48K + <braunr> where is the device ? + <teythoon> % settrans -ca tmp $(fsysopts tmp) + <braunr> 15:56 < teythoon> well, both tmpfs and ext2fs print an appropriate + value for "device" as last argument + <teythoon> 48K + <braunr> i don't see it + <braunr> really ? + <teythoon> yes + <braunr> what about ext2fs ? + <braunr> hm ok i see + <teythoon> % fsysopts / + <teythoon> ext2fs --writable --no-inherit-dir-group --sync=10 + --store-type=typed device:hd0s1 + <braunr> i don't think you should consider that as devices + <braunr> but really translator specific options + <pinotree> agree + <teythoon> I don't ;) + <teythoon> b/c the translator calling convention is hardcoded in the mount + utility + <braunr> ? + <teythoon> I think it's reasonable to assume that this mapping can be + reversed + <pinotree> theorically you can write a translator that takes no arguments, + but just options + <braunr> the 48K string for tmpfs is completely meaningless + <braunr> in fstab, it should be none + <pinotree> "tmpfs" + <braunr> the linux equivalent is the size option + <braunr> no, none + <braunr> it's totally ignored + <braunr> and it's recommended to set none rather than the type to avoid + confusion + <teythoon> u sure? + <teythoon> % settrans -cga tmp /hurd/tmpfs --mode=666 6M + <teythoon> % settrans -cga tmp /hurd/tmpfs --mode=666 6M + <teythoon> % fsysopts tmp + <teythoon> /hurd/tmpfs --writable --no-inherit-dir-group --no-sync 6M + <braunr> i've not explained myself clearly + <braunr> it's not ignored by the translator + <braunr> but in fstab, it should be in the options field + <braunr> it's not the source + <braunr> clearly not + <teythoon> ah + <braunr> now i'm talking about fstab, but iirc the format is similar in + mtab/mounts + <pinotree> close, but not the same + <braunr> yes, close + <teythoon> ok, so I'll put a method into libfshelp so that translators can + explicitly set a device and patch all existing translators to do so? + <braunr> teythoon: what i meant is that, for virtual vile systems (actually + file systems with no underlying devices), the device field is normally + ignored + <braunr> teythoon: why do you need that for exactly + <teythoon> right + <pinotree> do they even have a "device" field? + <braunr> (i can see why but i'd like more visibility) + <braunr> pinotree: not yet + <braunr> pinotree: that's what he wants to add + <braunr> but i'd like to see if there is another way to get the information + <braunr> 16:05 < braunr> teythoon: why do you need that for exactly + <teythoon> well if I'm constructing a mtab entry I need a value for the + device field + <braunr> do we actually need it to be valid ? + <teythoon> not necessarily I guess + <braunr> discuss it with your mentors then + <youpi> it has to be valid for e2fsck checks etc. + <braunr> doesn't e2fsck check fstab actually ? + <youpi> i.e. actually for the cases where it's trivial + <youpi> fstab doesn't tell it whether it's mounted + <youpi> I mean fsck checking whether it's mounted + <youpi> not fsck -a + <braunr> oh + <braunr> couldn't we ask the device instead ? + <braunr> looks twisted too + <youpi> that'd mean patching a lot of applications which do similar checks + <braunr> yes + <braunr> teythoon: propose an interface for that with your mentors then + <teythoon> yeah, but couldn't you lay it out a little, I mean would it be + one procedure or like three? + <braunr> 16:04 < teythoon> ok, so I'll put a method into libfshelp so that + translators can explicitly set a device and patch all existing + translators to do so? + <teythoon> ok + <braunr> why three ? + <teythoon> no, I mean when adding stuff to fsys.defs + <braunr> i understood that + <braunr> but why three ? :) + <teythoon> it'd be more generic + <braunr> really ? + <braunr> please show a quick example of what you have in mind + <teythoon> i honestly don't know, thus I'm asking ;) + <braunr> well first, this device thing bothers me + <braunr> when you look at how we set up our ext2fs translators, you can see + they use device:xxx + <braunr> and not /dev/xxx + <braunr> but ok, let's assume it's harmless + <teythoon> ok, but isn't the first way actually better? + <braunr> i think it ends up being the same + <braunr> ideally, that's what we want to use as device path + <teythoon> but you can recreate a storeio translator using the device:xxx + info, the node is useless for that + <braunr> so that we don't need to explicitely set it + <braunr> ? + <braunr> what do you mean ? + <teythoon> well, fsysopts / tells currently tells me device:hd0s1 + <braunr> for /, there isn't much choice + <braunr> /dev isn't there yet + <teythoon> ah, got it + <teythoon> that's why it differs... + <braunr> differs ? + <braunr> from what ? + <braunr> other ext2fs translators are set the same way by the debian + installer for example + <teythoon> % fsysopts /media/scratch + <teythoon> /hurd/ext2fs --writable --no-inherit-dir-group /dev/hd1s1 + <teythoon> here it uses the path to the node + <braunr> that's weird + <braunr> was that done by the debian installer ? + <teythoon> ah no, that was me + <braunr> :p + <braunr> $ fsysopts /home + <braunr> /hurd/ext2fs --writable --no-inherit-dir-group --store-type=device + hd0s6 + <braunr> so as you can see, it's not that simple to infer the device path + <teythoon> oho, yet another way ;) + <teythoon> right then + <pinotree> isn't device:hd0s1 as shortcut for specifying the store type, as + done with --store-type=device hd0s1? + <braunr> but perhaps we don't need to + <braunr> yes it is + <pinotree> iirc it's something libstore does, per-store prefixes + <braunr> ah that sucks + <braunr> teythoon: you may need to normalize those strings + <braunr> so that they match what's in fstab + <braunr> i.e. unix /dev paths + <braunr> otherwise e2fsck still won't be able to find the translators + mounting the device + <braunr> well, if it's mounted actually + <braunr> it just needs to find the matching line in mtab aiui + <braunr> so perhaps a libfshelp function for that, yes + <teythoon> braunr: so you suggest adding a normalizing function to + libfshelp that creates a /dev/path? + <braunr> yes + <braunr> used by the call you intend to add, which returns that device + string as found in fstab + <teythoon> found in fstab? so this would only work for translators managed + by fstab? + <braunr> no + <teythoon> ah + <teythoon> a string like the ones found in fstab? + <braunr> yes + <braunr> so that fsck and friends are able to know whether a device is + mounted or not + <braunr> i don't see any other purpose for that string in mtab + <braunr> you'd take regular paths as they are, convert device:xxx to + /dev/xxx, and return "none" for the rest i suppose + <teythoon> ok + <braunr> i'm not even sure it's right + <braunr> youpi: are you sure it's required ? + <teythoon> well it's a start and I think it's not too much work + <braunr> aiui, e2fsck may simply find the mount point in fstab, and ask the + translator if it's mounted + <teythoon> we can refine this later on maybe? + <braunr> or rather, init scripts, using mountpoint, before starting e2fsck + <braunr> teythoon: sure + <teythoon> there's this mountpoint issue... I need to run fsysopts / + --update early in the boot process + <teythoon> otherwise the device ids returned by stat(2)ing / are wrong and + mountpoint misbehaves + <teythoon> i guess b/c it's the rootfs + <braunr> device ids ? + <teythoon> % stat / | grep Device + <teythoon> Device: 3h/3d Inode: 2 Links: 22 + <braunr> do you mean the major/minor identifiers ? + <teythoon> I do. if I don't do the --update i get seemingly random values + <braunr> i guess that's expected + <braunr> we don't have major/minor values + <braunr> well, they're emulated + <teythoon> well, if that's fixable, that'd be really nice ;) + <braunr> we'll never have major/minor values + <teythoon> yeah, I understand that + <braunr> but they could be fixed by MAKEDEV when creating device nodes + <teythoon> but not having to call fsys_set_options on the rootfs to get the + emulation up to speed + <braunr> try doing it from grub + <braunr> not sure it's possible + <braunr> but worth checking + <teythoon> by means of an ext2fs flag? + <braunr> yes + <braunr> if there is one + <braunr> i don't know the --update flag, is it new from your work ? + <teythoon> braunr: no, it's been there before. -oremount gets mapped to + that + <braunr> it's documented by fsysopts, but not by the ext2fs translators + <teythoon> libdiskfs source says something about flushing buffers iirc + <braunr> -s + <braunr> what does it do ? + <braunr> teythoon: ok + <teythoon> braunr: so the plan is to automatically generate a device path + from the translators argz vector but to provide the functionality so + translators can set a more appropriate value? did I get the last part of + the discussion right? + <braunr> not set, return + <teythoon> yeah return from the procedure but settable using libfshelp? + <braunr> why settable ? + <braunr> you'd have a fsys call to obtain the dev string, and the server + side would call libfshelp on the fly to obtain a normalized value and + return it + <teythoon> ah, make a function overrideable that returns an appropriate + response? + <braunr> overrideable ? + <teythoon> like netfs_append_args + <braunr> you wouldn't change the command line, no + <teythoon> isn't that done using weak references or something? + <teythoon> no I know + <braunr> sorry i'm lost then + <teythoon> never mind, I'll propose a patch early to get your feedback + <youpi> braunr: am I sure that _what_ is required? + <youpi> the device? + <youpi> e2fsck surely needs it, yes + <braunr> a valid device path, yes + <youpi> it can't rely only on fstab + <braunr> yes + <youpi> since users may mount things by hand + <braunr> i've used strace on it and it does perform lookups there + <braunr> (although i also saw uuid magic that i guess wouldn't work yet on + the hurd) + + +## IRC, freenode, #hurd, 2013-07-03 + + <teythoon> I added a procedure to fsys.defs, added a server stub to my + tmpfs translator and wrote a simple client, but something hasn't picked + up the new message yet + <teythoon> % ./mtab tmp + <teythoon> ./mtab: get_translator_info: (ipc/mig) bad request message ID + <teythoon> I guess it's libhurduser.so from glibc, not sure though... + <braunr> glibc would only have the client calls + <braunr> what is "% ./mtab tmp" ? + <teythoon> mtab is my mtab tool/soon to be a translator testing thing, tmp + is an active tmpfs with the appropriate server stub + <braunr> so mtab has the client call, right ? + <teythoon> yes + <braunr> then tmpfs doesn't + <teythoon> so what to do about it? + <teythoon> i set LD_LIBRARY_PATH to my hurd builds lib dir, is that + preserved by settrans -a? + <pinotree> not really + <braunr> not at all + <braunr> there is a wiki entry about that iirc + <pinotree> http://darnassus.sceen.net/~hurd-web/hurd/debugging/translator/ + <teythoon> yeah, I read it too once + <teythoon> ah + <braunr> on the other hand, using export to set the environment should do + the work + <teythoon> yes, that did the trick, thanks :) + * teythoon got his EOPNOPSUPP... *nomnomnom + <braunr> ? + <braunr> same error ? + <teythoon> well I stubbed it out + <braunr> oh + <teythoon> no, that's what I've been expecting ;) + <pinotree> great + <braunr> :) + <braunr> yes that's better than "mig can't find it" + <teythoon> braunr: in that list of active and passive translators that will + have to be maintained, do you expect it should carry more information + other than the relative path to that translator? + <braunr> like what ? + <teythoon> dunno, maybe like a port to any active translator there + <teythoon> should we care if any active translator dies and remove the + entry if there's no passive translator that could restart it again? + <braunr> don't add anything until you see it's necessary or really useful + <braunr> yes + <braunr> think of something like sshfs + <braunr> when you kill it, it's not reported by mount any more + <teythoon> well, for a dynamically allocated list of strings I could use + the argz stuff, but if we'd ever add anything else, we'd need a linked + list or something, maybe a hash table + <teythoon> yes, I thought that'd be useful + <braunr> use libihash for no + <braunr> now + <teythoon> braunr: but what would I use as keys? the relative path should + be unique (unless translators are stacked... hmmm), but that's the value + I'd like to store and ihash keys are pointers + <teythoon> stacked translators are an kinda interesting case for mtab + anyways... + <braunr> why not store the string address ? + <braunr> i suppose that, for stacked translators, the code querying + information would only return the topmost translator + <braunr> since this is the one which matters for regular clients (if i'm + right) + <teythoon> wouldn't that map strings that are equal but stored at different + locations to different values? + <teythoon> that'd defeat the point + <teythoon> I suppose so, yes + <braunr> then add a layer that looks for existing strings before adding + <braunr> the list should normally be small so a linear lookup is fine + <teythoon> yeah sure, but then there's little advantage of using ihash in + the first place, isn't it? + <braunr> over what ? + <teythoon> over not using it at all + <braunr> how would you store the list then ? + <teythoon> it's either ll or ll+ihash + <braunr> uh no + <braunr> let me check + <braunr> there is ihash_iterate + <braunr> so you don't need a linked list + <teythoon> so how do I store my list of strings to deduplicate the keys? + <braunr> you store pointers + <braunr> and on addition, you iterate over all entries, making sure none + matches the new one + <braunr> and if it does, you replace it i guess + <braunr> depending on how you design the rest + <teythoon> in an dynamically allocated region of memory? + <braunr> i don't understand + <braunr> your strings should be dynmaically allocate, yes + <teythoon> no the array of char * + <braunr> your data structure being managed by libihash, you don't care + about allocation + <braunr> what array ? + <teythoon> ah, got it... + <teythoon> right. + <braunr> there is only one structure here, an ihash of char * + <teythoon> yes, I got the picture ;) + <braunr> goo + <braunr> d + <braunr> actually, the lookup wouldn't be linear since usually, hash tables + have stale entries + <teythoon> heh... what forest?!? + <braunr> but that's ok + <braunr> teythoon: ? + <teythoon> the one I couldn't make out b/c of all the trees... + <braunr> ? + <teythoon> ah, it's not important. there is this saying over here, not sure + if there's an english equivalent + <braunr> ok got it + <braunr> we have the same in french + <teythoon> I ran into a problem with my prototype + <teythoon> if an translator is set in e. g. diskfs_S_file_set_translator, + how do I get the path to that node? + <teythoon> I believe there cannot be a way to do that, b/c the mapping is + not bijective + <braunr> it doesn't have to be + <teythoon> ok, so how do I get *a* path for this node? + <braunr> that's another question + <braunr> do you see how the node is obtained ? + <braunr> np = cred->po->np; + <teythoon> yes + <braunr> the translation occurred earlier + <braunr> you need to find where + <braunr> then perhaps, you'll need to carry the path along + <braunr> or if you're lucky, it will still be there somewhere + <teythoon> the translation from path to node? + <braunr> yes + <teythoon> doesn't that happen in the client? and the client hands a file_t + to the file_set_translator routine? + <braunr> the relative lookup can't happen in the client + <braunr> the server can (and often does) retain information between two + RPCs + <teythoon> uh, I can access information from a previous rpc? is that + considered safe? + <braunr> think of open() then read() + <braunr> a simple int doesn't carry enough information + <braunr> that's why it's a descriptor + <teythoon> ah, the server retains some state, sure + <braunr> what it refers to is the state retained between several calls + <braunr> the object being invoked by clients + <braunr> teythoon: what is the "passive" parameter passed to + diskfs_S_file_set_translator ? + <teythoon> braunr: argz vector of the passive translator + <braunr> so it is a name + <braunr> but we also want active translators + <braunr> and what is active ? + <teythoon> not the name of the node though + <teythoon> active is the port (?) to the active translator + <teythoon> I guess + <braunr> fsys_t, looks that way yes + <braunr> i suppose you could add the path to the peropen structure + <teythoon> ok + <braunr> see diskfs_make_peropen + <teythoon> braunr: but translation happens in dir_lookup + <teythoon> in all places I've seen diskfs_make_peropen used, the path is + not available + <teythoon> why did you point me to diskfs_make_peropen? + <teythoon> s/dir_lookup/diskfs_lookup/ + <teythoon> diskfs_lookup operates on struct node, so the path would have to + be stored there, right? + <braunr> teythoon: dir_lookup should call diskfs_make_peropen + <braunr> at least diskfs_S_dir_lookup does + <braunr> and the path is present there + <teythoon> braunr: right + + <teythoon> hrm... I added a path field to struct peropen and initialize it + properly in diskfs_make_peropen, but some bogus values keep creeping in + :/ + <braunr> first of all, make it a dynamically allocated string + <teythoon> it is + <braunr> not a fixed sized embedded array + <braunr> good + <teythoon> yes + <braunr> if you really need help debugging what's happening, feel free to + post your current changes somewhere + <teythoon> there is a struct literal in fsys-getroot.c, but i fixed that as + well + <teythoon> % ./mtab tmp + <teythoon> none tmp ../tmpfs/tmpfs writable,no-inherit-dir-group,no-sync 0 + 0 + <teythoon> none tmp/bar ../tmpfs/tmpfs + writable,no-inherit-dir-group,no-sync 0 0 + <teythoon> none tmp/foo ../tmpfs/tmpfs + writable,no-inherit-dir-group,no-sync 0 0 + <teythoon> none tmp/foo/bar ../tmpfs/tmpfs + writable,no-inherit-dir-group,no-sync 0 0 + <teythoon> :) + + +## IRC, freenode, #hurd, 2013-07-10 + + <teythoon> btw, I read getcwd.c and got the idea + <teythoon> however this situation is different afaict + <teythoon> getcwd has a port to the current working directory, right? + <teythoon> so they can do open_dir with .. as relative path + <teythoon> but all I've got is a port referencing the node the translator + is being attached to + <teythoon> s/open_dir/dir_lookup/ + <teythoon> and that is not necessarily a directory, so dir_lookup fails + with not a directory + <teythoon> as far as I can see it is not possible to get the directory a + node is in from a port referencing that node + <teythoon> dir_lookup has to be handled by all nodes, not just directories + <teythoon> but file nodes only support "looking up" the empty string + <teythoon> not empty, but null: + <teythoon> This call is required to be supported by all files (even + non-directories) if the filename is null, and should function in that + case as a re-open of the file. */ + <braunr> why do you want the directory ? + <braunr> 10:40 < teythoon> as far as I can see it is not possible to get + the directory a node is in from a port referencing that node + <teythoon> to readdir(3) it and figure out the name of the node the + translator is bound to + <teythoon> similar to what getcwd does + <braunr> that's out of the question + <teythoon> wasn't that was youpi was suggesting? + <braunr> you may have a lot of nodes in there, such a lookup shouldn't be + done + <braunr> i didn't see that detail + <teythoon> "│ Concerning storing the path, it's a bit sad to have to do + that, and + <teythoon> │ it'll become wrong if one moves the mount points. Another + way would + <teythoon> │ be to make the client figure it out by itself from a port to + the mount + <teythoon> │ point, much like glibc/sysdeps/mach/hurd/getcwd.c. It'll be + slower, but + <teythoon> │ should be safer. The RPC would thus return an array of + ports to the + <teythoon> │ mount points instead of an array of strings. + <braunr> yes i remember that + <braunr> but i didn't understand well how getcwd work + <braunr> s + <braunr> another scalability issue + <braunr> not a big one though, we rarely have translators in directories + with thousands of nodes + <braunr> so why not + <braunr> teythoon: do it as youpi suggested + <braunr> well if you can + <braunr> eh + <braunr> if not, i don't know + <braunr> 10:47 < teythoon> │ it'll become wrong if one moves the mount + points. Another way would + <teythoon> yes, I know... :/ + <teythoon> well, I'm not even sure it is possible to get the directory a + node is in from the port referencing the node + <teythoon> as in, I'm not sure if the information is even there + <teythoon> b/c a filesystem is a tree, directories are nodes and files are + leafs + <teythoon> all non-leaf nodes reference their parent to allow traversing + the tree starting from any directory + <teythoon> but why would a leaf reference its parent(s - in case of + hardlinks)? + <braunr> uh, for the same reason ? + <teythoon> sure, it would be nice to do that, but I dont think this is + possible on unixy systems + <braunr> ? + <teythoon> you cannot say fchdir(2) to a fd that references a file + <braunr> do you mean /path/to/file/../ ? + <teythoon> yes + <teythoon> only that /path/to/file is given as fd or port + <braunr> when i pasted + <braunr> 10:49 < braunr> 10:47 < teythoon> │ it'll become wrong if one + moves the mount points. Another way would + <braunr> i was actually wondering if it was true + <teythoon> ah + <braunr> why can't the path be updated at the same time ? + <braunr> it's a relative path anyway + <braunr> completely managed by the parent translator + <teythoon> ah + <teythoon> right + <teythoon> it's still kind of hacky, but I cannot see how to do this + properly + <braunr> hacky ? + <teythoon> but yes, updating the path should work I guess + <teythoon> or sad + <braunr> what i find hacky is to set translators in two passes + <braunr> otherwise we'd only keep the translator paths + <braunr> not all paths + <teythoon> true + <braunr> but then, it only concerns open nodes + <braunr> and again, there shouldn't be too many of them + <braunr> so actually it's ok + <teythoon> braunr: I understand the struct nodes are cached in libdiskfs, + so wouldn't it be easier to attach the path to that struct instead of + struct peropen so that all peropen objects reference the same node + object? + <teythoon> so that the path can be updated if anyone dir_renames it + <teythoon> *all peropen objects derived from the same file name that is + <braunr> teythoon: i'm not sure + <braunr> nodes could be real nodes (i.e. inodes) + <braunr> there can be several paths for the same inode + <teythoon> braunr: I'm aware of that, but didn't we agree the other day + that any path would do? + <braunr> i don't remember we did + <braunr> i don't know the details well, but i don't think setting a + translator on a hard link should set the translator at the inode level + <braunr> on the other hand, if a new inode is created to replace the + previous one (or stack over it), then storing the path there should be + fine + <teythoon> braunr: I don't think I can update the paths if they're stored + in the peropen struct + <teythoon> how would I get a reference to all those peropen objects? + <braunr> ? + <braunr> first, what's the context when you talkb about updating paths ? + <teythoon> well, youpi was concerned about renaming a mount point + <teythoon> and you implied that this could be managed + <braunr> can we actually do that btw ? + <teythoon> what? + <braunr> renaming a mount point + <teythoon> yep, just tried + <braunr> i mean, on a regular unix system like linux + <braunr> $ mv test blah + <braunr> mv: cannot move `test' to `blah': Device or resource busy + <braunr> (using sshfs so YMMV) + <pinotree> do you have anything (shells, open files, etc) inside it? + <braunr> no + <braunr> i'll try with an empty loop-mounted ext4 + <teythoon> I was testing on the Hurd, worked fine there even with a shell + inside + <braunr> same thing + <braunr> i consider it a bug + <braunr> we may want to check what posix says about it + <teythoon> o_O + <braunr> and decide not to support renaming + <teythoon> why? + <pinotree> start a discussion in ml, maybe roland can chime in + <braunr> it complicates things + <braunr> ah yes + <teythoon> sure, but I can move or rename a directory, why should it be + different with a mount point? + <braunr> because it's two of them + <braunr> they're stacked + <braunr> if we do want to support that, we must be very careful about + atomically updating all the stack + <teythoon> ok + <teythoon> braunr: I'm trying to detect dying translators to remove them + from the list of translators + <teythoon> what port can I use for that purpose? + <teythoon> if I use the bootstrap port, can I then use the same method as + init/init.c uses? just defining a do_mach_notify_dead_name function and + the multiplexer will call this? + <braunr> teythoon: possibly + <teythoon> braunr: we'll see shortly... + <teythoon> I get KERN_INVALID_CAPABILITY indicating that my bootstrap port + is invalid + <teythoon> when calling mach_port_request_notification to get the dead name + notification I mean + <braunr> is the translator already started when you do that ? + <teythoon> yes, at least I think so, I'm hooking into + diskfs_S_file_set_translator and that gets an active translators port + <teythoon> also the mach docs suggests that the notification port is + invalid, not the name port referencing the translator + <braunr> i guess it shouldn't + <braunr> oh + <braunr> please show the code + <braunr> but beware, if the translator is started, assume it could die + immediately + <teythoon> braunr: http://paste.debian.net/15371/ line 87 + <braunr> teythoon: notify can't be bootstrap + <braunr> what do you have in mind when writing this ? + <braunr> i'm not sure i follow + <teythoon> I want to be notified if an active translator goes away to + remove it from the list of translators + <braunr> ok but then + <braunr> create a send-once right + <braunr> and wait on it + <braunr> also, why do you want to be notified ? + <braunr> isn't this already done ? + <braunr> or can't do it lazily on access attempt ? + <braunr> +you + <teythoon> in the client? + <braunr> in the parent server + <braunr> what happens currently when a translator dies + <braunr> is the parent notified ? + <braunr> or does it give an invalid right ? + <teythoon> ah, i think so + <braunr> then you don't need to do it again + <teythoon> right, I overlooked that |