[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] [[!toc]] # IRC, freenode, #hurd, 2013-06-23 braunr: sorry for the late reply. Honestly to say, the school works had taken most of my time these days. I haven't got any siginificant progress now. I am trying to write a little debugger demo on Hurd. braunr: things goes more hard than I think, these are some differences between ptrace() on Hurd and Linux. I am trying to solve this. # IRC, freenode, #hurd, 2013-06-24 this is my weekly report http://hacklu.com/blog/gsoc-weekly-report1-117/. and I have two main questions when I read the gdb source code. 1/What is the S_exception_raise_request()? 2/what is the role of ptrace in gdb port on Hurd? hacklu: where did you see S_exception_raise_request? in gdb/gnu-nat.c ah, in gdb yeah. and I have read the . is says the S_ start means server stub. yes what happens is that gnu_wait keeps calling mach_msg to get a message then it passes that message to the various stubs servers see just below, it calls exc_server, among others and that's exc_server which ends up calling S_exception_raise_request, if the message is an exception_raise request exc_server is a mere multiplexer, actually S_exception_raise_request is the implementation of the request part (so one half of a typical RPC) of the Mach exception interface. See gdb/exc_request.defs in GDB and include/mach/exc.defs in Mach. youpi: how gnu_wait pass one message to exc_server? in which function? in gnu_wait() && !exc_server (&msg.hdr, &reply.hdr) oh, I see this. firstly I think it is a type check simply. see the comment: "handle what we got" The Hurd's proc server also is involved in the exception passing protocol (see its source code). tschwinge: I will check the source code later. is the exception take place in this way: 1. the inferior call ptrace(TRACE_ME). 2.the gdb call task_set_exception_port. 3. mach send a notification to the exception port set before. 4. gdb take some action. hacklu: Yes, that's it, roughly. The idea is that GDB replaces a process' standard exception port, and replaces it "with itself", so that when the process that is debugged receives and exception (by Mach sending a exception_raise RPC), GDB can then catch that and act accordingly. hacklu: As for your other questions, about ptrace: As you can see in [glibc]/sysdeps/mach/hurd/ptrace.c, ptrace on Hurd is simply a wrapper around vm_read/write and more interfaces. hacklu: As the GDB port for Hurd is specific to Hurd by definition, you can also directly use these calls in GDB for Hurd. ..., as it is currently done. and in detail, the part 3 mach send a notification to the excetption port is like this: gnu_wait get the message in mach_msg, and then pass it to exc_serer by exc_server(),then exc_server call S_exception_raise_request()? ? tschwinge: yeah, I have see the ptrace.c. I was wonder about nobody use ptrace in Hurd except TRACEME... hacklu: Right about »and in detail, [...]«. hacklu: It would be very good (and required for your understanding anyway), if you could write up a list of things that happens when a process (both under the control of GDB as well as without GDB) is sent an exception (due to a breakpoint instruction, for example). Let me look something up. tschwinge: what's the function of exc_server? if I can get the notification in mach_msg(). to multiplex the message i.e. decoding it, etc. up to calling the S_ function with the proper parameters exc_server being automatically generated, that saves a lot of code That is generated by MIG from the gdb/exc_request.defs file. You'll find the generated file in the GDB build directory. I have wrote down the filenames. after this I will check that. hacklu: I suggest you also have a look at the Mach 3 Kernel Principles book, . This also has some explanation of the thread/task's exception mechanism. And of course, explains the RPC mechanism, which the exception mechanism is built upon. And then, really make a step-by-step list of what happens; this should help to better visualize what's going on. ok. later I will update this list on my blog. hacklu: I cannot tell off-hand why GDB on Hurd is using ptrace(PTRACE_TRACEME) instead of doing these calls manually. I will have to look that up, too. tschwinge: thanks. hacklu: Anyway -- you're asking sensible questions, so it seems you're making progress/are on track. :-) tschwinge: there is something harder than I had thought, I haven't got any meaningful progress. sorry for this. hacklu: That is fine, and was expected. :-) (Also, you're still busy with university.) I will show more time and enthusiasm on this. hacklu: Oh, and one thing that may be confusing: as you may have noticed, the names of the same RPC functions are sometimes slightly different if different *.defs files. What is important is the subsystem number, such as 2400 in [GDB]/gdb/exc_request.defs (and then incremented per each routine/simpleroutine/skip directive). hacklu: Just for completeness, [hurd]/hurd/subsystems has a list of RPC subsystems we're using. And the name given to routine 2400, for example, is just a "friendly name" that is then used locally in the code where the *.defs file has been processed by MIG. What a clumsy explanation of mine. But you'll get the idea, I think. ;-) hacklu: And don't worry about your progress -- you're making a lot of progress already (even if it doesn't look like it, because you're not writing code), but the time spent on understanding these complex issues (such as the RPC mechanism) definitely counts as progress, too. tschwinge: not clearly to got it as I am not sensitive with the MIG's grammer. But I know, the exc is the routine 2400's alias name? hacklu: I'd like to have you spend enough time to understand these fundamental concepts now, and then switch to "hacking mode" (write code) later, instead of now writing code but not understanding the concepts behind it. I have wrote a bit code to validate my understanding when I read the soruce code. But the code not run. http://pastebin.com/r3wC5hUp The subsystem directive [...]. As well, let me just point you to the documentation: , MIG - THE MACH INTERFACE GENERATOR, chapter 2.1 Subsystem identification. hacklu: Yes, writing such code for testing also is a good approach. I will have to look at that in more detail, too. * tschwinge guesses hacklu is probably laughing when seeing the years these documents were written in (1989, etc.). ;-) mach_msg make no sense in my code, and the process just hang. kill -9 can't stop is either. hacklu: do you understand why kill -KILL might not work now ? braunr: no, but I found I can use gdb to attach to that process, then quit in gdb, the process quit too. maybe that process was waiting a resume. something like that yes iirc it's related to a small design choice in the proc server something that put processes in an uninterruptible state when being debugged iirc ? if i recall cl=orrectly correctly* like D status in linux? or T there has been a lot of improvements regarding signal handling in linux over time so it's not really comparable now but that's the idea in ps, i see the process STAT is THumx did you see that every process on the hurd has at least two threads ? no, but I have see that in hurd, the exception handler can't live in the same context with the victim. so there must be at least two threads. I think hacklu: yes that thread also handles regular signals in addition to mach exceptions (there are two levels of multiplexing in servers, first locating the subsystem, then the server function) hacklu: if what i wrote is confusing, don't hesitate to ask for clarifications (i really don't intend to make things confusing) braunr: I don't know what you say about the "multiplexing in servers". For instance, is it means how to pass message from mach_msg to exc_server in gnu_wait()? hacklu: i said that the "message thread" handles both mach exceptions and unix signals hacklu: these are two different interfaces (and in mach terms, subsystems) hacklu: see hurd/msg.defs for the msg interface (which handles unix signals) hacklu: to handle multiple interfaces in the same thread, servers need to first find the right subsystem this is done by subsequently calling all demux functions until one returns true (finding the right server function is done by these demux functions) hacklu: see hurd/msgportdemux.c in glibc to see how it's done there it's short actually, i'll past it here : return (_S_exc_server (inp, outp) || _S_msg_server (inp, outp)); hacklu: did that help ? braunr: a bit more confusing. one "message thread" handles exceptions and signals, means the message thread need to recive message from two port. then pass the message to the right server which handle the message. the server also should pick the right subsystem from a lot of subsystems to handle the msg. is this ? the message thread is a server thread (which means every normal process is actually also a server, receiving exceptions and signals) there may be only two ports, or more, it doesn't matter much, the port set abstraction takes care of that so the message thread directly pass the msg to the right subsystem? not directly as you can see it tries them all until one is able to handle the incoming message i'm not sure it will help you with gdb, but it's important to understand for a better general understanding of the system ugly sentence ah, I see. like this in gnu-nat.c if(!notify_server(&msg.hdr, &reply.hdr) && !exc_server(&msg.hdr...) yes the thread just ask one by one. be careful about the wording the thread doesn't "send requests" it runs functions (one might be tempted to think there are other worker threads waiting for a "main thread" to handle demultiplexing messages) I got it. the notify_server function is just run in the same context in "message thread",and there is no RPC here. yes and the notify_server code is generater by mig automatically. yes # IRC, freenode, #hurd, 2013-06-29 [[!tag open_issue_documentation]] I just failed to build the demo on this. http://walfield.org/pub/people/neal/papers/hurd-misc/ipc-hello.c or, example in machsys.doc called simp_ipc.c we don't use cthreads anymore, but pthreads pinotree: em.. and I also failed to find the in example of that i don't know maybe the code in that book out-of-date hacklu: mig and mach ipc documentation is quite dated unfortunately, and so are many examples floating around the net btw, I have one more question. when I read . I find this state: When an exception occurs in a thread, the thread sends an exception message to its exception port, blocking in the kernel waiting for the receipt of a reply. It is assumed that some task is listening to this port, using the exc_serverfunction to decode the messages and then call the linked in catch_exception_raise. It is the job of catch_exception_raiseto handle the exception and decide the course of action for thread. that says, it assumed another task to recieve the msg send to one thread's exception port. why another task? I remmebered, there are at least two threads in one task, one is handle the exception stuffs. there are various reasons first is, the thread causing the exception is usually not waiting for a message next, it probably doesn't have all the info needed to resolve the exception (depending on the system design) and yes, the second thread in every hurd process is the msg thread, handling both mach exceptions and hurd signals but in this state, I can't find any thing with the so called msg thread ? if exist a task to do the work, why we need this thread? this thread is the "task" ? the msg thread is the thread handling exceptions for the other threads in one task wording is important here a task is a collection of resources so i'm only talking about threads really 14:11 < hacklu> assumed that some task is listening to this this is wrong a task can't listen only a thread can in you words, the two thread is in the same task? yes 14:32 < braunr> and yes, the second thread in every hurd process is the msg thread, handling both mach exceptions and hurd signals process == task here yeah, I always think the two thread stay in one task. but I found that state in . so I confuzed s/confuzed/confused statement you mean if two thread stay in the same task. and the main thread throw a exception, the other thread to handle it? depends on how it's configured the thread receiving the exceptions might not be in the same task at all on the hurd, only the second thread of a task receives exception s I just wonder how can the second thread catch the exception from its containning task forget about tasks tasks are resource containers they don't generate or catch exceptions only threads do for each thread, there is an exception port that is, one receive right, and potentially many send rights the kernel uses a send right to send exceptions the msg thread waits for messages on the receive right that's all ok. if I divide zero in main thread, the kernel will send a msg to the main thread's exception port. and then, the second thread(in the same task) is waiting on that port. so he get the msg. is it right? don't focus on main versus msg thread it applies to all other threads as well otherwise, you're right ok, just s/main/first no main *and* all others except msg main *and* all others except msg ? the msg thread gets exception messages for all other threads in its task (at least, that's how the hurd configures things) got it. if the msg thread throw exception either, who server for himself? i'm not sure but i guess it's simply forbidden i used gdb to attach a little progrom which just contains a divide zero. and I only found the msg thread is in the glibc. yes where is the msg thread located in. it's created by glibc is it glibc/hurd/catch-exc.c? that's the exception handling code, yes there are some differences between the code and the state in . state or statement ? staement which one ? http://pastebin.com/ZTBrUAsV When an exception occurs in a thread, the thread sends an exception message to its exception port, blocking in the kernel waiting for the receipt of a reply. It is assumed that some task is listening (most likely with mach_msg_server) to this port, using the exc_serverfunction to decode the messages and then call the linked in catch_exception_raise. It is the job of catch_exception_raiseto handle the exception and decide the course of action for thread. The state of the blocked thread can be examined with thread_get_state. what difference ? in the code, I can't find things like exc_server,mach_msg_server uh ok it's a little tangled but not that much you found the exception handling code, and now you're looking for what calls it simple see _hurdsig_fault_init from that statemnet I thought there are another _task_ do the exception things for all of the systems thread before you have told me the task means the msg thread. again 14:47 < braunr> forget about tasks 14:47 < braunr> tasks are resource containers 14:47 < braunr> they don't generate or catch exceptions 14:47 < braunr> only threads do yeah, I think that document need update. no it's a common misnomer once you're used to mach concepts, the statement is obvious braunr: so I need read more :) _hurdsig_fault_init send exceptions for the signal thread to the proc server? why come about _proc_ server? no it gives the proc server a send right for signals exceptions are a mach thing, signals are a hurd thing the important part is err = __thread_set_special_port (_hurd_msgport_thread, THREAD_EXCEPTION_PORT, sigexc); this one set the exception port? yes hm wait actually no, wrong part :) this sets the excpetion port for the msg thread (which i will call the signal thread as mentioned in glibc) but the comment above this line, Direct signal thread exceptions to the proc server means what? that the proc server handles exceptions on the signal thread the term signal thread equals the term msg thread? yes so, the proc server handles the exceptions throwed by the msg thread? looks that way feels a little strange. why ? this thread isn't supposed to cause exceptions if it does, something is deeply wrong, and something must clean that task up and the proc server seems to be the most appropriate place from where to do it why need a special server to just work the msg thread? I don't think that thread will throw exception frequentlly what does frequency have to do with anything here ? ok the appropriate code is _hurdsig_init the port for receiving exceptions is _hurd_msgport the body of the signal thread is _hurd_msgport_receive aha, in the _hurd_msgport_receive I have finally found the while(1) loop mach_msg_server(). so the code is conform with the documents. braunr: [21:18] what does frequency have to do with anything here ? yes, I have totally understood your words now. thank you very much. :) # IRC, freenode, #hurd, 2013-07-01 hi. this is my weekly report. http://hacklu.com/blog/gsoc-weekly-report2-124/ welcome to any comment teythoon: I only get clear about the rpc stuff. seems a lot behind my plan good progress :) I have wrote the details of the exception handle which was asked by tschwing_ last week. Am I all right in my post? hacklu: as far as I understand signals, yes :) youpi: thanks for god, I am on the right way finally... :) the mig book says simpleroutine is the one use to implement asyn RPCs which doesn't expect an reply. But I have found a place to pass an reply port to the RPC interface which has been declared as simpleroutine hacklu: probably the simpleroutine hardcodes a reply port? hacklu: about _hurd_internal_post_signal, this is the hairiest part of GNU/Hurd, signal handling simply because it's the hairiest part of POSIX :) you probably want to just understand that it implements the POSIXity of signal delivering i.e. deliver/kill/suspend the process as appropriate I don't think you'll need to dive more aha. it will save a lot of time. it seems like the wait_for_inferior() in gdb. which also has too many lines and too many goto hacklu: btw, which simpleroutine were you talking about ? I forget where it is, I am finding it now. which version of gdb are you looking the source of? (in mine, wait_for_inferior is only 45 lines long) I dont know how to pick the verison, I just use the git version. maybe I give a wrong name. ok youpi:I remembered, my experience comes from here http://www.aosabook.org/en/gdb.html. (All of this activity is managed by wait_for_inferior. Originally this was a simple loop, waiting for the target to stop and then deciding what to do about it, but as ports to various systems needed special handling, it grew to a thousand lines, with goto statements criss-crossing it for poorly understood reasons.) youpi: the simpleroutine is gdb/gdb/exc_request.defs so there is indeed an explicit reply port but simpleroutine is for no-reply use. why use reply port here? AIUI, it's simply a way to make the request asynchronous, but still permit an answer ok, I will read the mig book carefully. hacklu: as youpi says a routine can be broken into two simpleroutines that's why some interfaces have interface.defs, interface_request.defs and interface_reply.defs files nlightnfotis: in mach terminology, a right *is* a capability the only thing mach doesn't easily provide is a way to revoke them individually braunr: Right. And ports are associated with the process server and the kernel right? I mean, from what I have understood, if a process wants to send a signal to another one, it has to do so via the ports to that process held by the process server and it has to establish its identity before doing so, so that it can be checked if it has the right to send to that port. yes do process own any ports? or are all their ports associated with the process server? *processes mach ports were intended for a lot of different uses but in the hurd, they mostly act as object references the process owning the receive right (one at most per port) implements the object processes owning send rights invoke methods on the object use portinfo to find out about the rights in a task (process is the unix terminology, task is the mach terminologyà ) i use them almost interchangeably ahh yes, I remember about the last bit. And mach tasks have a 1 to 1 association with user level processes (the ones associated with the process server) the proc server is a bit special because it has to know about all processes yes In context of [[open_issues/libpthread/t/fix_have_kernel_resources]]: hacklu: if you ever find out about either glibc or the proc server creating one receive right for each thread, please let me know # IRC, freenode, #hurd, 2013-07-07 how fork() goes? see sysdeps/mach/hurd/fork.c in glibc' sources when the father has two thread( main thread and the signal thead), if the father call fork, then the child inmediatelly call exev() to change the excute file. how many thread in the children? For instance, the new execute file also have two thread. will the exev() destroyed two threads and then create two new? s/exev()/excv() s/exev()/exec() :) what libhurduser-2.13.so does? where can I find this source? contains all the client stubs for hurd-specific RPCs it is generated and built automatically within the glibc build process and what is the "proc" server? what handles in user spaces the processes so if I call proc_wait_request(), I will go into the S_proc_wait_reply? thanks, I have found that. # IRC, freenode, #hurd, 2013-07-08 hi, this is my weekly report. http://hacklu.com/blog/gsoc-weekly-report3-137/ this week I have met a lot of obstacles. And I am quite desired to participate in this meeting. hacklu: So from your report, the short version is: you've been able to figure out how the things work that you were looking at (good!), and now there are some new open questions that you're working on now. hacklu: That sounds good. We can of course try to help with your open questions, if you're stuck figuring them out on your own. tschwinge: the most question is: what is the proc server? why need to call proc_get_reqeust() before the mach_msg()? and Is there exist any specific running sequence between father and child task after fork()? And I found the inferior always call the trace_me() in the same time(the trace me printf always in the same line of the output log). which I have post in my report. hacklu: The fork man-page can provide a high-level answer to your Q3: »The child process is created with a single thread—the one that called fork(). The entire virtual address space of the parent is replicated in the child, including the states of mutexes, condition variables, and other pthreads objects [...]« hacklu: What happens in GNU Hurd is that the signal thread is also "cloned" (additionally to the thread which called fork), but then it (the signal thread) is re-started from the beginning. (So this is very much equivalent to creating a new signal thread.) hacklu: Then, upon exec, a new memory image is created/loaded, replacing the previous one. [glibc]/sysdeps/mach/hurd/execve.c. What actually happens with the existing thread (in particular, the signal thread) I don't know off-hand. Then answer is probably found in [glibc]/hurd/hurdexec.c -- and perhaps some code of the exec server ([hurd]/exec/). I have checked the status of my regiter mail to FSF. it says it had arrived in USA. hacklu: OK, good. hacklu: This is some basic information about the observer_* functions is GDB: http://sourceware.org/gdb/current/onlinedocs/gdbint/Algorithms.html#index-notifications-about-changes-in-internals-57 »3.10 Observing changes in gdb internals«. tschwinge: not too clear. I will think this latter. and what is the proc server? hacklu: /hurd/proc, maps unix processes to mach threads afaiui teythoon: question is, the mach_msg() will never return unless I called proc_wait_request() first. hacklu: sorry, I've no idea ;) teythoon: :) hacklu: I will have to look into that myself, too; don't know the answer off-hand. hacklu: In your blog you write proc_get_request -- but such a functions doesn't seems to exist? tschwinge: s/proc_get_request/proc_wait_request called in gun_wait() [gnu-nat.c] hacklu: Perhaps the wait man-page's description of WUNTRACED gives a clue: »also return if a child has stopped [...]«. But it also to me is not yet clear, how this relates to the mach_mag call, and how the proc server exactly is involved in it. I'm reading various source code files. At least, I don't undestand why it is required for an exception to be forwarded. if I need to read the proc server source code? I can see how it to become relevant for the case that GDB has to be informed that the debugee has exited normally. hacklu: Yeah, probably you should spend some time with that, as it will likely help to get a clearer picture of the situation, and is relevant for other interactions in GDB, too. hacklu: By the way, if you find that pieces of the GDB source code (especially the Hurd files of it) are insufficiently documented, it's a very good idea, once you have figured out something, to add more source code comments to the existing code. Or writed these down separately, if that is easier. which is the proc server? hurd/exec ? that ok, I already comment things on my notes. hacklu: [Hurd]/proc/ hacklu: And [Hurd]/hurd/process*.defs got it hacklu: I'll have to experiment a bit with your HDebugger example, but I'm out of time right now, sorry. Will continue later. tschwinge: yep, the HDebugger has a problem, if you put the sleep() after the printf in the just_print(), thing will hang. tschwinge: and I am a little curious about how do you find my code? I dont't remember I have mentioned that :) tschwinge: I have post my gihub link in the last week report, I found that. hacklu: That's how I found it, yes. tschwinge: :) # IRC, freenode, #hurd, 2013-07-14 hi. what is a process's msgport? And where can I find the msg_sig_post_untraced_request()? (msg_sig_post* in [hurd]/hurd/msg_defs) this is my debugger demo code https://github.com/hacklu/HDebugger.git use make test to run the demo. I put a breakpoint before the second printf in hello_world(inferior program). but I can't resume execution from that. could somebody give me some suggestions? thanks so much. hacklu: % make test make: *** No rule to make target `exc_request_S.c', needed by `all'. Stop. teythoon: updated, forget to git add that file . hacklu_: cool, seems to work now will look into this tomorrow :) exit teythoon: not work. the code can,t resume from a breakpoint # IRC, freenode, #hurd, 2013-07-15 hi, this is my weekly report. http://hacklu.com/blog/gsoc-weekly-report4-148/ sadly to unsolve the question of resume from breakpoint. hacklu: have you tried to figure out what gdb does to resume a process? teythoon: hi. em, I have tried, but haven't find the magic in gdb yet. have you tried rpctrace'ing gdb? no, rpctrace has too many noise. I turned on the debug in gdb. I don't want rpctrace start gdb as its child task. if it can attach at some point instead of at start hacklu: you don't need to use gdb interactively, you could pipe some commands to it teythoon: that sounds a possible way. I am try it, thank you youpi: gdb can't work correctlly with rpctrace even in batch mode. get something like this "rpctrace: get an unknown send right from process 2151" hacklu: well, ideally, fix rpctrace ); ;) hacklu: but you can also as on the list, perhaps somebody knows what you need ok. or I should debug gdb more deeply. do both so either of them may win first braunr: I have found that, if there is no exception appears, the signal thread will not be createed. Then there is only one thread in the task. # IRC, freenode, #hurd, 2013-07-17 braunr: ping hacklu__: yes ? I have reply your email i don't understand "I used this (&_info)->suspend_count to get the sc value." before the thread_info call ? no, after the call but you have a null pointer