From 9933cec0a18ae2a3d752f269d1bb12c19f51199d Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sun, 21 Jul 2013 15:35:02 -0400 Subject: IRC. --- capability.mdwn | 20 +- community/gsoc/2013/hacklu.mdwn | 617 ++++++++++++++ community/gsoc/2013/nlightnfotis.mdwn | 450 ++++++++++ community/gsoc/project_ideas/mtab.mdwn | 98 +-- community/gsoc/project_ideas/mtab/discussion.mdwn | 907 +++++++++++++++++++++ hurd.mdwn | 1 + hurd/coding_style.mdwn | 59 ++ hurd/console/discussion.mdwn | 10 +- hurd/debugging.mdwn | 27 + hurd/debugging/rpctrace.mdwn | 4 + hurd/translator.mdwn | 2 + hurd/translator/auth.mdwn | 13 +- hurd/translator/firmlink.mdwn | 14 +- hurd/translator/netio.mdwn | 2 + hurd/translator/proc.mdwn | 53 ++ hurd/translator/procfs/jkoenig/discussion.mdwn | 20 - hurd/translator/socketio.mdwn | 30 + hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn | 129 ++- libpthread.mdwn | 41 +- microkernel/mach/deficiencies.mdwn | 128 +++ microkernel/mach/gnumach/debugging.mdwn | 5 +- .../mach/gnumach/interface/syscall/mach_print.mdwn | 29 + microkernel/mach/history.mdwn | 134 +++ microkernel/mach/mig.mdwn | 6 +- microkernel/mach/mig/structured_data.mdwn | 119 +++ open_issues/64-bit_port.mdwn | 20 +- open_issues/anatomy_of_a_hurd_system.mdwn | 189 +++-- open_issues/arm_port.mdwn | 1 - open_issues/code_analysis.mdwn | 14 + open_issues/dde.mdwn | 44 + open_issues/gdb_signal_handler.mdwn | 403 +++++++++ open_issues/glibc/0.4.mdwn | 4 +- open_issues/glibc/debian.mdwn | 67 ++ open_issues/glibc/debian/experimental.mdwn | 60 ++ open_issues/glibc/t/tls-threadvar.mdwn | 12 + open_issues/libc_variant_selection.mdwn | 25 +- open_issues/libnetfs_argument_parsing.mdwn | 62 ++ open_issues/libpthread.mdwn | 197 ++++- .../libpthread/t/fix_have_kernel_resources.mdwn | 181 +++- open_issues/libpthread_assertion_thread_prevp.mdwn | 20 + open_issues/mondriaan_memory_protection.mdwn | 85 ++ open_issues/some_todo_list.mdwn | 15 + open_issues/translator_stdout_stderr.mdwn | 87 +- open_issues/user-space_device_drivers.mdwn | 92 ++- 44 files changed, 4243 insertions(+), 253 deletions(-) create mode 100644 community/gsoc/2013/hacklu.mdwn create mode 100644 community/gsoc/2013/nlightnfotis.mdwn create mode 100644 community/gsoc/project_ideas/mtab/discussion.mdwn create mode 100644 hurd/coding_style.mdwn create mode 100644 hurd/translator/proc.mdwn create mode 100644 hurd/translator/socketio.mdwn create mode 100644 microkernel/mach/mig/structured_data.mdwn create mode 100644 open_issues/gdb_signal_handler.mdwn create mode 100644 open_issues/libnetfs_argument_parsing.mdwn create mode 100644 open_issues/mondriaan_memory_protection.mdwn diff --git a/capability.mdwn b/capability.mdwn index 7219cdce..0ebe5cd4 100644 --- a/capability.mdwn +++ b/capability.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2007, 2008, 2010, 2011 Free Software Foundation, -Inc."]] +[[!meta copyright="Copyright © 2007, 2008, 2010, 2011, 2013 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -77,6 +77,22 @@ port|microkernel/mach/port]]. As in UNIX (see above), they are not [[persistent|persistency]]. +## IRC, freenode, #hurd, 2013-07-01 + + I have read plenty of documents, and wrapped my head around + most Hurd concepts, but I still have not understood well what + capabilities are. + Mmm, which capabilities? + AIUI, the Hurd doesn't really have a notion of capabilites, just a + notion of owning a port right + From what I have understood (from the critique) they + reference ports so they objects can be referenced via them + (which provides processes a way for doing things) + ok, so we are talking about the same thing, I guess + ahh, that's cool. I thought there was more to the story that + I couldn't understand + + # Further Reading * [[Mach port|microkernel/mach/port]] diff --git a/community/gsoc/2013/hacklu.mdwn b/community/gsoc/2013/hacklu.mdwn new file mode 100644 index 00000000..d0185c60 --- /dev/null +++ b/community/gsoc/2013/hacklu.mdwn @@ -0,0 +1,617 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!toc]] + + +# IRC, freenode, #hurd, 2013-06-23 + + braunr: sorry for the late reply. Honestly to say, the school + works had taken most of my time these days. I haven't got any + siginificant progress now. I am trying to write a little debugger demo on + Hurd. + braunr: things goes more hard than I think, these are some + differences between ptrace() on Hurd and Linux. I am trying to solve + this. + + +# IRC, freenode, #hurd, 2013-06-24 + + this is my weekly report + http://hacklu.com/blog/gsoc-weekly-report1-117/. + and I have two main questions when I read the gdb source code. + 1/What is the S_exception_raise_request()? 2/what is the role of + ptrace in gdb port on Hurd? + hacklu: where did you see S_exception_raise_request? + in gdb/gnu-nat.c + ah, in gdb + yeah. and I have read the . is says the S_ + start means server stub. + yes + what happens is that gnu_wait keeps calling mach_msg + to get a message + then it passes that message to the various stubs servers + see just below, it calls exc_server, among others + and that's exc_server which ends up calling + S_exception_raise_request, if the message is an exception_raise request + exc_server is a mere multiplexer, actually + S_exception_raise_request is the implementation of the request + part (so one half of a typical RPC) of the Mach exception interface. + See gdb/exc_request.defs in GDB and include/mach/exc.defs in + Mach. + youpi: how gnu_wait pass one message to exc_server? in which + function? + in gnu_wait() + && !exc_server (&msg.hdr, &reply.hdr) + oh, I see this. + firstly I think it is a type check simply. + see the comment: "handle what we got" + The Hurd's proc server also is involved in the exception + passing protocol (see its source code). + tschwinge: I will check the source code later. is the exception + take place in this way: 1. the inferior call ptrace(TRACE_ME). 2.the gdb + call task_set_exception_port. 3. mach send a notification to the + exception port set before. 4. gdb take some action. + hacklu: Yes, that's it, roughly. The idea is that GDB replaces + a process' standard exception port, and replaces it "with itself", so + that when the process that is debugged receives and exception (by Mach + sending a exception_raise RPC), GDB can then catch that and act + accordingly. + hacklu: As for your other questions, about ptrace: As you can + see in [glibc]/sysdeps/mach/hurd/ptrace.c, ptrace on Hurd is simply a + wrapper around vm_read/write and more interfaces. + hacklu: As the GDB port for Hurd is specific to Hurd by + definition, you can also directly use these calls in GDB for Hurd. + ..., as it is currently done. + and in detail, the part 3 mach send a notification to the + excetption port is like this: gnu_wait get the message in mach_msg, and + then pass it to exc_serer by exc_server(),then exc_server call + S_exception_raise_request()? ? + tschwinge: yeah, I have see the ptrace.c. I was wonder about + nobody use ptrace in Hurd except TRACEME... + hacklu: Right about »and in detail, [...]«. + hacklu: It would be very good (and required for your + understanding anyway), if you could write up a list of things that + happens when a process (both under the control of GDB as well as without + GDB) is sent an exception (due to a breakpoint instruction, for example). + Let me look something up. + tschwinge: what's the function of exc_server? if I can get the + notification in mach_msg(). + to multiplex the message + i.e. decoding it, etc. up to calling the S_ function with the + proper parameters + exc_server being automatically generated, that saves a lot of code + That is generated by MIG from the gdb/exc_request.defs file. + You'll find the generated file in the GDB build directory. + I have wrote down the filenames. after this I will check that. + hacklu: I suggest you also have a look at the Mach 3 Kernel + Principles book, + . + This also has some explanation of the thread/task's exception + mechanism. + And of course, explains the RPC mechanism, which the exception + mechanism is built upon. + And then, really make a step-by-step list of what happens; this + should help to better visualize what's going on. + ok. later I will update this list on my blog. + hacklu: I cannot tell off-hand why GDB on Hurd is using + ptrace(PTRACE_TRACEME) instead of doing these calls manually. I will + have to look that up, too. + tschwinge: thanks. + hacklu: Anyway -- you're asking sensible questions, so it seems + you're making progress/are on track. :-) + tschwinge: there is something harder than I had thought, I haven't + got any meaningful progress. sorry for this. + hacklu: That is fine, and was expected. :-) (Also, you're + still busy with university.) + I will show more time and enthusiasm on this. + hacklu: Oh, and one thing that may be confusing: as you may + have noticed, the names of the same RPC functions are sometimes slightly + different if different *.defs files. What is important is the subsystem + number, such as 2400 in [GDB]/gdb/exc_request.defs (and then incremented + per each routine/simpleroutine/skip directive). + hacklu: Just for completeness, [hurd]/hurd/subsystems has a + list of RPC subsystems we're using. + And the name given to routine 2400, for example, is just a + "friendly name" that is then used locally in the code where the *.defs + file has been processed by MIG. + What a clumsy explanation of mine. But you'll get the idea, I + think. ;-) + hacklu: And don't worry about your progress -- you're making a + lot of progress already (even if it doesn't look like it, because you're + not writing code), but the time spent on understanding these complex + issues (such as the RPC mechanism) definitely counts as progress, too. + tschwinge: not clearly to got it as I am not sensitive with the + MIG's grammer. But I know, the exc is the routine 2400's alias name? + hacklu: I'd like to have you spend enough time to understand + these fundamental concepts now, and then switch to "hacking mode" (write + code) later, instead of now writing code but not understanding the + concepts behind it. + I have wrote a bit code to validate my understanding when I read + the soruce code. But the code not run. http://pastebin.com/r3wC5hUp + The subsystem directive [...]. As well, let me just point you + to the documentation: + , + MIG - THE MACH INTERFACE GENERATOR, chapter 2.1 Subsystem identification. + hacklu: Yes, writing such code for testing also is a good + approach. I will have to look at that in more detail, too. + * tschwinge guesses hacklu is probably laughing when seeing the years these + documents were written in (1989, etc.). ;-) + mach_msg make no sense in my code, and the process just hang. kill + -9 can't stop is either. + hacklu: do you understand why kill -KILL might not work now ? + braunr: no, but I found I can use gdb to attach to that process, + then quit in gdb, the process quit too. + maybe that process was waiting a resume. + something like that yes + iirc it's related to a small design choice in the proc server + something that put processes in an uninterruptible state when + being debugged + iirc ? + if i recall cl=orrectly + correctly* + like D status in linux? + or T + there has been a lot of improvements regarding signal handling in + linux over time so it's not really comparable now + but that's the idea + in ps, i see the process STAT is THumx + did you see that every process on the hurd has at least two + threads ? + no, but I have see that in hurd, the exception handler can't live + in the same context with the victim. so there must be at least two + threads. I think + hacklu: yes + that thread also handles regular signals + in addition to mach exceptions + (there are two levels of multiplexing in servers, first locating + the subsystem, then the server function) + hacklu: if what i wrote is confusing, don't hesitate to ask for + clarifications (i really don't intend to make things confusing) + braunr: I don't know what you say about the "multiplexing in + servers". For instance, is it means how to pass message from mach_msg to + exc_server in gnu_wait()? + hacklu: i said that the "message thread" handles both mach + exceptions and unix signals + hacklu: these are two different interfaces (and in mach terms, + subsystems) + hacklu: see hurd/msg.defs for the msg interface (which handles + unix signals) + hacklu: to handle multiple interfaces in the same thread, servers + need to first find the right subsystem + this is done by subsequently calling all demux functions until one + returns true + (finding the right server function is done by these demux + functions) + hacklu: see hurd/msgportdemux.c in glibc to see how it's done + there + it's short actually, i'll past it here : + return (_S_exc_server (inp, outp) || + _S_msg_server (inp, outp)); + hacklu: did that help ? + braunr: a bit more confusing. one "message thread" handles + exceptions and signals, means the message thread need to recive message + from two port. then pass the message to the right server which handle the + message. the server also should pick the right subsystem from a lot of + subsystems to handle the msg. is this ? + the message thread is a server thread + (which means every normal process is actually also a server, + receiving exceptions and signals) + there may be only two ports, or more, it doesn't matter much, the + port set abstraction takes care of that + so the message thread directly pass the msg to the right + subsystem? + not directly as you can see + it tries them all until one is able to handle the incoming message + i'm not sure it will help you with gdb, but it's important to + understand for a better general understanding of the system + ugly sentence + ah, I see. like this in gnu-nat.c if(!notify_server(&msg.hdr, + &reply.hdr) && !exc_server(&msg.hdr...) + yes + the thread just ask one by one. + be careful about the wording + the thread doesn't "send requests" + it runs functions + (one might be tempted to think there are other worker threads + waiting for a "main thread" to handle demultiplexing messages) + I got it. + the notify_server function is just run in the same context in + "message thread",and there is no RPC here. + yes + and the notify_server code is generater by mig automatically. + yes + + +# IRC, freenode, #hurd, 2013-06-29 + +[[!tag open_issue_documentation]] + + I just failed to build the demo on + this. http://walfield.org/pub/people/neal/papers/hurd-misc/ipc-hello.c + or, example in machsys.doc called simp_ipc.c + we don't use cthreads anymore, but pthreads + pinotree: em.. and I also failed to find the + in example of + that i don't know + maybe the code in that book out-of-date + hacklu: mig and mach ipc documentation is quite dated + unfortunately, and so are many examples floating around the net + + btw, I have one more question. when I read . I find this state: When an exception occurs in a thread, the + thread sends an exception message to + its exception port, blocking in the kernel waiting for the receipt + of a reply. It is + assumed that some task is listening to this + port, using the exc_serverfunction to decode the messages and + then call the + linked in catch_exception_raise. It is the job of + catch_exception_raiseto handle the exception and decide the course of + action for thread. + that says, it assumed another task to recieve the msg send to one + thread's exception port. why another task? + I remmebered, there are at least two threads in one task, one is + handle the exception stuffs. + there are various reasons + first is, the thread causing the exception is usually not waiting + for a message + next, it probably doesn't have all the info needed to resolve the + exception + (depending on the system design) + and yes, the second thread in every hurd process is the msg + thread, handling both mach exceptions and hurd signals + but in this state, I can't find any thing with the so called msg + thread + ? + if exist a task to do the work, why we need this thread? + this thread is the "task" + ? + the msg thread is the thread handling exceptions for the other + threads in one task + wording is important here + a task is a collection of resources + so i'm only talking about threads really + 14:11 < hacklu> assumed that some task is listening to this + this is wrong + a task can't listen + only a thread can + in you words, the two thread is in the same task? + yes + 14:32 < braunr> and yes, the second thread in every hurd process + is the msg thread, handling both mach exceptions and hurd signals + process == task here + yeah, I always think the two thread stay in one task. but I found + that state in . so I confuzed + s/confuzed/confused + statement you mean + if two thread stay in the same task. and the main thread throw a + exception, the other thread to handle it? + depends on how it's configured + the thread receiving the exceptions might not be in the same task + at all + on the hurd, only the second thread of a task receives exception + s + I just wonder how can the second thread catch the exception from + its containning task + forget about tasks + tasks are resource containers + they don't generate or catch exceptions + only threads do + for each thread, there is an exception port + that is, one receive right, and potentially many send rights + the kernel uses a send right to send exceptions + the msg thread waits for messages on the receive right + that's all + ok. if I divide zero in main thread, the kernel will send a msg to + the main thread's exception port. and then, the second thread(in the same + task) is waiting on that port. so he get the msg. is it right? + don't focus on main versus msg thread + it applies to all other threads + as well + otherwise, you're right + ok, just s/main/first + no + main *and* all others except msg + main *and* all others except msg ? + the msg thread gets exception messages for all other threads in + its task + (at least, that's how the hurd configures things) + got it. + if the msg thread throw exception either, who server for himself? + i'm not sure but i guess it's simply forbidden + i used gdb to attach a little progrom which just contains a divide + zero. and I only found the msg thread is in the glibc. + yes + where is the msg thread located in. + it's created by glibc + is it glibc/hurd/catch-exc.c? + that's the exception handling code, yes + there are some differences between the code and the state in . + state or statement ? + staement + which one ? + http://pastebin.com/ZTBrUAsV + When an exception occurs in a thread, the thread sends an exception + message to + its exception port, blocking in the kernel waiting for the receipt of a + reply. It is + assumed that some task is listening (most likely with mach_msg_server) + to this + port, using the exc_serverfunction to decode the messages and then + call the + linked in catch_exception_raise. It is the job of + catch_exception_raiseto handle the exception and decide the course of + action for thread. The state of the + blocked thread can be examined with thread_get_state. + what difference ? + in the code, I can't find things like exc_server,mach_msg_server + uh + ok it's a little tangled + but not that much + you found the exception handling code, and now you're looking for + what calls it + simple + see _hurdsig_fault_init + from that statemnet I thought there are another _task_ do the + exception things for all of the systems thread before you have told me + the task means the msg thread. + again + 14:47 < braunr> forget about tasks + 14:47 < braunr> tasks are resource containers + 14:47 < braunr> they don't generate or catch exceptions + 14:47 < braunr> only threads do + yeah, I think that document need update. + no + it's a common misnomer + once you're used to mach concepts, the statement is obvious + braunr: so I need read more :) + _hurdsig_fault_init send exceptions for the signal thread to the + proc server? + why come about _proc_ server? + no it gives the proc server a send right for signals + exceptions are a mach thing, signals are a hurd thing + the important part is + err = __thread_set_special_port (_hurd_msgport_thread, + THREAD_EXCEPTION_PORT, sigexc); + this one set the exception port? + yes + hm wait + actually no, wrong part :) + this sets the excpetion port for the msg thread (which i will call + the signal thread as mentioned in glibc) + but the comment above this line, Direct signal thread exceptions + to the proc server means what? + that the proc server handles exceptions on the signal thread + the term signal thread equals the term msg thread? + yes + so, the proc server handles the exceptions throwed by the msg + thread? + looks that way + feels a little strange. + why ? + this thread isn't supposed to cause exceptions + if it does, something is deeply wrong, and something must clean + that task up + and the proc server seems to be the most appropriate place from + where to do it + why need a special server to just work the msg thread? I don't + think that thread will throw exception frequentlly + what does frequency have to do with anything here ? + ok the appropriate code is _hurdsig_init + the port for receiving exceptions is _hurd_msgport + the body of the signal thread is _hurd_msgport_receive + aha, in the _hurd_msgport_receive I have finally found the + while(1) loop mach_msg_server(). + so the code is conform with the documents. + braunr: [21:18] what does frequency have to do with + anything here ? yes, I have totally understood your words now. thank you + very much. + :) + + +# IRC, freenode, #hurd, 2013-07-01 + + hi. this is my weekly + report. http://hacklu.com/blog/gsoc-weekly-report2-124/ welcome to any + comment + teythoon: I only get clear about the rpc stuff. seems a lot behind + my plan + good progress :) + I have wrote the details of the exception handle which was asked + by tschwing_ last week. Am I all right in my post? + hacklu: as far as I understand signals, yes :) + youpi: thanks for god, I am on the right way finally... :) + the mig book says simpleroutine is the one use to implement asyn + RPCs which doesn't expect an reply. But I have found a place to pass an + reply port to the RPC interface which has been declared as simpleroutine + hacklu: probably the simpleroutine hardcodes a reply port? + + hacklu: about _hurd_internal_post_signal, this is the hairiest part + of GNU/Hurd, signal handling + simply because it's the hairiest part of POSIX :) + you probably want to just understand that it implements the + POSIXity of signal delivering + i.e. deliver/kill/suspend the process as appropriate + I don't think you'll need to dive more + aha. + it will save a lot of time. + it seems like the wait_for_inferior() in gdb. which also has too + many lines and too many goto + hacklu: btw, which simpleroutine were you talking about ? + I forget where it is, I am finding it now. + which version of gdb are you looking the source of? + (in mine, wait_for_inferior is only 45 lines long) + I dont know how to pick the verison, I just use the git + version. maybe I give a wrong name. + ok + youpi:I remembered, my experience comes from here + http://www.aosabook.org/en/gdb.html. (All of this activity is managed by + wait_for_inferior. Originally this was a simple loop, waiting for the + target to stop and then deciding what to do about it, but as ports to + various systems needed special handling, it grew to a thousand lines, + with goto statements criss-crossing it for poorly understood + reasons.) + youpi: the simpleroutine is gdb/gdb/exc_request.defs + so there is indeed an explicit reply port + but simpleroutine is for no-reply use. why use reply port here? + AIUI, it's simply a way to make the request asynchronous, but still + permit an answer + ok, I will read the mig book carefully. + hacklu: as youpi says + a routine can be broken into two simpleroutines + that's why some interfaces have interface.defs, + interface_request.defs and interface_reply.defs files + nlightnfotis: in mach terminology, a right *is* a capability + the only thing mach doesn't easily provide is a way to revoke them + individually + braunr: Right. And ports are associated with the process + server and the kernel right? I mean, from what I have understood, if a + process wants to send a signal to another one, it has to do so via the + ports to that process held by the process server + and it has to establish its identity before doing so, so + that it can be checked if it has the right to send to that port. + yes + do process own any ports? or are all their ports associated + with the process server? + *processes + mach ports were intended for a lot of different uses + but in the hurd, they mostly act as object references + the process owning the receive right (one at most per port) + implements the object + processes owning send rights invoke methods on the object + use portinfo to find out about the rights in a task + (process is the unix terminology, task is the mach terminologyà + ) + i use them almost interchangeably + ahh yes, I remember about the last bit. And mach tasks have + a 1 to 1 association with user level processes (the ones associated with + the process server) + the proc server is a bit special because it has to know about all + processes + yes + +In context of [[open_issues/libpthread/t/fix_have_kernel_resources]]: + + hacklu: if you ever find out about either glibc or the proc server + creating one receive right for each thread, please let me know + + +# IRC, freenode, #hurd, 2013-07-07 + + how fork() goes? + see sysdeps/mach/hurd/fork.c in glibc' sources + when the father has two thread( main thread and the signal thead), + if the father call fork, then the child inmediatelly call exev() to + change the excute file. how many thread in the children? + For instance, the new execute file also have two thread. + will the exev() destroyed two threads and then create two new? + s/exev()/excv() + s/exev()/exec() :) + + what libhurduser-2.13.so does? + where can I find this source? + contains all the client stubs for hurd-specific RPCs + it is generated and built automatically within the glibc build + process + + and what is the "proc" server? + what handles in user spaces the processes + so if I call proc_wait_request(), I will go into the + S_proc_wait_reply? + thanks, I have found that. + + +# IRC, freenode, #hurd, 2013-07-08 + + hi, this is my weekly + report. http://hacklu.com/blog/gsoc-weekly-report3-137/ + this week I have met a lot of obstacles. And I am quite desired to + participate in this meeting. + hacklu: So from your report, the short version is: you've been + able to figure out how the things work that you were looking at (good!), + and now there are some new open questions that you're working on now. + hacklu: That sounds good. We can of course try to help with + your open questions, if you're stuck figuring them out on your own. + tschwinge: the most question is: what is the proc server? why need + to call proc_get_reqeust() before the mach_msg()? + and Is there exist any specific running sequence between father + and child task after fork()? And I found the inferior always call the + trace_me() in the same time(the trace me printf always in the same line + of the output log). which I have post in my report. + hacklu: The fork man-page can provide a high-level answer to + your Q3: »The child process is created with a single thread—the one that + called fork(). The entire virtual address space of the parent is + replicated in the child, including the states of mutexes, condition + variables, and other pthreads objects [...]« + hacklu: What happens in GNU Hurd is that the signal thread is + also "cloned" (additionally to the thread which called fork), but then it + (the signal thread) is re-started from the beginning. (So this is very + much equivalent to creating a new signal thread.) + hacklu: Then, upon exec, a new memory image is created/loaded, + replacing the previous one. [glibc]/sysdeps/mach/hurd/execve.c. What + actually happens with the existing thread (in particular, the signal + thread) I don't know off-hand. Then answer is probably found in + [glibc]/hurd/hurdexec.c -- and perhaps some code of the exec server + ([hurd]/exec/). + I have checked the status of my regiter mail to FSF. it says it + had arrived in USA. + hacklu: OK, good. + hacklu: This is some basic information about the observer_* + functions is GDB: + http://sourceware.org/gdb/current/onlinedocs/gdbint/Algorithms.html#index-notifications-about-changes-in-internals-57 + »3.10 Observing changes in gdb internals«. + tschwinge: not too clear. I will think this latter. and what is + the proc server? + hacklu: /hurd/proc, maps unix processes to mach threads afaiui + teythoon: question is, the mach_msg() will never return unless I + called proc_wait_request() first. + hacklu: sorry, I've no idea ;) + teythoon: :) + hacklu: I will have to look into that myself, too; don't know + the answer off-hand. + hacklu: In your blog you write proc_get_request -- but such a + functions doesn't seems to exist? + tschwinge: s/proc_get_request/proc_wait_request called in + gun_wait() [gnu-nat.c] + hacklu: Perhaps the wait man-page's description of WUNTRACED + gives a clue: »also return if a child has stopped [...]«. But it also to + me is not yet clear, how this relates to the mach_mag call, and how the + proc server exactly is involved in it. + I'm reading various source code files. + At least, I don't undestand why it is required for an exception + to be forwarded. + if I need to read the proc server source code? + I can see how it to become relevant for the case that GDB has + to be informed that the debugee has exited normally. + hacklu: Yeah, probably you should spend some time with that, as + it will likely help to get a clearer picture of the situation, and is + relevant for other interactions in GDB, too. + hacklu: By the way, if you find that pieces of the GDB source + code (especially the Hurd files of it) are insufficiently documented, + it's a very good idea, once you have figured out something, to add more + source code comments to the existing code. Or writed these down + separately, if that is easier. + which is the proc server? hurd/exec ? + that ok, I already comment things on my notes. + hacklu: [Hurd]/proc/ + hacklu: And [Hurd]/hurd/process*.defs + got it + hacklu: I'll have to experiment a bit with your HDebugger + example, but I'm out of time right now, sorry. Will continue later. + tschwinge: yep, the HDebugger has a problem, if you put the + sleep() after the printf in the just_print(), thing will hang. + tschwinge: and I am a little curious about how do you find my + code? I dont't remember I have mentioned that :) + tschwinge: I have post my gihub link in the last week report, I + found that. + hacklu: That's how I found it, yes. + tschwinge: :) diff --git a/community/gsoc/2013/nlightnfotis.mdwn b/community/gsoc/2013/nlightnfotis.mdwn new file mode 100644 index 00000000..43f9b14c --- /dev/null +++ b/community/gsoc/2013/nlightnfotis.mdwn @@ -0,0 +1,450 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!toc]] + + +# IRC, freenode, #hurd, 2013-06-29 + + so, how is your golang port going? + I just started working on it. I had been reading + documentation so far. Maybe over reading as people told me when I asked + for their feedback + but I will report on what I have done (technically tomorrow, + and post it in the mailing list too. + + Hey guys, what could possibly cause the following error + message when executing a program in the Hurd? "./dumper: Could not open + note: (system server) error with unknown subsystem" + My program is one that opens a file and dumps it into stdout + pinotree: the code I am using is the one present here + http://www.gnu.org/software/hurd/hacking-guide/hhg.html under paragraph + 6.1 + I investigated it a bit but can not find a lead. I seem to + have all the rights to open the file that I want to dump to stdout + what if you reset errno to 0 just after all the declarations in + main, before the instructions? + will check this out and get back to you. + sure :) + pinotree: Now it suggests that it can't get the number of + readable files, which the source suggests that is normal behavior. + Thanks for your assistance. + + +# IRC, freenode, #hurd, 2013-07-01 + + youpi: from my part I can report that I have started working + with the code, and doing as Thomas suggested. I was about to write my + report yesterday, but I am facing some build errors on the HURD, which I + would like to investigate further before I write my report. + that's why I decided to write it later in the day. + I don't think you have to wait + you can simply write in your report that you are having build + errors + ok. I will have it written and delivered later in the day. + braunr: that's cool. I think my reading has paid for + itself. And you may be pleased to know that I have gotten my hands dirty + with the code. I was about to write report yesterday, but some build + errors with the gcc (that I am investigating atm) are holding me + off. Will have that written later in the day. + don't hesitate to ask help about build errors + don't wait too much + you need to progress on what matters, and not be blocked by + secondary problems + I will see myself asking for help rather sooner than later, + but I would like to investigate it myself, and attempt to solve the + issues that occur to me before resort to bugging you guys. + sure + just not too long + too long being a day or so + these were my build_results on the hurd + they were linker errors + + https://gist.github.com/NlightNFotis/5896188#file-build_results + I am trying to build gcc on a linux 32 bit environment. It + also has some issues but not linker errors + will resolve them to see if the linker errors are + reproducible on linux + oh, lex stuff + should be easy enough + + +# IRC, freenode, #hurd, 2013-07-05 + + I have not made much progress, but I see myself working with + it. + I have managed to build gcc go on Linux + but Hurd seems to have some issues + it seems to randomly crash + the build process? + not quite randomly it seems to be though + yeah + I have noticed that there is a pattern + it does crash after some time + ^^ + but it doesn't crash at specific files + define crash + at some times it may crash during compiling insn-emit.c + (hello guys) + hi braunr :) + braunr: hey there! It does seem to keep on compiling this + file for a very long time (I have let it do so for 10, 20, 30 minutes) + but the result is the same + and it does so for different files for different build + options + ok so it doesn't crash + it just doesn't complete + is the virtual machine eating 100% cpu during that time ? + I can still type at the terminal, but I can't send a term + signal + I can report that QEMU does hold 100% of one core at that + time, (like it keeps processing) but there is no output on the terminal + ok + of course I can type at the terminal + but nothing happens + any idea of the size of the files involved ? + I am checking it out right now + before this goes any further, let me report on my + investigation + i expect that to be our classic writeback thread storm issue + initially, I thought it might be that it run out of memory + even though I know that compilation is not memory intensive, + rather, cpu intensive + anyway I increased the size of ram available to the vm + from 1024 mb to 1536 + that didn't seem to have any effect. The "crash" still + happens at the same time, at the same files + use freeze + not crash + crash is very misleading here + freeze it is then. + anyway + then it striked me that it might be that the hard disk size + (3gb) might be too small (considering the gcc git repo is 1gb+) + so I resized the qemu image to 8gb of hdd size + the new size is acknowledged by the vm + for gcc in debug mode? might still not be enough + but still it has no effect - it seems to follow its freezing + patterns + giving your work, i'd have not less than 15-20 + i'd use 32 + *given + but that's because i like power of twos + pinotree: thanks for the advice. Right now I was gonna + increase the swap size + according to vmstat in the hurd + swap size is 173 mb + don't know if it does have an impact + it may but before rushing + if you need swap, you're doomed anyway + consider swap highly unreliable on the hurd + please show the output of df -h on the file system you're using to + build + ideally, i'd recommend using separate / and /home file systems + it really improves reliability + I don't think it swaps to be honest; however that's + something that my mentor thomas had suggested (increasing swap size) so I + am gonna try it at some time. + or have a separate file system in a subdi and work on it + yes, /home or whatever suits you + just not / + braunr: pinotree: thanks both for your advice. Will do now, + and report on the results. + that's not all + 11:17 < braunr> please show the output of df -h on the file system + you're using to build + braunr: I am on it. Oh and btw, everytime I am forced to + close the vm (due to the freezes) when I restart it ext2 reports that the + file system was not cleanly unmounted and does some repair to some + files. I am trying to find an explanation for that, but I can think of + many things + well obviously + ext2 has no journaling + the file system was not cleanly unmounted since you restarted it + with a cold reset + braunr: df -h comes out with this: "df: cannot read table of + mounted file systems" + also, even if you manage to always shut down correctly, when + fsck runs because of the maximum mount count it'd find errors anyway (so + we have some bug) + nlightnfotis: df -h /path/to/build/dir + pinotree: not really bugs but it could be cleaned up + filesystem: - Size 2.8G Used 2.8G Avail 0 Use% 100% Mounted + on / + wow + nlightnfotis: see + that seems to explain many things + ^^ + thanks for that braunr! + you resized the disk, but not the partition and the file system + braunr: well, if something in ext2 (or its libs) leaves issues + in the fs, i'd call that a bug :> + yeah, that was utterly stupid of me + pinotree: they're not issues + nlightnfotis: be careful, mach needs a reboot every time you + change a partition table + nlightnfotis: important thing is that you found the issue :) + then only, you can use resize2fs + braunr: weird, I thought mach nowadays can reload the partition + tables? + braunr: doesn't d-i need that? + maybe a recent change i forgot + or maybe fdisk still reports the error although it's fine + in doubt, rebooting is still safe :p + or maybe youpi hacked it into d-is gnumach + i doubt it would be there for the installer only :) + if it's there, it's there + i just don't know it + braunr: teythoon: and everyone else that helped me. Thanks + you all guys. This was something that was driving me crazy. Will do all + that you suggested and report back on my status + + +# IRC, freenode, #hurd, 2013-07-08 + + tschwinge, I have managed to overcome most of the obstacles + I had initially faced with my project + but I still had some build errors, that's why I have not + reported yet. Wanna try to see if I can resolve them today, and write my + report in the afternoon. + nlightnfotis: So, from a quick look into the IRC backlog, it + was a "simple" out of disk space problem? %-) That happens. + nlightnfotis: And yes, GCC needs a lot of disk space. + nlightnfotis: What kind of build errors are you seeing now? + tschwinge, yeah I felt stupid at the time, but it didn't + actually strike me that the file system didn't see the extra space. Also + it took me some time to figure out that in order to mount the new + partition, I only had to edit /etc/fstab + always tried to mount it with the ext2 translator + and the translator kept dying + but it's all figured out now + the latest build errors I am seeing are these + nlightnfotis: o_O you used fstab and it worked? + yeah + nlightnfotis: that's unexpected from my perspective... + I only had to add the new partition into fstab + teythoon: I can pastebin my fstab if you wanna take a look + at it + tschwinge: these were my latest build errors + https://www.dropbox.com/s/b0pssdnfa22ajbp/build_results + nlightnfotis: I'm pretty sure that mount -a isn't done on hurd + w/o pinos runsystem.sysv + weird + tschwinge: I have also tried to build gcc with "make -w" + which from what I know supresses the errors that stopped compilation + but the weird thing is that gcc nearly took forever to build + nlightnfotis: could you do a showtrans /your/mountpoint? + teythoon: /hurd/ext2fs /dev/hd0s3 + nlightnfotis: ok, so you've set a passive translator and an + active is started on demand + it must be a passive translator + nlightnfotis: this is the hurd way of doing things, fstab is + unrelated + it seems to persist during reboots + yes, exactly + teythoon: my fstab if you wanna take a look + http://pastebin.com/ef94JPhG + after I added /dev/hd0s3 to fstab along with its mountpoint, + and restarting the hurd, only then I did manage to use that partition + before doing so I tried pretty much anything involving + mounting the partition and setting the ext2fs translator for it, but it + kept dying + of course it was a ext2 filesystem + err, perhaps adding to fstab simply triggered an fsck at reboot? + nlightnfotis: might have been that you needed to reboot mach so + that it picks up the new partition table + youpi: I thought this was fixed, the partition reloading I mean? + that is needed, yes + let me check + youpi: it could be, though, to be honest, my hurd system + does an fsck all the time at boot + how do you manage to do that w/o rebooting for d-i? + (I don't remember whether device busy is detected) + teythoon: by making all translators go away, iirc + nlightnfotis: btw, you have ~/gcc_new as mountpoint in your + fstab, pretty sure that this cannot work, the path has to be absolute and + no ~ expansion is done + tbh it does work, and it's weird + nlightnfotis: it works b/c of the passive translator you set, + not b/c of the fstab entry + teythoon: should I change it? + probably, yes + Well, that is probably not used anywhere. + tschwinge: not yet but soon ;) + Isn't /etc/fstab only consulted for fsck. + atm yes + Anyway, it is definitely a very good idea to have a partition + separate from the rootfs for doing actual work. + I think I described that in one of the first GSoC coodridation + emails. In the long one. + teythoon: Oh it struck me now! Is it because tilde expansion + is only happening in bash, but /etc/fstab is read before bash is + initialized? + nlightnfotis: Instead of fumbling around with partitioning of + disk images, it may be easier in your KVM/QEMU setup to simply add a new + disk using -hdb [file] (or similar). + nlightnfotis: Basically, yes. + nlightnfotis: fstab is not related with bash in any way + anyway, it shouldn't matter now, it seems to be working, and + I wouldn't like fiddling around with it and messing it up now. I will + continue with resolving the gcc issues. + But /etc/fstab has its very own "language" (layout), so tilde + expansion will never be done there. + nlightnfotis: df -h ~/gcc_new/ + tschwinge: size 24G Used: 4.2G Avail 18G + OK, that's fine. + As you can see on + , GCC + will easily need some GiB. + tschwinge: I have some questions about GCC: out of curiosity + how much time does it take to compile it on your machine? Because + yesterday I tried a -w (suppress warnings) build and it seemed to take + forever + mind you the vm has 1536 ram available (I have read + somewhere that it can utilise such an amount) and the vm is KVM enabled + without disabling g++, it can easily take hours + nlightnfotis: The build error is unexpected, because I had + addressed that issue in a recent patch. :-) + nlightnfotis: This is wrong: »checking whether setcontext + clobbers TLS variables... [...] yes«. Please check your sources, that + they correspond to the current version of the upstream + tschwinge/t/hurd/go branch. + nlightnfotis: Quoting from that wiki page: »This takes up + around 3.5 GiB, and needs roughly 3.5 h on kepler.SCHWINGE and 15 h on + coulomb.SCHWINGE.« The latter is my Hurd machine. + That's however with Java and Ada enabled, and a full + three-stages bootstrap. + ah, right, there's java & ada too + tschwinge: git branch (in the repo): master, + *tschwinge/t/hurd/go + in debian they are built separately + What I asked you to do is configure »--disable-bootstrap + --enable-languages=go«. + So that should be a lot quicker. + tschwinge: oh yes, everytime I have tried to compile gcc I + have done with these configurations + But still a few hours perhaps. + that's what I did yesterday too. + OK, good. :-) + A bootstrap build is a good way to check the just-built GCC for + sanity, but we expect that it is fine, as we concentrate on the GCC Go + port. + the only "extra" configuration yesterday was my "-w" flag to + make, because those errors were actually triggered by -Werror + Let me read up what make -w does. ;-) + ah, yes, d/w I have read and understood what the bootstrap + build is. Seems like we don't need it atm + afaik it suppresses all warnings + youpi: gcj no more + the way gcc builds, it does convert (some) warnings to + errors + Hmm. -w, --print-directory Print a message containing the + working directory before and after other processing. + youpi: doko folded gcj and gdc into gcc-4.8 to "workaround" + Built-Using + nlightnfotis: Ah, that'S configure --enable-werror or something + like that. + pinotree: right + yep, and -w suppresses it + (from what I have understood) + nlightnfotis: Are you thinking about make -k? + Yeah, I guess. + let me see what -k does + youpi: (just to make builds even more lightweight, eh) + yeah, -k should do too, I shall try it + But: if gcc -Werror fails, even with make -k, the build will + not be able to come to a successful end, because that one complation + artefact that failed will be missing. + so I shall try again with -w (supressed warnings) + Configureing with --disable-werror (or similar) will "help" if + -Werror is the default, and the build fails due to that. + from what I have understood these "errors" are not something + critical: it's only that function prototypes for these functions are + missing + I have seen the code there, and even "default" gcc generated + prototypes (from the first usage of the function) should do, so I can't + understand why it might be a serious problem if I tell gcc to skip that + point + nlightnfotis: Ah, now I see. You don't mean make -w, but + rather gcc -w: »-w Inhibit all warning messages.« + But really, there shouldn't be such warnings/errors that make + the build fail. + yeah + nlightnfotis: In your GCC sources directory, what does this + tell: git rev-parse HEAD + And, is the checkout clean: git status + The latter will take some time. + git status takes an awful amount of time + last I checked + but git rev-parse HEAD + produces this result: + 91840dfb3942a8d241cc4f0e573e5a9956011532 + OK, that's correct. So probably some of the checked out files + are not in a pristine state? + I shall run a git clean and see. If that doesn't work too, + maybe I shall reclone the repository? + there's nothing foreign to the repo that I have added, only + lib gmp, lib mpc and lib mpfr (and they are in their own folders inside + my gcc working directory) + nlightnfotis: You shouldn't need to do the latter if you + instead run: apt-get build-dep gcc-4.8 + I remember having done that inside the Hurd, but it always + resulted in an error from what I can recall + let me check this out + yes + nlightnfotis: Whenever you use Git on Hurd, pass the --quiet + flag, to avoid the rare but possible corruption issue described on + + and . + tschwinge: Forgive me for that. I will set up an alias + immediately. + nlightnfotis: I don't know if an alias is possible, because -- + I think -- you'll need to do things like: git fetch --quiet + So pass --quiet to subcommands. + oh. ok. + nlightnfotis: What you can also do, is shut down your Hurd VM, + and mount the disk image on GNU/Linux (mount with offset to get the right + partition), and then run a diff -ru against a Git clone done on + GNU/Linux, and see whether there are any unexpected differences outside + of the .git/ directory. + sounds like a plan. I will check this out today then :) + tschwinge: if all else fails, then recloning the repo with + --quiet passed should work, right? + Yes, that's probably the most straight-forward check to do. + Heh, yes to both these questions. :-) + nlightnfotis: Oh, you don't even have to re-clone, but rather + re-check-out the branch. + I was thinking of recloning just to bring the whole + repository to a pristine state + So something like (inside the source directory): rm -rf ./* + (remove any files, but leave .* in place, in particular the .git/ + directory), followd by git checkout -f HEAD --quiet + nlightnfotis: But before doing that, please do the diff first, + so that we know (hopefully) where the erroneous build results were coming + from. + considering the Copyright assignment files, I have sent them + from day 1 (that is the 20th of June). I have not heard anything about + those documents to date (sadly) + what's worst is that although I have a reference number to + track those documents, their (greek postal office) tracking service sucks + so badly, that one day it's offline, the next it suggests it can't find + the object in their database, the next it says it is still in the local + post office + let me check it out now + still nothing from their online service + let me call them + tschwinge: I called the post office regarding the copyright + papers. They told me that the same day (the 20th of June) it left from + Herakleion, Crete to Athens and the same day it must have left the + country heading towards the US. They also told me it takes about 1 week + for it to arrive. + nlightnfotis: OK, so probably waiting at the FSF office to be + processed. Let's allow for some more time. After all, this is not + critical for your progress. diff --git a/community/gsoc/project_ideas/mtab.mdwn b/community/gsoc/project_ideas/mtab.mdwn index d6f04385..0c0bb87b 100644 --- a/community/gsoc/project_ideas/mtab.mdwn +++ b/community/gsoc/project_ideas/mtab.mdwn @@ -11,6 +11,8 @@ License|/fdl]]."]]"""]] [[!meta title="mtab"]] +[[!tag open_issue_hurd]] + In traditional monolithic system, the kernel keeps track of all mounts; the information is available through `/proc/mounts` (on Linux at least), and in a very similar form in `/etc/mtab`. @@ -73,99 +75,3 @@ Possible mentors: Olaf Buddenhagen (antrik), Carl Fredrik Hammar (cfhammar) Exercise: Make some improvement to any of the existing Hurd translators. Especially those in [hurdextras](http://www.nongnu.org/hurdextras/) are often quite rudimentary, and it shouldn't be hard to find something to improve. - - -# Related Discussion - -## IRC, freenode, #hurd, 2013-04-17 - - thinking how to get the listing. traversing would be - ineffecient, trying to come up with something better - what listing ? - and traversing what ? - mtab - well i assumed so - be more precise please - when the translator is done initalized are written to /etc/mtab will be provided - by the translator, and when some one want to read the info just read it - this way if their is some credentials like ftp sites pass username can be - masked by the translator - if some trans dont want to list them, no need to write to - file | while unmounting (sorry i couldnt find the right word) , it - will pass the mount node address | will have special - structure to remove/add mounts example "a /mount-to /mount-from" = add - , "r /mount-to" = remove here "/mount-to" will be unique for every - mount - this have a draw back , we would have to trust trans for the - listed data | also "/mount-to" + "/mount-from" could be used a - combination for making sure that other trans unable remove others trans - mount data - sorry but "also "/mount-to" + "/mount-from" could be used a - combination for making sure that other trans unable remove others trans - mount data" this is a bad idea if we had to print the whole thing - braunr, whats ur opinion? - you don't need a mtab to "unmount" things on hurd - kuldeepdhaka: hum, have you read the project idea ? - - http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/mtab/ - A more promising approach is to have mtab exported by a special - translator, which gathers the necessary information on demand. This could - work by traversing the tree of translators, asking each one for mount - points attached to it. - pinotree, not to unmount, i mean is to remove the - - for a first implementation, i'd suggest a recursive traversal of - root-owned translators - braunr, hum, but it did stated it as inefficient - where ? - para 5 , line 3 - and line 6 - no - traversing "all" nodes would be inefficient - translators which host the nodes of other translators could - maintain a simple list of active translators - ext2fs, e.g. (if that's not already the case) could keep the list - of the translators it started - we can already see that list with pstree for example - but this new list would only retain those relevant for mtab - i.e. root-owned ones - i would not limit to those though - and then filter on their type (e.g. file system ones) - pinotree: why ? - this way you could have proper per-user /proc/$pid/mounts info - we could also very easily have a denial of service - but how will the mount point and source point will be - listed? - they're returned by the translator - k - you ask /, it returns its store and its options, and asks its - children recursively - a /home translator would return its store and its options - etc.. - each translator would build the complete path before returning it - sort of, it's very basic - but that would be a very hurdish way to do it - shall /etc/mtab should be made seek-able and what should be - the filesize? content are generated on demand so, it could arise problem - (fsize:0 , seek-able:no), ur opinions? - kuldeepdhaka: it should have all the properties of a regular file - the filesize would be determined after it's generated - being empty doesn't imply it's not seekable - content is generated on demand so, could cause problem while - seeking and filesize, shall i still program as regular file? - in two different read, it could generate different content, - though same seek pos is used... - what ? - the content is generated on open - ooh, ok - - -## IRC, freenode, #hurd, 2013-06-04 - - how to see list of all connected translators? - you can't directly - you can use ps to list processes and guess which are translators - (e.g. everything starting with /hurd/) - a recursive call to obtain such a list would be useful - similar to what's needed to implement /proc/mounts diff --git a/community/gsoc/project_ideas/mtab/discussion.mdwn b/community/gsoc/project_ideas/mtab/discussion.mdwn new file mode 100644 index 00000000..0e322c11 --- /dev/null +++ b/community/gsoc/project_ideas/mtab/discussion.mdwn @@ -0,0 +1,907 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +# IRC, freenode, #hurd, 2013-04-17 + + thinking how to get the listing. traversing would be + ineffecient, trying to come up with something better + what listing ? + and traversing what ? + mtab + well i assumed so + be more precise please + when the translator is done initalized are written to /etc/mtab will be provided + by the translator, and when some one want to read the info just read it + this way if their is some credentials like ftp sites pass username can be + masked by the translator + if some trans dont want to list them, no need to write to + file | while unmounting (sorry i couldnt find the right word) , it + will pass the mount node address | will have special + structure to remove/add mounts example "a /mount-to /mount-from" = add + , "r /mount-to" = remove here "/mount-to" will be unique for every + mount + this have a draw back , we would have to trust trans for the + listed data | also "/mount-to" + "/mount-from" could be used a + combination for making sure that other trans unable remove others trans + mount data + sorry but "also "/mount-to" + "/mount-from" could be used a + combination for making sure that other trans unable remove others trans + mount data" this is a bad idea if we had to print the whole thing + braunr, whats ur opinion? + you don't need a mtab to "unmount" things on hurd + kuldeepdhaka: hum, have you read the project idea ? + + http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/mtab/ + A more promising approach is to have mtab exported by a special + translator, which gathers the necessary information on demand. This could + work by traversing the tree of translators, asking each one for mount + points attached to it. + pinotree, not to unmount, i mean is to remove the + + for a first implementation, i'd suggest a recursive traversal of + root-owned translators + braunr, hum, but it did stated it as inefficient + where ? + para 5 , line 3 + and line 6 + no + traversing "all" nodes would be inefficient + translators which host the nodes of other translators could + maintain a simple list of active translators + ext2fs, e.g. (if that's not already the case) could keep the list + of the translators it started + we can already see that list with pstree for example + but this new list would only retain those relevant for mtab + i.e. root-owned ones + i would not limit to those though + and then filter on their type (e.g. file system ones) + pinotree: why ? + this way you could have proper per-user /proc/$pid/mounts info + we could also very easily have a denial of service + but how will the mount point and source point will be + listed? + they're returned by the translator + k + you ask /, it returns its store and its options, and asks its + children recursively + a /home translator would return its store and its options + etc.. + each translator would build the complete path before returning it + sort of, it's very basic + but that would be a very hurdish way to do it + shall /etc/mtab should be made seek-able and what should be + the filesize? content are generated on demand so, it could arise problem + (fsize:0 , seek-able:no), ur opinions? + kuldeepdhaka: it should have all the properties of a regular file + the filesize would be determined after it's generated + being empty doesn't imply it's not seekable + content is generated on demand so, could cause problem while + seeking and filesize, shall i still program as regular file? + in two different read, it could generate different content, + though same seek pos is used... + what ? + the content is generated on open + ooh, ok + + +# IRC, freenode, #hurd, 2013-06-04 + + how to see list of all connected translators? + you can't directly + you can use ps to list processes and guess which are translators + (e.g. everything starting with /hurd/) + a recursive call to obtain such a list would be useful + similar to what's needed to implement /proc/mounts + + +# IRC, freenode, #hurd, 2013-06-25 + +In context of [[microkernel/mach/mig/documentation/structured_data]]. + + should I go for an iterator like interface instead? + btw, what's the expected roundtrip time? + don't think that way + consider the round trip delay as varying + y, is it that bad? + no + but the less there is the better + we think the same with system calls even if they're faster + the delay itself isn't the real issue + look at how proc provides information + (in procfs for example) + + +## IRC, freenode, #hurd, 2013-06-26 + + so tell me about the more hurdish way of dealing with that issue + creating a specialized translator for this? + 11:45 < pinotree> there's also + http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/mtab/ + about that topic + you need to avoid thinking with centralization in mind + the hurd is a distributed system in practice + i think proc is the only centralized component in there + braunr: would having an mtab translator and having fs + translators register to thae be acceptable? + that* + teythoon: why do you want to centralize it ? + translators already register themselves when they get attached to + a node + we don't want an additional registration + have you read the link we gave you ? + I did and I got the message, but isn't the concept of + /proc/mounts or a mtab file already a centralized one? + that doesn't mean the implementation has to be + and no, i don't think it's centralized actually + it's just a file + you can build a file from many sources + or if we do it your way recursing on fs translators *but* + restricting this to root owned translators also suffering from + centralization wrt to the root user? I mean the concept of all mounted + filesystems does not apply cleanly to the hurd + i don't understand + restricting to the root user doesn't mean it's centralized + trust has nothing to do with being centralized + I guess I'm not used to thinking this way + teythoon: i guess that's the main reason why so few developers + work on the hurd + also the way fs notification is done is also centralized, that + could also be done recursively + what doyou call fs notification ? + and the information I need could just be stuffed into the same + mechanism + fs translators being notified of system shutdown + right + that gets a bit complicated because the kernel is also a + centralized component + it knows every memory object and their pagers + it manages all virtual memory + there are two different issues here + syncing memory and shutting down file systems + the latter could be done recursively, yes + i wonder if the former could be delegated to external pagers as + well + teythoon: but that's not the focus of your work aiui, it would + take much time + sure, but missing an mtab file or better yet /proc/mounts could + be an issue for me, at least a cosmetic one, if not a functional one + i understand + and hacking up a quick solution for that seemed like a good + exercise + i suggest you discuss it with your mentors + they might agree to a temporary centralized solution + although i don't think it's much simpler than the recursive one + braunr: would that be implemented in libdiskfs and friends? + teythoon: i'm not sure, it might be a generic fs operation + libnetfs etc.. are also mount points + so where would it go if it was generic? + libfshelp perhaps + translator startup is handled in start-translator-long.c, so in + case a startup is successful, I'd add it to a list? + i'd say so, yes + would that cover all cases, passive and active translators? + that's another question + do we consider passive translators as mounted ? + ah, that was not what i meant + i know + but it's related + start b/c of accessing a passive one vs. starting an active one + using settrans + start_translator_xxx only spawn active translators + it's the same + ok + the definition of a passive translator is that it starts the + active translator on access + yeah I can see how that wouldn't be hard to implement + i think we want to include passive translators in the mount table + so registration must happen before starting the active one + so it's a) keeping a list of active translators and b) add an + interface to query fs translators for this list and c) an interface to + query mtab style information? + keeping a list of all translators attached + and yes + well + a is easy + b is the real work + c would be procfs using b + oh? I thought recursing on the translators and querying info + would be separate operations? + why so ? + the point is querying recursively :) + and when i say recursively, it's only a logical view + ok, yes, it can be implemented this way, so we construct the + list while recursing on the translators + i think it would be better to implement it the way looking up a + node is done + in a loop, using a stack? + iteratively + a translator would provide information about itself (if + supported), and referrences to translators locally registered to it + could you point me to the node lookup? + ah, yes + eg., you ask /, it tells you it's on /dev/hd0, read-write, with + options, and send rights to /home, /proc, etc.. + well rights, references + it could be the path itself + rights as in a port to the translators? + i think the path would be better but i'm not sure + it would also allow you to check the permissions of the node + before querying + path would be nicer in the presence of stacked translators + and obviously you'd have the path right away, no need to provide + it in the reply + true + + braunr: if we want to list passive translators (and I agree, we + should), it isn't sufficient to touch libfshelp, as setting a passive + translator is not handled there, only the startup + teythoon: doesn't mean you can't add something there that other + libraries will use + so yes, not sufficient + + +## IRC, freenode, #hurd, 2013-06-29 + + braunr: diskfs_S_fsys_set_options uses diskfs_node_iterate to + recurse on active translators if do_children is given + braunr: I wonder how fast that is in practice + braunr: if it's fast enough, there might not even be a need for + a new function in fsys.defs + and no need to keep a list of translators for that reason + braunr: if it's not fast enough, then diskfs_S_fsys_set_options + could use the list to speed this up + teythoon: on all nodes ? + braunr: i believe so, yes, see libdiskfs/fsys-options.c + teythoon: well, if it really is all node, you clearly don't want + that + + +## IRC, freenode, #hurd, 2013-07-01 + + I've ment to ask, the shiny new fsys_get_translators interface, + should it return the options for the queried translator or not? + i don't think it should + ok + let's walk through why it shouldn't + may I assume that the last argument returned by fsys_get_options + is the "source"? + how would you know these options ? + the source ? + I wouldn't actually + yes, you wouldn't + you'd have to ask the translators for that + so the only thing you can do is point to them + well, the device to include in the mtab file + and the client asks + i don't know fsys_get_options tbh + well, both tmpfs and ext2fs print an appropriate value for + "device" as last argument + looks like a bad interface to me + options should be options + there should be a specific call for the device + but if everyone agrees with the options order, you can do it that + way for now i guess + one that could be used to recreate the "mount" using either + mount or settrans + just comment it where appropriate + I thought that'd be the point? + ? + % fsysopts tmp + /hurd/tmpfs --writable --no-inherit-dir-group --no-sync 48K + where is the device ? + % settrans -ca tmp $(fsysopts tmp) + 15:56 < teythoon> well, both tmpfs and ext2fs print an appropriate + value for "device" as last argument + 48K + i don't see it + really ? + yes + what about ext2fs ? + hm ok i see + % fsysopts / + ext2fs --writable --no-inherit-dir-group --sync=10 + --store-type=typed device:hd0s1 + i don't think you should consider that as devices + but really translator specific options + agree + I don't ;) + b/c the translator calling convention is hardcoded in the mount + utility + ? + I think it's reasonable to assume that this mapping can be + reversed + theorically you can write a translator that takes no arguments, + but just options + the 48K string for tmpfs is completely meaningless + in fstab, it should be none + "tmpfs" + the linux equivalent is the size option + no, none + it's totally ignored + and it's recommended to set none rather than the type to avoid + confusion + u sure? + % settrans -cga tmp /hurd/tmpfs --mode=666 6M + % settrans -cga tmp /hurd/tmpfs --mode=666 6M + % fsysopts tmp + /hurd/tmpfs --writable --no-inherit-dir-group --no-sync 6M + i've not explained myself clearly + it's not ignored by the translator + but in fstab, it should be in the options field + it's not the source + clearly not + ah + now i'm talking about fstab, but iirc the format is similar in + mtab/mounts + close, but not the same + yes, close + ok, so I'll put a method into libfshelp so that translators can + explicitly set a device and patch all existing translators to do so? + teythoon: what i meant is that, for virtual vile systems (actually + file systems with no underlying devices), the device field is normally + ignored + teythoon: why do you need that for exactly + right + do they even have a "device" field? + (i can see why but i'd like more visibility) + pinotree: not yet + pinotree: that's what he wants to add + but i'd like to see if there is another way to get the information + 16:05 < braunr> teythoon: why do you need that for exactly + well if I'm constructing a mtab entry I need a value for the + device field + do we actually need it to be valid ? + not necessarily I guess + discuss it with your mentors then + it has to be valid for e2fsck checks etc. + doesn't e2fsck check fstab actually ? + i.e. actually for the cases where it's trivial + fstab doesn't tell it whether it's mounted + I mean fsck checking whether it's mounted + not fsck -a + oh + couldn't we ask the device instead ? + looks twisted too + that'd mean patching a lot of applications which do similar checks + yes + teythoon: propose an interface for that with your mentors then + yeah, but couldn't you lay it out a little, I mean would it be + one procedure or like three? + 16:04 < teythoon> ok, so I'll put a method into libfshelp so that + translators can explicitly set a device and patch all existing + translators to do so? + ok + why three ? + no, I mean when adding stuff to fsys.defs + i understood that + but why three ? :) + it'd be more generic + really ? + please show a quick example of what you have in mind + i honestly don't know, thus I'm asking ;) + well first, this device thing bothers me + when you look at how we set up our ext2fs translators, you can see + they use device:xxx + and not /dev/xxx + but ok, let's assume it's harmless + ok, but isn't the first way actually better? + i think it ends up being the same + ideally, that's what we want to use as device path + but you can recreate a storeio translator using the device:xxx + info, the node is useless for that + so that we don't need to explicitely set it + ? + what do you mean ? + well, fsysopts / tells currently tells me device:hd0s1 + for /, there isn't much choice + /dev isn't there yet + ah, got it + that's why it differs... + differs ? + from what ? + other ext2fs translators are set the same way by the debian + installer for example + % fsysopts /media/scratch + /hurd/ext2fs --writable --no-inherit-dir-group /dev/hd1s1 + here it uses the path to the node + that's weird + was that done by the debian installer ? + ah no, that was me + :p + $ fsysopts /home + /hurd/ext2fs --writable --no-inherit-dir-group --store-type=device + hd0s6 + so as you can see, it's not that simple to infer the device path + oho, yet another way ;) + right then + isn't device:hd0s1 as shortcut for specifying the store type, as + done with --store-type=device hd0s1? + but perhaps we don't need to + yes it is + iirc it's something libstore does, per-store prefixes + ah that sucks + teythoon: you may need to normalize those strings + so that they match what's in fstab + i.e. unix /dev paths + otherwise e2fsck still won't be able to find the translators + mounting the device + well, if it's mounted actually + it just needs to find the matching line in mtab aiui + so perhaps a libfshelp function for that, yes + braunr: so you suggest adding a normalizing function to + libfshelp that creates a /dev/path? + yes + used by the call you intend to add, which returns that device + string as found in fstab + found in fstab? so this would only work for translators managed + by fstab? + no + ah + a string like the ones found in fstab? + yes + so that fsck and friends are able to know whether a device is + mounted or not + i don't see any other purpose for that string in mtab + you'd take regular paths as they are, convert device:xxx to + /dev/xxx, and return "none" for the rest i suppose + ok + i'm not even sure it's right + youpi: are you sure it's required ? + well it's a start and I think it's not too much work + aiui, e2fsck may simply find the mount point in fstab, and ask the + translator if it's mounted + we can refine this later on maybe? + or rather, init scripts, using mountpoint, before starting e2fsck + teythoon: sure + there's this mountpoint issue... I need to run fsysopts / + --update early in the boot process + otherwise the device ids returned by stat(2)ing / are wrong and + mountpoint misbehaves + i guess b/c it's the rootfs + device ids ? + % stat / | grep Device + Device: 3h/3d Inode: 2 Links: 22 + do you mean the major/minor identifiers ? + I do. if I don't do the --update i get seemingly random values + i guess that's expected + we don't have major/minor values + well, they're emulated + well, if that's fixable, that'd be really nice ;) + we'll never have major/minor values + yeah, I understand that + but they could be fixed by MAKEDEV when creating device nodes + but not having to call fsys_set_options on the rootfs to get the + emulation up to speed + try doing it from grub + not sure it's possible + but worth checking + by means of an ext2fs flag? + yes + if there is one + i don't know the --update flag, is it new from your work ? + braunr: no, it's been there before. -oremount gets mapped to + that + it's documented by fsysopts, but not by the ext2fs translators + libdiskfs source says something about flushing buffers iirc + -s + what does it do ? + teythoon: ok + braunr: so the plan is to automatically generate a device path + from the translators argz vector but to provide the functionality so + translators can set a more appropriate value? did I get the last part of + the discussion right? + not set, return + yeah return from the procedure but settable using libfshelp? + why settable ? + you'd have a fsys call to obtain the dev string, and the server + side would call libfshelp on the fly to obtain a normalized value and + return it + ah, make a function overrideable that returns an appropriate + response? + overrideable ? + like netfs_append_args + you wouldn't change the command line, no + isn't that done using weak references or something? + no I know + sorry i'm lost then + never mind, I'll propose a patch early to get your feedback + braunr: am I sure that _what_ is required? + the device? + e2fsck surely needs it, yes + a valid device path, yes + it can't rely only on fstab + yes + since users may mount things by hand + i've used strace on it and it does perform lookups there + (although i also saw uuid magic that i guess wouldn't work yet on + the hurd) + + +## IRC, freenode, #hurd, 2013-07-03 + + I added a procedure to fsys.defs, added a server stub to my + tmpfs translator and wrote a simple client, but something hasn't picked + up the new message yet + % ./mtab tmp + ./mtab: get_translator_info: (ipc/mig) bad request message ID + I guess it's libhurduser.so from glibc, not sure though... + glibc would only have the client calls + what is "% ./mtab tmp" ? + mtab is my mtab tool/soon to be a translator testing thing, tmp + is an active tmpfs with the appropriate server stub + so mtab has the client call, right ? + yes + then tmpfs doesn't + so what to do about it? + i set LD_LIBRARY_PATH to my hurd builds lib dir, is that + preserved by settrans -a? + not really + not at all + there is a wiki entry about that iirc + http://darnassus.sceen.net/~hurd-web/hurd/debugging/translator/ + yeah, I read it too once + ah + on the other hand, using export to set the environment should do + the work + yes, that did the trick, thanks :) + * teythoon got his EOPNOPSUPP... *nomnomnom + ? + same error ? + well I stubbed it out + oh + no, that's what I've been expecting ;) + great + :) + yes that's better than "mig can't find it" + braunr: in that list of active and passive translators that will + have to be maintained, do you expect it should carry more information + other than the relative path to that translator? + like what ? + dunno, maybe like a port to any active translator there + should we care if any active translator dies and remove the + entry if there's no passive translator that could restart it again? + don't add anything until you see it's necessary or really useful + yes + think of something like sshfs + when you kill it, it's not reported by mount any more + well, for a dynamically allocated list of strings I could use + the argz stuff, but if we'd ever add anything else, we'd need a linked + list or something, maybe a hash table + yes, I thought that'd be useful + use libihash for no + now + braunr: but what would I use as keys? the relative path should + be unique (unless translators are stacked... hmmm), but that's the value + I'd like to store and ihash keys are pointers + stacked translators are an kinda interesting case for mtab + anyways... + why not store the string address ? + i suppose that, for stacked translators, the code querying + information would only return the topmost translator + since this is the one which matters for regular clients (if i'm + right) + wouldn't that map strings that are equal but stored at different + locations to different values? + that'd defeat the point + I suppose so, yes + then add a layer that looks for existing strings before adding + the list should normally be small so a linear lookup is fine + yeah sure, but then there's little advantage of using ihash in + the first place, isn't it? + over what ? + over not using it at all + how would you store the list then ? + it's either ll or ll+ihash + uh no + let me check + there is ihash_iterate + so you don't need a linked list + so how do I store my list of strings to deduplicate the keys? + you store pointers + and on addition, you iterate over all entries, making sure none + matches the new one + and if it does, you replace it i guess + depending on how you design the rest + in an dynamically allocated region of memory? + i don't understand + your strings should be dynmaically allocate, yes + no the array of char * + your data structure being managed by libihash, you don't care + about allocation + what array ? + ah, got it... + right. + there is only one structure here, an ihash of char * + yes, I got the picture ;) + goo + d + actually, the lookup wouldn't be linear since usually, hash tables + have stale entries + heh... what forest?!? + but that's ok + teythoon: ? + the one I couldn't make out b/c of all the trees... + ? + ah, it's not important. there is this saying over here, not sure + if there's an english equivalent + ok got it + we have the same in french + I ran into a problem with my prototype + if an translator is set in e. g. diskfs_S_file_set_translator, + how do I get the path to that node? + I believe there cannot be a way to do that, b/c the mapping is + not bijective + it doesn't have to be + ok, so how do I get *a* path for this node? + that's another question + do you see how the node is obtained ? + np = cred->po->np; + yes + the translation occurred earlier + you need to find where + then perhaps, you'll need to carry the path along + or if you're lucky, it will still be there somewhere + the translation from path to node? + yes + doesn't that happen in the client? and the client hands a file_t + to the file_set_translator routine? + the relative lookup can't happen in the client + the server can (and often does) retain information between two + RPCs + uh, I can access information from a previous rpc? is that + considered safe? + think of open() then read() + a simple int doesn't carry enough information + that's why it's a descriptor + ah, the server retains some state, sure + what it refers to is the state retained between several calls + the object being invoked by clients + teythoon: what is the "passive" parameter passed to + diskfs_S_file_set_translator ? + braunr: argz vector of the passive translator + so it is a name + but we also want active translators + and what is active ? + not the name of the node though + active is the port (?) to the active translator + I guess + fsys_t, looks that way yes + i suppose you could add the path to the peropen structure + ok + see diskfs_make_peropen + braunr: but translation happens in dir_lookup + in all places I've seen diskfs_make_peropen used, the path is + not available + why did you point me to diskfs_make_peropen? + s/dir_lookup/diskfs_lookup/ + diskfs_lookup operates on struct node, so the path would have to + be stored there, right? + teythoon: dir_lookup should call diskfs_make_peropen + at least diskfs_S_dir_lookup does + and the path is present there + braunr: right + + hrm... I added a path field to struct peropen and initialize it + properly in diskfs_make_peropen, but some bogus values keep creeping in + :/ + first of all, make it a dynamically allocated string + it is + not a fixed sized embedded array + good + yes + if you really need help debugging what's happening, feel free to + post your current changes somewhere + there is a struct literal in fsys-getroot.c, but i fixed that as + well + % ./mtab tmp + none tmp ../tmpfs/tmpfs writable,no-inherit-dir-group,no-sync 0 + 0 + none tmp/bar ../tmpfs/tmpfs + writable,no-inherit-dir-group,no-sync 0 0 + none tmp/foo ../tmpfs/tmpfs + writable,no-inherit-dir-group,no-sync 0 0 + none tmp/foo/bar ../tmpfs/tmpfs + writable,no-inherit-dir-group,no-sync 0 0 + :) + + +## IRC, freenode, #hurd, 2013-07-10 + + btw, I read getcwd.c and got the idea + however this situation is different afaict + getcwd has a port to the current working directory, right? + so they can do open_dir with .. as relative path + but all I've got is a port referencing the node the translator + is being attached to + s/open_dir/dir_lookup/ + and that is not necessarily a directory, so dir_lookup fails + with not a directory + as far as I can see it is not possible to get the directory a + node is in from a port referencing that node + dir_lookup has to be handled by all nodes, not just directories + but file nodes only support "looking up" the empty string + not empty, but null: + This call is required to be supported by all files (even + non-directories) if the filename is null, and should function in that + case as a re-open of the file. */ + why do you want the directory ? + 10:40 < teythoon> as far as I can see it is not possible to get + the directory a node is in from a port referencing that node + to readdir(3) it and figure out the name of the node the + translator is bound to + similar to what getcwd does + that's out of the question + wasn't that was youpi was suggesting? + you may have a lot of nodes in there, such a lookup shouldn't be + done + i didn't see that detail + "│ Concerning storing the path, it's a bit sad to have to do + that, and + │ it'll become wrong if one moves the mount points. Another + way would + │ be to make the client figure it out by itself from a port to + the mount + │ point, much like glibc/sysdeps/mach/hurd/getcwd.c. It'll be + slower, but + │ should be safer. The RPC would thus return an array of + ports to the + │ mount points instead of an array of strings. + yes i remember that + but i didn't understand well how getcwd work + s + another scalability issue + not a big one though, we rarely have translators in directories + with thousands of nodes + so why not + teythoon: do it as youpi suggested + well if you can + eh + if not, i don't know + 10:47 < teythoon> │ it'll become wrong if one moves the mount + points. Another way would + yes, I know... :/ + well, I'm not even sure it is possible to get the directory a + node is in from the port referencing the node + as in, I'm not sure if the information is even there + b/c a filesystem is a tree, directories are nodes and files are + leafs + all non-leaf nodes reference their parent to allow traversing + the tree starting from any directory + but why would a leaf reference its parent(s - in case of + hardlinks)? + uh, for the same reason ? + sure, it would be nice to do that, but I dont think this is + possible on unixy systems + ? + you cannot say fchdir(2) to a fd that references a file + do you mean /path/to/file/../ ? + yes + only that /path/to/file is given as fd or port + when i pasted + 10:49 < braunr> 10:47 < teythoon> │ it'll become wrong if one + moves the mount points. Another way would + i was actually wondering if it was true + ah + why can't the path be updated at the same time ? + it's a relative path anyway + completely managed by the parent translator + ah + right + it's still kind of hacky, but I cannot see how to do this + properly + hacky ? + but yes, updating the path should work I guess + or sad + what i find hacky is to set translators in two passes + otherwise we'd only keep the translator paths + not all paths + true + but then, it only concerns open nodes + and again, there shouldn't be too many of them + so actually it's ok + braunr: I understand the struct nodes are cached in libdiskfs, + so wouldn't it be easier to attach the path to that struct instead of + struct peropen so that all peropen objects reference the same node + object? + so that the path can be updated if anyone dir_renames it + *all peropen objects derived from the same file name that is + teythoon: i'm not sure + nodes could be real nodes (i.e. inodes) + there can be several paths for the same inode + braunr: I'm aware of that, but didn't we agree the other day + that any path would do? + i don't remember we did + i don't know the details well, but i don't think setting a + translator on a hard link should set the translator at the inode level + on the other hand, if a new inode is created to replace the + previous one (or stack over it), then storing the path there should be + fine + braunr: I don't think I can update the paths if they're stored + in the peropen struct + how would I get a reference to all those peropen objects? + ? + first, what's the context when you talkb about updating paths ? + well, youpi was concerned about renaming a mount point + and you implied that this could be managed + can we actually do that btw ? + what? + renaming a mount point + yep, just tried + i mean, on a regular unix system like linux + $ mv test blah + mv: cannot move `test' to `blah': Device or resource busy + (using sshfs so YMMV) + do you have anything (shells, open files, etc) inside it? + no + i'll try with an empty loop-mounted ext4 + I was testing on the Hurd, worked fine there even with a shell + inside + same thing + i consider it a bug + we may want to check what posix says about it + o_O + and decide not to support renaming + why? + start a discussion in ml, maybe roland can chime in + it complicates things + ah yes + sure, but I can move or rename a directory, why should it be + different with a mount point? + because it's two of them + they're stacked + if we do want to support that, we must be very careful about + atomically updating all the stack + ok + braunr: I'm trying to detect dying translators to remove them + from the list of translators + what port can I use for that purpose? + if I use the bootstrap port, can I then use the same method as + init/init.c uses? just defining a do_mach_notify_dead_name function and + the multiplexer will call this? + teythoon: possibly + braunr: we'll see shortly... + I get KERN_INVALID_CAPABILITY indicating that my bootstrap port + is invalid + when calling mach_port_request_notification to get the dead name + notification I mean + is the translator already started when you do that ? + yes, at least I think so, I'm hooking into + diskfs_S_file_set_translator and that gets an active translators port + also the mach docs suggests that the notification port is + invalid, not the name port referencing the translator + i guess it shouldn't + oh + please show the code + but beware, if the translator is started, assume it could die + immediately + braunr: http://paste.debian.net/15371/ line 87 + teythoon: notify can't be bootstrap + what do you have in mind when writing this ? + i'm not sure i follow + I want to be notified if an active translator goes away to + remove it from the list of translators + ok but then + create a send-once right + and wait on it + also, why do you want to be notified ? + isn't this already done ? + or can't do it lazily on access attempt ? + +you + in the client? + in the parent server + what happens currently when a translator dies + is the parent notified ? + or does it give an invalid right ? + ah, i think so + then you don't need to do it again + right, I overlooked that diff --git a/hurd.mdwn b/hurd.mdwn index ed88f09f..77401278 100644 --- a/hurd.mdwn +++ b/hurd.mdwn @@ -77,6 +77,7 @@ in the *unstable* branch of the Debian archive. # Developer References +* [[Coding_Style]] * [[Rules]] * [[Trackers]] * [[Building]] diff --git a/hurd/coding_style.mdwn b/hurd/coding_style.mdwn new file mode 100644 index 00000000..cc1e3cf3 --- /dev/null +++ b/hurd/coding_style.mdwn @@ -0,0 +1,59 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_documentation]] + +Some coding style comments that are specific to Hurd systems. + +[[!toc]] + + +# Freeing Port Rights + +## IRC, freenode, #hurd, 2013-07-01 + + do I have to explicitly free ports in a short lived process like + mount? + better take the habit of doing that anyway + how do I recognize that I have to free something? mig spec? + i'd say no + mig does it for you + gnumach reference manual + not memory, like port rights + but no, really, for short lived processes it's ok + yes, port rights + like memory, you don't free stuff in short lived processes :p + mach does it correctly when the task is destroyed + but there are two use cases for rights + those you create manually + and those mig creates for its own purpose + ignore those used by mig, they matter only in very specific parts + of glibc and other very low level stuff + teythoon: keep in mind that there are two flavours of resources + with port rights + but how do I *know* from looking at say fs.defs that I have to + free anything I get? + rights themselves, and the user reference count per right + eh, that's complicated + in a complete RPC call, you must watch two things usually + out of line memory + and right references + except otherwise mentioned, you don't have to free anything + freeing passed memory should be obvious (e.g. "out" keyword on a + memory range) + for right references, it's less obvious + refer to the mach server writing guide i guess + what does the dealloc qualifier do in mig defs? + basically, send rights can be created from a receive right + (make_send), or another send right (copy_send) + it tells mig which function to call once an RPC has returned + all this is described in the mach server writing guide + and it's tricky + quite error-prone so check with portinfo diff --git a/hurd/console/discussion.mdwn b/hurd/console/discussion.mdwn index f887d826..0022ec23 100644 --- a/hurd/console/discussion.mdwn +++ b/hurd/console/discussion.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,6 +10,8 @@ License|/fdl]]."]]"""]] [[!tag open_issue_documentation]] +[[!toc]] + # IRC, OFTC, #debian-hurd, 2012-09-24 @@ -40,3 +42,9 @@ License|/fdl]]."]]"""]] hacking? to fix that is… some hacking yes + + +# IRC, freenode, #hurd, 2013-07-10 + + http://xkbcommon.org/ ‘¡û sounds interesting for our console + translator diff --git a/hurd/debugging.mdwn b/hurd/debugging.mdwn index f4b5eba5..ae9b7bef 100644 --- a/hurd/debugging.mdwn +++ b/hurd/debugging.mdwn @@ -26,3 +26,30 @@ License|/fdl]]."]]"""]] * [[glibc]] * [[translator]]s * [[trap_in_the_kernel]] + + +# IRC, freenode, #hurd, 2013-06-30 + + braunr: I don't understand your question totally, but I want to + know how do you do this inspecting? i have a small test program + that creates a thread, and inspect its state before any thread dies + i use portinfo + and rpctrace + (there is also vminfo but you're not likely to need it for what + you're doing right now) + I have used rpctrace before, but portinfo, I will try it. + is portinfo show a process's all port use log? + not log + current state + dump the port name space? + yes + I found some names are not continuous. how this come out? + continuous ? + 101:send 103:send + missing 102 + some are freed + a lot actually + every RPC needs a reply port + a temporary receive right to get replies from servers + so we can reuse the name which are freed before + of course diff --git a/hurd/debugging/rpctrace.mdwn b/hurd/debugging/rpctrace.mdwn index a5c1a6e9..1570df4c 100644 --- a/hurd/debugging/rpctrace.mdwn +++ b/hurd/debugging/rpctrace.mdwn @@ -182,6 +182,10 @@ See `rpctrace --help` about how to use it. uhu, there's a TODO just above that assertion :) +* IRC, freenode, #hurd, 2013-07-05 + + wish: make rpctrace decode the results of io_stat rpcs + # See Also diff --git a/hurd/translator.mdwn b/hurd/translator.mdwn index d4eaf950..37dac794 100644 --- a/hurd/translator.mdwn +++ b/hurd/translator.mdwn @@ -90,6 +90,7 @@ The [[concept|concepts]] of translators creates its own problems, too: * [[hello]] * [[auth]] * [[exec]] +* [[proc]] * [[pfinet]] * [[pflocal]] * [[hostmux]] @@ -112,6 +113,7 @@ The [[concept|concepts]] of translators creates its own problems, too: * [[procfs]] * [[nsmux]] * [[netio]] +* [[socketio]] * [[tarfs]] * [[gopherfs]] * [[smbfs]] diff --git a/hurd/translator/auth.mdwn b/hurd/translator/auth.mdwn index d9e70ec2..7fd4832c 100644 --- a/hurd/translator/auth.mdwn +++ b/hurd/translator/auth.mdwn @@ -1,12 +1,19 @@ -[[!meta copyright="Copyright © 2008 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2008, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +The *auth server* (or, *authentification server*). + +It is stated by `/hurd/init`. + + +# Documentation [[*The_Authentication_Server*|documentation/auth]], the transcript of a talk about the details of the authentication mechanisms in the Hurd by Wolfgang diff --git a/hurd/translator/firmlink.mdwn b/hurd/translator/firmlink.mdwn index 038879db..b53396b0 100644 --- a/hurd/translator/firmlink.mdwn +++ b/hurd/translator/firmlink.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -20,3 +20,15 @@ License|/fdl]]."]]"""]] infinity0: firmlinks ah thanks i'll look that up braunr: oh, true, I forgot about that one + + +# Open Issues + + * [[!GNU_Savannah_bug 29809]] + + * IRC, freenode, #hurd, 2013-07-07 + + ok, found the firmlink-mv issue + will commit that + \o/ + (bit of mach_print in translators, took just a few hours) diff --git a/hurd/translator/netio.mdwn b/hurd/translator/netio.mdwn index 44c35cf1..429072d4 100644 --- a/hurd/translator/netio.mdwn +++ b/hurd/translator/netio.mdwn @@ -11,6 +11,8 @@ License|/fdl]]."]]"""]] `netio` is a translator designed for creating socket ports through the filesystem. +See also [[socketio]]. + # Source diff --git a/hurd/translator/proc.mdwn b/hurd/translator/proc.mdwn new file mode 100644 index 00000000..98940f87 --- /dev/null +++ b/hurd/translator/proc.mdwn @@ -0,0 +1,53 @@ +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +The *proc server* (or, *process server*) implements some aspects of [[Unix]] +processes. + +It is stated by `/hurd/init`. + + +# "Unusual" PIDs + +[[!tag open_issue_hurd]] + + +## IRC, freenode, #hurd, 2012-08-10 + + too bad the proc server has pid 0 + top & co won't show it + + +## IRC, OFTC, #debian-hurd, 2012-09-18 + + youpi: did you see + https://enc.com.au/2012/09/careful-with-pids/' + ? + nope + + +## IRC, OFTC, #debian-hurd, 2013-06-23 + + I've got this idea about the pid 1 issue you mentioned + can't we just make init pid 1? + I mean the mapping is rather arbitrary, we could make our init + pid 2 or something and start sysvs init as pid 1 + not totally sure it is that arbitrary up to the first 4-5 pids + y is that? + at least i see in hurd's code that /hurd/init is assumed as + pid=1 + hurds init that has to stick around for the fs translator sync? + hurd's init does the basic server startup + iirc it also takes care of shutdown/reboot + that's what I meant + and if it wouldn't have to stick around for the translator sync + it could just exec sysvinit + I just think it's easier to patch hurd than to remove the + assumption that init is pid 1 from sysvinit diff --git a/hurd/translator/procfs/jkoenig/discussion.mdwn b/hurd/translator/procfs/jkoenig/discussion.mdwn index 197461b8..fcda453e 100644 --- a/hurd/translator/procfs/jkoenig/discussion.mdwn +++ b/hurd/translator/procfs/jkoenig/discussion.mdwn @@ -441,26 +441,6 @@ Needed by glibc's `pldd` tool (commit on "internal" (but exported) glibc functions -# "Unusual" PIDs - -Not actually related to procfs, but here seems to be a convenient place for -filing these: - - -## IRC, freenode, #hurd, 2012-08-10 - - too bad the proc server has pid 0 - top & co won't show it - - -## IRC, OFTC, #debian-hurd, 2012-09-18 - - youpi: did you see - https://enc.com.au/2012/09/careful-with-pids/' - ? - nope - - # CPU Usage ## IRC, freenode, #hurd, 2013-01-30 diff --git a/hurd/translator/socketio.mdwn b/hurd/translator/socketio.mdwn new file mode 100644 index 00000000..479d5749 --- /dev/null +++ b/hurd/translator/socketio.mdwn @@ -0,0 +1,30 @@ +[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +`socketio` is a translator designed for creating socket ports through the +filesystem. + +See also [[netio]]. + + +# IRC, freenode, #hurd, 2013-06-30 + + http://lists.gnu.org/archive/html/bug-hurd/2003-05/msg00069.html + this was supposed to be much better than our current netio + (which doesn't really have any documentation btw) + + http://web.archive.org/web/20060117085538/http://duesseldorf.ccc.de/~moritz/files/socketio.c.gz + youpi: socketio looks nice. any reason in particular why you are + working on it? + teythoon: I was looking at the firewall stuff, and wondering about + Zheng Da's work, and seen netio, thus wondered "what is it about + already?" and found there was no documentation, so dug into the mailing + list archives, only to find it was supposed to be precated in favour of + socketio, etc. :) diff --git a/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn index 5228515f..16a23405 100644 --- a/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn +++ b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -270,3 +271,129 @@ See also: [[open_issues/resource_management_problems/pagers]]. only problem is that the current defpager implementation can't really handle that... at least that's my understanding of the situation + + +# IRC, freenode, #hurd, 2013-07-05 + + btw, why does the tmpfs translator have to talk to the pager? + to get more control about how the memory is paged out? + read lot's of irc logs about tmpfs on the wiki, but I couldn't + find the answer to that + teythoon: did you read this? + http://www.gnu.org/software/hurd/hurd/translator/tmpfs/tmpfs_vs_defpager.html + mcsim: I did + teythoon: Last discussion, i think has very good point. + To provide memory objects you should implement pager interface + And if you implement pager interface you are the one who is asked + to write data to backing storage to evict them + But tmpfs doesn't do this + mmm, clients doing mmap... + teythoon: You don't have mmap + teythoon: mmap is implemented on top of mach interface + teythoon: I mean you don't have mmap at this level + mcsim: sure, but that's close enough for me at this point + teythoon: diskfs interface requires implementor to provide a memory + object port (send right) + Guest8183: Why tmpfs requires defpager + how did you get to talk about that ? + I was just asked + Guest8183: it's just so unsettling that tmpfs has to be started + as root :/ + teythoon: why ? + *** Guest8183 (~rbraun@dalaran.sceen.net) is now known as braunr_ + braunr_: b/c starting translators isn't a privileged operation, + and starting a tmpfs translator that doesn't even access any device but + "just" memory shouldn't require any special privileges as well imho + so why is tmpfs not based on say libnetfs? b/c it is used for + d-i and someone (apt?) mmaps stuff? + being libdiskfs-based isn't much the issue, iirc + http://lists.gnu.org/archive/html/bug-hurd/2013-03/msg00014.html + too + teythoon: AFAIK apt uses mmap, yes + teythoon: right + a ramfs is actually tricky to implement well + braunr_: What do you mean under "to implement well"? + as efficiently as possible + i.e. being as close as possible to the page cache for minimum + overhead + braunr: AFAIK ramfs should not use swap partition, so page cache + shouldn't be relevant for it. + i'm talking about a ramfs in general + not the specific linux ramfs + in linux, what they call ramfs is the tiny version of tmpfs that + doesn't use swap + i actually don't like "tmpfs" much + memfs may be more appropriate + anyway + braunr: I see. And do you consider defpager variant as "close as + possible to the page cache"? + not far at least + if we were able to use it for memory obects, it would be nice + but defpager only gets attached to objects when they're evicted + before that, anonymous (or temporary, in mach terminology) objects + have no backing store + this was probably designed without having tmpfs in mind + i wonder if it's possible to create a memory object without a + backing store + what should happen to it if kernel decides to evict it? + it sets the default pager as its backing store and pushes it out + that's how it works now, but you said "create a memory object + without a backing store" + mach can do that + i was wondering if we could do that too from userspace + mach does not evict such objects, unless it bound a defpager to + them + but how can you handle this in userspace? + i mean, create a memory object with a null control port + mcsim: is that clearer ? + suppose you create such object, how kernel will evict it if kernel + does not know who is responsible for eviction of this object? + it does + 16:41 < braunr> it sets the default pager as its backing store and + pushes it out + that's how i intend to do it on x15 at least + but it's much simpler there because uvm provides better separation + between anonymous and file memory + whereas they're much too similar in mach vm + than what the difference between current situation, when you + explicitly invoke defpager to create object and implicit method you + propose? + you don't need a true defpager unless you actually have swap + ok + now I see + it also saves the communication overhead when initializing the + object + thank you + which may be important since we use ramfs for speed mostly + agree + it should also simplify the defpager implementation, since it + would only have a single client, the kernel + which may also be important with regard to global design + one thing which is in my opinion very wrong with mach is that it + may be a client + a well designed distributed system should normally not allow on + component to act as both client and server toward another + i.e. the kernel should only be a server, not a client + and there should be a well designed server hierarchy to avoid + deadlocks + (such as the one we had in libpager because of that) + And how about filesystem? It acts both as server and as client + yes + but not towards the same other component + application -> file system -> kernel + no "<->" + the qnx documentation explains that quite well + let me see if i can find the related description + Basically, I've got your point. And I would rather agree that + kernel should not act as client + mcsim: + http://www.qnx.com/developers/docs/6.4.0/neutrino/sys_arch/ipc.html#Robust + one way to implement that (and qnx does that too) is to make + pagers act as client only + they sleep in the kernel, waiting for a reply + and when the kernel needs to evict something, a reply is sent + (qnx doesn't actually do that for paging, but it's a general idea) + braunr: how hierarchy of senders is enforced? + it's not + developers must take care + same as locking, be careful about it diff --git a/libpthread.mdwn b/libpthread.mdwn index 0a518996..fc5c0974 100644 --- a/libpthread.mdwn +++ b/libpthread.mdwn @@ -60,45 +60,8 @@ even if the current number of threads is lower. The same issue exists in [[hurd/libthreads]]. - -### IRC, freenode, #hurd, 2013-05-09 - - braunr: Speaking of which, didn't you say you had another "easy" - task? - bddebian: make a system call that both terminates a thread and - releases memory - (the memory released being the thread stack) - this way, a thread can completely terminates itself without the - assistance of a managing thread or deferring work - braunr: That's "easy" ? :) - bddebian: since it's just a thread_terminate+vm_deallocate, it is - something like thread_terminate_self - But a syscall not an RPC right? - in hurd terminology, we don't make the distinction - the only real syscalls are mach_msg (obviously) and some to get - well known port rights - e.g. mach_task_self - everything else should be an RPC but could be a system call for - performance - since mach was designed to support clusters, it was necessary that - anything not strictly machine-local was an RPC - and it also helps emulation a lot - so keep doing RPCs :p - - -#### IRC, freenode, #hurd, 2013-05-10 - - i'm not sure it should only apply to self though - youpi: can we get a quick opinion on this please ? - i've suggested bddebian to work on a new RPC that both terminates - a thread and releases its stack to help fix libpthread - and initially, i thought of it as operating only on the calling - thread - do you see any reason to make it work on any thread ? - (e.g. a real thread_terminate + vm_deallocate) - (or any reason not to) - thread stack deallocation is always a burden indeed - I'd tend to think it'd be useful, but perhaps ask the list +The current implementation in libpthread is +[[buggy|libpthread/t/fix_have_kernel_resources]]. # Open Issues diff --git a/microkernel/mach/deficiencies.mdwn b/microkernel/mach/deficiencies.mdwn index d1cdeb54..03e4a8b0 100644 --- a/microkernel/mach/deficiencies.mdwn +++ b/microkernel/mach/deficiencies.mdwn @@ -2190,3 +2190,131 @@ In context of [[open_issues/multithreading]] and later [[open_issues/select]]. i really need to dig into this code at some point :) well you may but you may not see that property from the code itself + + +## IRC, freenode, #hurd, 2013-06-28 + + so tell me about x15 if you are in the mood to talk about that + what do you want to know ? + well, the high level stuff first + like what's the big picture + the big picture is that x15 is intended to be a "better mach for + the hurd + " + mach is too general purpose + its ipc mechanism too powerful + too complicated, error prone, and slow + so i intend to build something a lot simpler and faster :p + so your big picture includes actually porting hurd? i thought i + read somewhere that you have a rewrite in mind + it's a clone, yes + x15 will feature mostly sync ipc, and no high level types inside + messages + the ipc system call will look like what qnx does + send-recv from the client, recv/reply/reply-recv from the server + but doesn't sync mean that your context switch will have to be + quite fast? + how does that differ from the async approach ? + (keep in mind that almost all hurd RPCs are synchronous) + yes, I know, and it also affects async mode, but a slow switch + is worse for the sync case, isn't it? + ok so your ipc will be more agnostic wrt to what it transports? + unlike mig I presume? + no it's the same + yes + input will be an array, each entry denoting either memory or port + rights + (or directly one entry for fast ipcs) + memory as in pointers? + (well fast ipc when there is only one entry to avoid hitting a + table) + pointer/size yes + hm, surely you want a way to avoid copying that, right? + the only operation will be copy (i.e. unlike mach which allows + sharing) + why ? + copy doesn't exclude zero copy + (zero copy being adjusting page tables with copy on write + techniques) + right + but isn't that too coarse, like in cow a whole page? + depends on the message size + or options provided by the caller, i don't know yet + oh, you are going to pack the memory anyway? + depends on the caller + i'm not yet sure about these details + ideally, i'd like to avoid serialization altogether + wouldn't that be like cheating b/c it's the first copy? + directly pass pointers/sizes from the sender address space, and + either really copy or use zero copy + right, but then you're back at the page size issue + yes + it's not a real issue + the kernel must support both ways + the minor issue is determining which way to choose + it's not a critical issue + my current plan is to always copy, unless the caller has + explicitely set a flag and is passing properly aligned buffers + u sure? I mean the caller is free to arange the stuff he intends + to send anyway he likes, how are you going to cow that then? + ok + right + properly aligned buffers :) + otherwise the kernel rejects the request + that's reasonable, yes + in addition to being synchronous, ipc will also take a special + path in the scheduler to directly use the client scheduling context + avoiding the sleep/wakeup overhead, and providing priority + inheritence by side effect + uh, but wouldn't dropping serialization create security and + reliability issues? if the receiver isn't doing a proper job sanitizing + its stuff + why would the client not sanitize ? + err + server + it has to anyway + sure, but a proper parser written once might be more robust, + even if it adds overhead + the serialization i mean + it's just a layer + even with high level types, you still need to sanitize + the real downside is loosing cross architecture portability + making the potential implementation of a single system image a lot + more restricted or difficult + but i don't care about that much + mach was built with this in mind though + it's a nice idea, but i don't believe anyone does ssi anymore + i don't know + and certainly not across architectures + there are few projects + anyway it's irrelevant currently + and my interface just restricts it, it doesn't prevent it + so i consider it an acceptable compromise + so, does it run? what does it do? + it certainly is, yes + for now, it manages memory (physical, virtual, kernel, and soon, + anonymous) + support multiple processors with the required posix scheduling + policies + (it uses a cute proportionally fair time sharing algorithm) + there are locks (spin locks, mutexes, condition variables) and + lockless stuff (à la rcu) + both x86 and x86_64 are supported + (even pae) + work queues + sounds impressive :) + :) + i also added basic debugging + stack trace (including getting the symbol table) handling + so yes, it's much much better than what i previously did + and on the right track + it already scales a lot better than mach for what it does + there are generic data structures (linked list, red-black tree, + radix tree) + the radix tree supports lockless lookups, so looking up both the + page cache and the ipc spaces is lockless) + that's nice :) + there are a few things using global locks, but there are TODOs + about them + even with that, it should be scalable enough for a start + and improving those parts shouldn't be too difficult diff --git a/microkernel/mach/gnumach/debugging.mdwn b/microkernel/mach/gnumach/debugging.mdwn index 71e92459..7e7cfb4e 100644 --- a/microkernel/mach/gnumach/debugging.mdwn +++ b/microkernel/mach/gnumach/debugging.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2007, 2008, 2009, 2011, 2012 Free Software +[[!meta copyright="Copyright © 2007, 2008, 2009, 2011, 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -75,6 +75,9 @@ When you're [[running_a_system_in_QEMU|hurd/running/qemu]] you can directly kernel](http://www.nongnu.org/qemu/qemu-doc.html#SEC48). +## [[open_issues/debugging_gnumach_startup_qemu_gdb]] + + # Code Inside the Kernel Alternatively you can use an approach like this one: add the following code diff --git a/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn b/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn index ca52dca5..a169e92e 100644 --- a/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn +++ b/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn @@ -59,3 +59,32 @@ License|/fdl]]."]]"""]] it [[Makefile]], [[mach_print.S]], [[main.c]]. + + +## IRC, freenode, #hurd, 2013-07-01 + + braunr: btw, we are missing the symbol in mach/Versions + youpi: what symbol ? + so the libc-provided RPC stub is not available + mach_printf + -f + it's a system calll + not exported + s/RPC/system call/ + that's expected + libc does provide stubs for system calls too + yes but not for this one + I don't see why we wouldn't want to include it + ?! it does + it's temporary + oh + there must be automatic parsing during build + sure + nice + + youpi: if we're going to make this system call exported by glibc, + i should change its interface first + it was meant as a very quick-and-dirty hack and directly accesses + the caller's address space without going through a special copy-from-user + function + not very portable diff --git a/microkernel/mach/history.mdwn b/microkernel/mach/history.mdwn index 776bb1d7..c22ea739 100644 --- a/microkernel/mach/history.mdwn +++ b/microkernel/mach/history.mdwn @@ -78,3 +78,137 @@ IRC, freenode, #hurd, 2012-08-29: https://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/About/About.html can't be anymore http://developer.apple.com/techpubs/macosx/Darwin/General/KernelProgramming/About/index.html + +IRC, freenode, #hurd, 2013-07-03: + + *** natsukao (~natsukao@dynamic-adsl-94-37-184-109.clienti.tiscali.it) has + joined channel #hurd + hi + on 2012-08-29: i wrote a part of messages that then were posted + on http://www.gnu.org/software/hurd/microkernel/mach/history.html + i am sorry to inform you that apple computer cuèertino.inc, has + moved the URL: https://ssl.apple.com/science/profiles/cornell + and i have not found nothing on the source code of that page, + i used lftp without any success + and then wget, nothing to do + i have not found a copy cache of + https://ssl.apple.com/science/profiles/cornell + next time we save the documents and we provide to do our + archive/s + so that will be always available the infos + *** natsukao (~natsukao@dynamic-adsl-94-37-184-109.clienti.tiscali.it) is + now known as pavlx + happy hacking !!!! + "paolo del bene" + p.s: i'll turn back as soon as possible + + i found the page of Darwin History, removed from apple compter + cupertino.inc + "Cached_ http___developer.apple.com_darwin_history.html" + the page http://developer.apple.com/darwin/history.html was moved + and now is available on: + + http://www.google.it/url?q=http://www.macmark.de/files/darwin_evolution.pdf%3FPHPSESSID%3D8b8683a81445f69d510734baa13aabfc&sa=U&ei=wMzTUb-NBIeFOOL4gNgE&ved=0CCQQFjAD&usg=AFQjCNFlLwC24nB5t14VUmedK4EmeE7Ohw + or simply: http://www.macmark.de/files/darwin_evolution.pdf + slides on: "Travel - Computer Science and Software Engineering" + www.csse.uwa.edu.au/~peterj/personal/PDFs/WWDC2004-6UP.pdf + about apple computer cupertino.inc, but there are many interesting + news + pavlx: uh, lot's of marketing noise from apple >,< + i found better material just now: + http://www.pcs.cnu.edu/~mzhang/CPSC450_550/2003CaseStudy/Mach_Presentation_DavidRamsey.ppt + teythoon, sorry, i turn back to sleep, see you later, paolo + W3C_Freedom@riseup.net + i'll charge of that page only things dedicated to GNU/HURD, but + slides are not mine, i found on internet + pavlx: sure, I didn't ment to offend you in any way + +IRC, freenode, #hurd, 2013-07-04: + + there are few problems: + http://www.gnu.org/software/hurd/microkernel/mach/history.html + on the page GrantBow wrote: Apple's Macintosh OSX (OS 10.x) is + based on Darwin. "Darwin uses a monolithic kernel based on ?FreeBSD 4.4 + and the OSF/mk Mach 3." Darwin also has a Kernel Programming Book. + the link to Darwin was moved, is not anymore + http://www.apple.com/macosx/technologies/darwin.html + then it's not FreeBSD 4.4 but BSD + and the link to Kernel Programming was moved is not + http://developer.apple.com/techpubs/macosx/Darwin/General/KernelProgramming/About/index.html + but + https://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/About/About.html + apple has moved the URL: + https://ssl.apple.com/science/profiles/cornell + apple has moved the URL: + http://www.apple.com/macosx/technologies/darwin.html + so on the website you can left few things about my old post: + from IRC, freenode, #hurd, 2012-08-29: needs to remove + http://dpaste.com/1286266/ + the new one will be: http://pastebin.com/yMm8tcTN + IRC, freenode, #hurd, 2013-07-04: + + was moved the page from apple.com about darwin kernel programming + as described on the https://www.gnu.org/software/hurd/microkernel/mach/history.html + + the link to Kernel Programming: + https://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/About/About.html + (anyway i searching with any key the things moved from apple) + about Darwin type http://apple.com/darwin + on the right side, towards the end of the website it says: Darwin + Technologies + click on it, or copy the URL in an other tab of your own browser, + and read: + https://developer.apple.com/library/mac/#documentation/MacOSX/Conceptual/OSX_Technology_Overview/SystemTechnology/SystemTechnology.html + and something is related to Darwin + and again : http://pastebin.com/DHkJDxy8 + # Mac OS X Server + + ... This kernel, known as Darwin, provides a stable, high-performance platform + for developing groundbreaking applications and system technologies. ... + http://www.apple.com/server/docs/MacOSX_Server_TO_300195.pdf + + # Mac OS X Server Command-Line Administration + + Page 1. Mac OS X Server Command-Line Administration For Version 10.3 + http://www.apple.com/server/docs/Command_Line.pdf + + # Press Info - Apple “Open Sources” Rendezvous + + ... Rendezvous is part of a broader open source release today from Apple at + http://developer.apple.com/darwin which includes the Darwin 6.0.1 ... + http://www.apple.com/pr/library/2002/09/25Apple-Open-Sources-Rendezvous.html + + # Press Info - Apple Releases Darwin 1.0 Open Source + + ... Apple Releases Darwin 1.0 Open Source. New ... modules. Darwin 1.0 gives + developers access to essential Mac OS X source code. ... + http://www.apple.com/pr/library/2000/04/05Apple-Releases-Darwin-1-0-Open-Source.html + + # Press Info - Apple's Mac OS X to Ship on March 24 + + ... Mac OS X is built upon an incredibly stable, open source, UNIX based + foundation called Darwin and features true memory protection, preemptive ... + http://www.apple.com/pr/library/2001/01/09Apples-Mac-OS-X-to-Ship-on-March-24.html + + # Press Info - Mac OS X “Gold Master” Released To ... + + ... Mac OS X is built upon an incredibly stable, open source, UNIX + basedfoundation called Darwin and features true memory protection ... + http://www.apple.com/pr/library/2001/03/07Mac-OS-X-Gold-Master-Released-To-Manufacturing.html + + * Press Info - Apple Announces Mac OS X “Jaguar” ... + + ... As an active member of the Open Source community, Apple has distributed + Open Directory technology through the Darwin Open Source Project. ... + http://www.apple.com/pr/library/2002/07/17Apple-Announces-Mac-OS-X-Jaguar-Server-Worlds-Easiest-to-Manage-UNIX-Based-Server-Software.html + and: + http://lists.apple.com/archives/darwinos-users/2005/Apr/msg00021.html + pavlx: it's hard to follow the changes you are talking + about. Perhaps you could simply edit these wiki pages? + anyway i am saying to you that i found a mailing list where are + availables the sources codes of darwin ppc-801 and x86 + and as last thing mac os x 10.4 + pavlx: what's all this about ? + i am sorry, i did changes on the wiki + but after the preview and after to have saved, it show again the + old chat of 2012 diff --git a/microkernel/mach/mig.mdwn b/microkernel/mach/mig.mdwn index 331b3bf4..d6340574 100644 --- a/microkernel/mach/mig.mdwn +++ b/microkernel/mach/mig.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2001, 2002, 2003, 2006, 2007, 2008, 2010 Free -Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2001, 2002, 2003, 2006, 2007, 2008, 2010, 2013 +Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -22,9 +22,9 @@ wait for a result on a newly created [[reply port|port]], decode return arguments from the reply message (*demarshalling*, or *unmarshalling*) and pass them to the client program. Similar actions are provided in the skeletons that are linked to server programs. - MIG allows very precise semantics to be specified about what the arguments are and how to be passed. +It has its problems with [[structured_data]], however. * [[Documentation]] diff --git a/microkernel/mach/mig/structured_data.mdwn b/microkernel/mach/mig/structured_data.mdwn new file mode 100644 index 00000000..1c8abe08 --- /dev/null +++ b/microkernel/mach/mig/structured_data.mdwn @@ -0,0 +1,119 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_mig]] + +# IRC, freenode, #hurd, 2013-06-25 + + is there a nice way to get structured data through mig that I + haven't found yet? + say an array of string triples + no + :/ + but you shouldn't need that + my use case is getting info about fs translators from init to + procfs + +[[community/gsoc/project_ideas/mtab]]. + + should I go for an iterator like interface instead? + depends + how many do you need ? + you could go for a variable sized array too + have a look at what already exists + records, maybe 10-15, depends on many fs translators are running + a variable sized array is ok if the size isn't too big (and when i + say too big, i mean hundreds of MiB) + an iterator is ok too if there aren't too many items + you may want to combine both (i think that's what proc does) + be aware that the maximum size of a message is limited to 512 MiB + yeah I saw the array[] of stuff stuff, but array[] of string_t + does not work, I guess b/c string_t is also an array + how would I send an array of variable length strings? + i'm not sure you can + or maybe out of line + somehow I expected mig to serialize arbitrary data structures, + maybe it's to old for that? + yeah, I read about uot of line, but that seems overkill + it is old yes + and not very user friendly in the end + let me check + we could stuff json into mig... + see proc_getallpids for example + we could get rid of low level serialization altogether :p + hah, exactly what I was looking at + (which is what i'll do in x15) + type pidarray_t = array[] of pid_t; + but that is trivial b/c its array[] of pid_t + and always have the server writing guide near you + yes + well, make one big string and an array of lengths :p + thought about that and said to myself, there must be a better + way that I haven't found yet + or one big string filled with real null-terminated c strings that + you keep parsing until you ate all input bytes + i'm almost certain there isn't + type string_t = c_string[1024]; /* XXX */ + yes + even that isn't really variable sized + you think anyone would object to me putting a json encoder in + /hurd/init? it is probably better than me at serializing stuff... + try with mig anyway + the less dependencies we have for core stuff, the simpler it is + but i agree, mig is painful + would it be too hacky if I abused the argz functions? they do + exactly what I'd need + + +## IRC, freenode, #hurd, 2013-06-26 + + there is https://code.google.com/p/protobuf-c/ and it has a rpc + mechanism and I believe one could plug arbitrary transports easily + please don't think about it + we really don't want to add another layer of serialization + it's better to completely redesign mach ipc anyway + and there is a project for that :p + ive seen x15 + just food for thought + i've studied google protocol buffers + and fyi, no, it wouldn't be easy to plug arbitrary transports on + top of mach + there is a lot of knowledge about mach ports in mig + +[[community/gsoc/project_ideas/mtab]]. + + but again I face the challenge of serializing a arbitrary sized + list of arbitrary sized strings + yes + list of ports is easier ;) but I think its worthwile + so what about abusing argz* for this? you think it's too bad a + hack? + no since it's in glibc + awesome :) + but i don't remember the details well and i'm not sure the way you + use it is safe + yeah, I might have got the details wrong, I hadn't had the + chance to test it ;) + + about this dynamic size problem + a "simple" varying size array should do + you can easily put all your strings in there + seperated by 0? + yes + that's exactly what the argz stuff does + you'll get the size of the array anyway, and consume it until + there is no byte left + good + but be careful with this too + since translators can be run by users, they somtimes can't be + trusted + and even a translator running as root may behave badly + so careful with parsing + noted diff --git a/open_issues/64-bit_port.mdwn b/open_issues/64-bit_port.mdwn index 66da44b9..b0c95612 100644 --- a/open_issues/64-bit_port.mdwn +++ b/open_issues/64-bit_port.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2011, 2012, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -137,3 +138,20 @@ See also [[microkernel/mach/gnumach/memory_management]]. hm actually no, it would require mcmodel=large hum, that's stupid, we can make the kernel run at -2g, and use 3g up to the sign extension hole for the kernel map + + +# IRC, freenode, #hurd, 2013-07-02 + +In context of [[mondriaan_memory_protection]]. + + BTW, it's not like I have an infinite amount of time for this, + but having 64-bit support would be valuable for me, so I might contribute + that back if it's not a too monumental task + I saw some discussions about 32bit apps on top of 64bit mach, but + I'd like a full 64bit system + any clues? + I suppose the compiler support is all there already + is MIG (and mach) the only piece missing? + the problem is the interfaces themselves + type widths + as passed between userspace and kernel diff --git a/open_issues/anatomy_of_a_hurd_system.mdwn b/open_issues/anatomy_of_a_hurd_system.mdwn index 11a1f754..75a62535 100644 --- a/open_issues/anatomy_of_a_hurd_system.mdwn +++ b/open_issues/anatomy_of_a_hurd_system.mdwn @@ -323,7 +323,9 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l etc.. -# IRC, freenode, #hurd, 2012-12-06 +# Service Directory + +## IRC, freenode, #hurd, 2012-12-06 what is the #1 feature that distinguished hurd from other operating systems. the concept of translators. (will read more when I get @@ -333,56 +335,7 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l and the VFS permissions to control access to those services -# IRC, freenode, #hurd, 2012-12-10 - - I want to work on hurd, but I think I'm going to start with - minix, I own the minix book 3rd ed. it seems like a good intro to - operating systems in general. like I don't even know what a semaphore is - yet. - well, enjoy learning :) - once I finish that book, what reading do you guys recommend? - other than the wiki - i wouldn't recommend starting with a book that focuses on one - operating system anyway - you tend to think in terms of what is done in that specific - implementation and compare everything else to that - tannenbaum is not only the main author or minix, but also the one - of the book http://en.wikipedia.org/wiki/Modern_Operating_Systems - - http://en.wikipedia.org/wiki/List_of_important_publications_in_computer_science#Operating_systems - should be a pretty good list :) - - -# IRC, freenode, #hurd, 2013-03-12 - - i have a question regarding ipc in hurd. if a task is created, does - it contain any default port rights in its space? i am trying to deduce - how one calls dir_lookup() on the root translator in glibc's open(). - mjjc: yes, there are some default port rights, but I don't - remember the details :/ - kilobug: do you know where i should search for details? - mjjc: hum either in the Hurd's hacking guide - https://www.gnu.org/software/hurd/hacking-guide/ or directly in the - source code of exec server/libc I would say, or just ask again the - question here later on to see if someone else has more information - ok, thanks - there's also rpctrace to, as the name says, trace all the rpc's - executed - some ports are introduced in new tasks, yes - see - http://www.gnu.org/software/hurd/hacking-guide/hhg.html#The-main-function - and - - http://www.gnu.org/software/hurd/gnumach-doc/Task-Special-Ports.html#Task-Special-Ports - yes, the second link was just what i was looking for, thanks - the second is very general - also, the first applies to translators only - if you're looking for how to do it for a non-translator - application, the answer is probably somewhere in glibc - _hurd_startup i'd guess - - -# IRC, freenode, #hurd, 2013-05-23 +## IRC, freenode, #hurd, 2013-05-23 Hi, is there any efficient way to control which backed translators are called via RPC with a user space program? @@ -409,18 +362,7 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l same as for regular files gnu_srs: this *must* be obvious for you to do any tricky work on the hurd - fsysopts /servers/socket/2 works by /1 gives Operation not - supported. - -[[!taglink open_issue_hurd]]. - - ah right, some servers don't implement that - work around this by using showtrans - fsysopts asks the server itself how it's running, usually giving - its command name and options - showtrans asks the parent how it starts a passive translator - attached to the node - Yes showtrans works :), thanks. + Anyway, if I create a test program calling io_stat I assume S_io_stat in pflocal is called. How to make the program call S_io_stat in pfinet instead? @@ -447,6 +389,127 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l implementation +## IRC, freenode, #hurd, 2013-06-30 + + hi, what is the replacer of netname_check_in? + + I want to ask another question. in my opinion, the rpc is the + mach's way, and the translator is the hurd's way. so somebody want to + lookup a service, it should not need to ask the mach kernel know about + this query. the hurd will take the control. + am I right? + no + that's nonsense + service lookups has never been in mach + first mach based systems used a service directory, whereas the + hurd uses the file system for that + you still need mach to communicate with either of those + how to understand the term of service directory here? + a server everyone knows + which gives references to other servers + usually, depending on the name + e.g. name_lookup("net") -> port right to network server + is that people use netname_check_in to register service in the + past? now used libtrivfs? + i don't know about netname_check_in + old mach (not gnumach) documentation might mention this service + directory + libtrivfs doesn't have much to do with that + on the hurd, the equivalent is the file system + maybe that is outdate, I just found that exist old doc, and old + code which can't be build. + every process knows / + the file system is the service directory + nodes refer to services + so the file system is the nameserver, any new service should + register in it before other can use + and the file system is distributed, so looking up a service may + require several queries + setting a translator is exactly that, registering a program to + service requests on a node + the file system isn't one server though + programs all know about /, but then, lookups are recursive + e.g. if you have / and /home, and are looking for + /home/hacklu/.profile, you ask / which tells you about /home, and /home + will give you a right to /home/hacklu/.profile + even in the past, the mach don't provide name register service, + there must be an other server to provide this service? + yes + what's nonsense in your sentence is comparing RPCs and translators + translators are merely servers attached to the file system, using + RPCs to communicate with the rest of the system + I know yet, the two just one thing. + no + two things :p + completely different and unrelated except for one using the other + ah, just one used aonther one. + is exist anyway to anounce service except settrans with file node? + more or less + tasks can have special ports + that's how one task knows about / for example + at task creation, a right to / is inserted in the new task + I think this is also a file node way. + no + if i'm right, auth is referenced the same way + and there is no node for auth + how the user get the port of auth with node? + it's given when a task is created + pre-set in the creation of one task? + i'm unconfortable with "pre-set" + inserted at creation time + auth is started very early + then tasks are given a reference to it + + +# IRC, freenode, #hurd, 2012-12-10 + + I want to work on hurd, but I think I'm going to start with + minix, I own the minix book 3rd ed. it seems like a good intro to + operating systems in general. like I don't even know what a semaphore is + yet. + well, enjoy learning :) + once I finish that book, what reading do you guys recommend? + other than the wiki + i wouldn't recommend starting with a book that focuses on one + operating system anyway + you tend to think in terms of what is done in that specific + implementation and compare everything else to that + tannenbaum is not only the main author or minix, but also the one + of the book http://en.wikipedia.org/wiki/Modern_Operating_Systems + + http://en.wikipedia.org/wiki/List_of_important_publications_in_computer_science#Operating_systems + should be a pretty good list :) + + +# IRC, freenode, #hurd, 2013-03-12 + + i have a question regarding ipc in hurd. if a task is created, does + it contain any default port rights in its space? i am trying to deduce + how one calls dir_lookup() on the root translator in glibc's open(). + mjjc: yes, there are some default port rights, but I don't + remember the details :/ + kilobug: do you know where i should search for details? + mjjc: hum either in the Hurd's hacking guide + https://www.gnu.org/software/hurd/hacking-guide/ or directly in the + source code of exec server/libc I would say, or just ask again the + question here later on to see if someone else has more information + ok, thanks + there's also rpctrace to, as the name says, trace all the rpc's + executed + some ports are introduced in new tasks, yes + see + http://www.gnu.org/software/hurd/hacking-guide/hhg.html#The-main-function + and + + http://www.gnu.org/software/hurd/gnumach-doc/Task-Special-Ports.html#Task-Special-Ports + yes, the second link was just what i was looking for, thanks + the second is very general + also, the first applies to translators only + if you're looking for how to do it for a non-translator + application, the answer is probably somewhere in glibc + _hurd_startup i'd guess + + # IRC, freenode, #hurd, 2013-06-15 ive been reading a little about exokernels or unikernels, and i diff --git a/open_issues/arm_port.mdwn b/open_issues/arm_port.mdwn index 8a2a037f..b07df939 100644 --- a/open_issues/arm_port.mdwn +++ b/open_issues/arm_port.mdwn @@ -273,4 +273,3 @@ architecture. braunr: OK, thanks. I'm interested on it, and didn't want to duplicate efforts. little addition: it may have started, but we don't know about it - diff --git a/open_issues/code_analysis.mdwn b/open_issues/code_analysis.mdwn index bdd2ae18..67798c6a 100644 --- a/open_issues/code_analysis.mdwn +++ b/open_issues/code_analysis.mdwn @@ -193,3 +193,17 @@ There is a [[!FF_project 276]][[!tag bounty]] on some of these tasks. * [Trinity: A Linux kernel fuzz tester (and then some)](http://www.socallinuxexpo.org/scale11x/presentations/trinity-linux-kernel-fuzz-tester-and-then-some), Dave Jones, The Eleventh Annual Southern California Linux Expo, 2013. + + * Mayhem, *an automatic bug finding system* + + IRC, freenode, #hurd, 2013-06-29: + + started reading the mayhem paper referenced here + http://lists.debian.org/debian-devel/2013/06/msg00720.html + that's nice work, they are doing symbolic execution of x86 + binary code, that's effectively model checking with some specialized + formulas + (too bad the mayhem code isn't available, damn those + academic people keeping the good stuff to themselvs...) + (and I really think that's bad practice, how should anyone + reproduce their results? that's not how science works imho...) diff --git a/open_issues/dde.mdwn b/open_issues/dde.mdwn index 65d84886..76b80211 100644 --- a/open_issues/dde.mdwn +++ b/open_issues/dde.mdwn @@ -598,3 +598,47 @@ In context of [[libpthread]]. hm, i haven't looked but, does someone know if virtio is included in netdde ? braunr: nope, there's an underlying virtio layer needed before + + +## IRC, freenode, #hurd, 2013-07-24 + + btw, I'd love to see libvirt support in hurd + I tried to hack up a dde based net translator + afaics they are very much like any other pci device, so the + infrastructure should be there + if anything I expect the libvirt stuff to be more easily + portable + what do you mean by "a dde based net translator" ? + ah, you mean virtio support in netdde ? + yes + virtio net is present in the kernel version we use for the dde + drivers + so I just copied the dde driver over, but I had no luck + compiling it + ok, but what would be the benefice over e1000 & co? + any of the dde drivers btw + youpi: less overhead + e1000 is already low overhead actually + there are less and less differences in strategies for driving a + real board, and a virtual one + we are seeing shared memory request buffer, dma, etc. in real + boards + which ends up being almost exactly what virtio does :) + ahci, for instance, really looks extremely like a virtio interface + (I know, it's a disk, but that's the same idea, and I do know what + I'm talking about here :) ) + that would actually be my next wish, a virtio disk driver, and + virt9p ;) + on the other hand, i wouldn't spend much time on a virtio disk + driver for now + the hurd as it is can't boot on a device that isn't managed by the + kernel + we'd need to change the boot protocol + ok, I wasn't planning to, just wanted to see if I can easily + hack up the virtio-net translator + well, as youpi pointed, there is little benefit to that as well + but if that's what you find fun, help yourself :) + I didn't know that, I assumed there was some value to the virtio + stuff + there is + but relatively to other improvements, it's low diff --git a/open_issues/gdb_signal_handler.mdwn b/open_issues/gdb_signal_handler.mdwn new file mode 100644 index 00000000..3084f7e3 --- /dev/null +++ b/open_issues/gdb_signal_handler.mdwn @@ -0,0 +1,403 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_gdb open_issue_glibc]] + + +# IRC, freenode, #hurd, 2013-07-07 + + Hi, I'm in GDB inside a handler for SIGHUP, after stepping out, gdb + will hang on instruction: <_hurd_sigstate_lock+88>: xchg + %edx,0x4(%eax) + here is my signal test pasted: http://pastebin.com/U72qw3FC + #include + #include + #include + + void * + my_handler(int signal, void *info, void *context) + { + printf("got SIGHUP\n"); + return NULL; + } + + void + install_handler (int signal) + { + struct sigaction sa; + sa.sa_sigaction = my_handler; + sa.sa_flags = SA_SIGINFO; + sigemptyset(&sa.sa_mask); + sigaction(signal, &sa, NULL); + } + + void test_sighup(void) + { + raise(SIGHUP); + } + + int main(int argc, char **argv){ + install_handler(SIGHUP); + test_sighup(); + exit(1); + } + zyg: thanks + zyg: what is the problem exactly ? + zyg: i mean, does it hand before attaching with gdb ? + braunr: it doesn't hang if runned without gdb. I've pasted here when + I step out of the handler, and get to the hanging instruction: + http://pastebin.com/nUyCx6Wj + $ gdb --args a.out + GNU gdb (GDB) 7.6-debian + Copyright (C) 2013 Free Software Foundation, Inc. + License GPLv3+: GNU GPL version 3 or later + This is free software: you are free to change and redistribute it. + There is NO WARRANTY, to the extent permitted by law. Type "show copying" + and "show warranty" for details. + This GDB was configured as "i486-gnu". + For bug reporting instructions, please see: + ... + Reading symbols from /home/shrek/a.out...(no debugging symbols found)...done. + (gdb) + + (gdb) display/i $pc + (gdb) handle SIGHUP pass stop print + Signal Stop Print Pass to program Description + SIGHUP Yes Yes Yes Hangup + (gdb) + + (gdb) run + Starting program: /home/shrek/a.out + [New Thread 3571.5] + + Program received signal SIGHUP, Hangup. + 0x010548ec in mach_msg_trap () + at /build/buildd-eglibc_2.17-6-hurd-i386-g946kE/eglibc-2.17/build-tree/hurd-i386-libc/mach/mach_msg_trap.S:2 + 2 /build/buildd-eglibc_2.17-6-hurd-i386-g946kE/eglibc-2.17/build-tree/hurd-i386-libc/mach/mach_msg_trap.S: No such file or directory. + 1: x/i $pc + => 0x10548ec : ret + (gdb) + + (gdb) si + 0x0804862d in my_handler () + 1: x/i $pc + => 0x804862d : push %ebp + (gdb) x/20xi 0x804862d + => 0x804862d : push %ebp + 0x804862e : mov %esp,%ebp + 0x8048630 : sub $0x18,%esp + 0x8048633 : movl $0x8048750,(%esp) + 0x804863a : call 0x8048500 + 0x804863f : mov $0x0,%eax + 0x8048644 : leave + 0x8048645 : ret + 0x8048646 : push %ebp + 0x8048647 : mov %esp,%ebp + 0x8048649 : sub $0x28,%esp + 0x804864c : movl $0x804862d,-0x14(%ebp) + 0x8048653 : movl $0x40,-0xc(%ebp) + 0x804865a : lea -0x14(%ebp),%eax + 0x804865d : add $0x4,%eax + 0x8048660 : mov %eax,(%esp) + 0x8048663 : call 0x80484d0 + 0x8048668 : movl $0x0,0x8(%esp) + 0x8048670 : lea -0x14(%ebp),%eax + 0x8048673 : mov %eax,0x4(%esp) + (gdb) + + (gdb) break *0x804863f + Breakpoint 1 at 0x804863f + (gdb) c + Continuing. + got SIGHUP + + Breakpoint 1, 0x0804863f in my_handler () + 1: x/i $pc + => 0x804863f : mov $0x0,%eax + (gdb) + + (gdb) si + 0x08048644 in my_handler () + 1: x/i $pc + => 0x8048644 : leave + (gdb) + 0x08048645 in my_handler () + 1: x/i $pc + => 0x8048645 : ret + (gdb) + 0x010708b2 in trampoline () from /lib/i386-gnu/libc.so.0.3 + 1: x/i $pc + => 0x10708b2 : add $0xc,%esp + (gdb) + 0x010708b5 in trampoline () from /lib/i386-gnu/libc.so.0.3 + 1: x/i $pc + => 0x10708b5 : ret + (gdb) + __sigreturn (scp=0x102988c) at ../sysdeps/mach/hurd/i386/sigreturn.c:30 + 30 ../sysdeps/mach/hurd/i386/sigreturn.c: No such file or directory. + 1: x/i $pc + => 0x1096340 <__sigreturn>: push %ebp + (gdb) + 0x01096341 30 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096341 <__sigreturn+1>: push %edi + (gdb) + 0x01096342 30 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096342 <__sigreturn+2>: push %esi + (gdb) + 0x01096343 30 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096343 <__sigreturn+3>: push %ebx + (gdb) + 0x01096344 30 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096344 <__sigreturn+4>: sub $0x2c,%esp + (gdb) + 0x01096347 30 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096347 <__sigreturn+7>: mov 0x40(%esp),%esi + (gdb) + 0x0109634b 30 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x109634b <__sigreturn+11>: call 0x11a0609 <__x86.get_pc_thunk.bx> + (gdb) + 0x011a0609 in __x86.get_pc_thunk.bx () from /lib/i386-gnu/libc.so.0.3 + 1: x/i $pc + => 0x11a0609 <__x86.get_pc_thunk.bx>: mov (%esp),%ebx + (gdb) + 0x011a060c in __x86.get_pc_thunk.bx () from /lib/i386-gnu/libc.so.0.3 + 1: x/i $pc + => 0x11a060c <__x86.get_pc_thunk.bx+3>: ret + (gdb) + 0x01096350 in __sigreturn (scp=0x102988c) at ../sysdeps/mach/hurd/i386/sigreturn.c:30 + 30 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096350 <__sigreturn+16>: add $0x15ccb0,%ebx + (gdb) + 35 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096356 <__sigreturn+22>: test %esi,%esi + (gdb) + 0x01096358 35 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096358 <__sigreturn+24>: je 0x10964f0 <__sigreturn+432> + (gdb) + 0x0109635e 35 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x109635e <__sigreturn+30>: testl $0x10100,0x4(%esi) + (gdb) + 0x01096365 35 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x1096365 <__sigreturn+37>: jne 0x10964f0 <__sigreturn+432> + (gdb) + __hurd_threadvar_location_from_sp (__sp=, __index=) at ../hurd/hurd/threadvar.h:94 + 94 ../hurd/hurd/threadvar.h: No such file or directory. + 1: x/i $pc + => 0x109636b <__sigreturn+43>: mov -0x38(%ebx),%ebp + (gdb) + __hurd_threadvar_location (__index=_HURD_THREADVAR_SIGSTATE) at ../hurd/hurd/threadvar.h:116 + 116 in ../hurd/hurd/threadvar.h + 1: x/i $pc + => 0x1096371 <__sigreturn+49>: mov %esp,%edx + (gdb) + __hurd_threadvar_location_from_sp (__sp=0x1029848, __index=_HURD_THREADVAR_SIGSTATE) at ../hurd/hurd/threadvar.h:94 + 94 in ../hurd/hurd/threadvar.h + 1: x/i $pc + => 0x1096373 <__sigreturn+51>: cmp 0x0(%ebp),%esp + (gdb) + 0x01096376 94 in ../hurd/hurd/threadvar.h + 1: x/i $pc + => 0x1096376 <__sigreturn+54>: jae 0x10964d0 <__sigreturn+400> + (gdb) + 0x0109637c 94 in ../hurd/hurd/threadvar.h + 1: x/i $pc + => 0x109637c <__sigreturn+60>: mov -0x15c(%ebx),%eax + (gdb) + 0x01096382 94 in ../hurd/hurd/threadvar.h + 1: x/i $pc + => 0x1096382 <__sigreturn+66>: and (%eax),%edx + (gdb) + 0x01096384 94 in ../hurd/hurd/threadvar.h + 1: x/i $pc + => 0x1096384 <__sigreturn+68>: mov -0x90(%ebx),%eax + (gdb) + 0x0109638a 94 in ../hurd/hurd/threadvar.h + 1: x/i $pc + => 0x109638a <__sigreturn+74>: add (%eax),%edx + (gdb) + _hurd_self_sigstate () at ../hurd/hurd/signal.h:165 + 165 ../hurd/hurd/signal.h: No such file or directory. + 1: x/i $pc + => 0x109638c <__sigreturn+76>: mov 0x8(%edx),%edi + (gdb) + 0x0109638f 165 in ../hurd/hurd/signal.h + 1: x/i $pc + => 0x109638f <__sigreturn+79>: test %edi,%edi + (gdb) + 0x01096391 165 in ../hurd/hurd/signal.h + 1: x/i $pc + => 0x1096391 <__sigreturn+81>: je 0x1096598 <__sigreturn+600> + (gdb) + __sigreturn (scp=0x102988c) at ../sysdeps/mach/hurd/i386/sigreturn.c:42 + 42 ../sysdeps/mach/hurd/i386/sigreturn.c: No such file or directory. + 1: x/i $pc + => 0x1096397 <__sigreturn+87>: mov %edi,(%esp) + (gdb) + 0x0109639a 42 in ../sysdeps/mach/hurd/i386/sigreturn.c + 1: x/i $pc + => 0x109639a <__sigreturn+90>: call 0x1051d70 <_hurd_sigstate_lock@plt> + (gdb) + 0x01051d70 in _hurd_sigstate_lock@plt () from /lib/i386-gnu/libc.so.0.3 + 1: x/i $pc + => 0x1051d70 <_hurd_sigstate_lock@plt>: jmp *0x864(%ebx) + (gdb) + _hurd_sigstate_lock (ss=ss@entry=0x1244008) at hurdsig.c:170 + 170 hurdsig.c: No such file or directory. + 1: x/i $pc + => 0x106bb90 <_hurd_sigstate_lock>: sub $0x1c,%esp + (gdb) + 0x0106bb93 170 in hurdsig.c + 1: x/i $pc + => 0x106bb93 <_hurd_sigstate_lock+3>: mov %ebx,0x14(%esp) + (gdb) + 0x0106bb97 170 in hurdsig.c + 1: x/i $pc + => 0x106bb97 <_hurd_sigstate_lock+7>: call 0x11a0609 <__x86.get_pc_thunk.bx> + (gdb) + 0x011a0609 in __x86.get_pc_thunk.bx () from /lib/i386-gnu/libc.so.0.3 + 1: x/i $pc + => 0x11a0609 <__x86.get_pc_thunk.bx>: mov (%esp),%ebx + (gdb) + 0x011a060c in __x86.get_pc_thunk.bx () from /lib/i386-gnu/libc.so.0.3 + 1: x/i $pc + => 0x11a060c <__x86.get_pc_thunk.bx+3>: ret + (gdb) + 0x0106bb9c in _hurd_sigstate_lock (ss=ss@entry=0x1244008) at hurdsig.c:170 + 170 in hurdsig.c + 1: x/i $pc + => 0x106bb9c <_hurd_sigstate_lock+12>: add $0x187464,%ebx + (gdb) + 0x0106bba2 170 in hurdsig.c + 1: x/i $pc + => 0x106bba2 <_hurd_sigstate_lock+18>: mov %esi,0x18(%esp) + (gdb) + 170 in hurdsig.c + 1: x/i $pc + => 0x106bba6 <_hurd_sigstate_lock+22>: mov 0x20(%esp),%esi + (gdb) + sigstate_is_global_rcv (ss=0x1244008) at hurdsig.c:162 + 162 in hurdsig.c + 1: x/i $pc + => 0x106bbaa <_hurd_sigstate_lock+26>: lea 0x57c0(%ebx),%eax + (gdb) + 0x0106bbb0 162 in hurdsig.c + 1: x/i $pc + => 0x106bbb0 <_hurd_sigstate_lock+32>: mov (%eax),%eax + (gdb) + 163 in hurdsig.c + 1: x/i $pc + => 0x106bbb2 <_hurd_sigstate_lock+34>: test %eax,%eax + (gdb) + 0x0106bbb4 163 in hurdsig.c + 1: x/i $pc + => 0x106bbb4 <_hurd_sigstate_lock+36>: je 0x106bbbc <_hurd_sigstate_lock+44> + (gdb) + 0x0106bbb6 163 in hurdsig.c + 1: x/i $pc + => 0x106bbb6 <_hurd_sigstate_lock+38>: cmpl $0x1,0x18(%esi) + (gdb) + 0x0106bbba 163 in hurdsig.c + 1: x/i $pc + => 0x106bbba <_hurd_sigstate_lock+42>: je 0x106bbe0 <_hurd_sigstate_lock+80> + (gdb) + _hurd_sigstate_lock (ss=ss@entry=0x1244008) at hurdsig.c:172 + 172 in hurdsig.c + 1: x/i $pc + => 0x106bbe0 <_hurd_sigstate_lock+80>: lea 0x4(%eax),%ecx + (gdb) + __spin_try_lock (__lock=0x124480c) at ../sysdeps/mach/i386/machine-lock.h:59 + 59 ../sysdeps/mach/i386/machine-lock.h: No such file or directory. + 1: x/i $pc + => 0x106bbe3 <_hurd_sigstate_lock+83>: mov $0x1,%edx + (gdb) + 0x0106bbe8 59 in ../sysdeps/mach/i386/machine-lock.h + 1: x/i $pc + => 0x106bbe8 <_hurd_sigstate_lock+88>: xchg %edx,0x4(%eax) + (gdb) + zyg: i don't get what you mean + are you starting it with gdb ? + braunr: yes: "gdb --args a.out" + ok + can't reproduce it + i get "Program received signal SIGHUP, Hangup. + " + then continue, then the program has exited + braunr: do you run it in gdb or without? + braunr: Ah "Program received signal SIGHUP, Hangup." is from + gdb.. try issue continue, not sure why gdb stops at SIGHUP (default?). + 10:34 < braunr> then continue, then the program has exited + gdb stops at signals + braunr: yes, try repeating that, but instead of continue, just issue + "si" + braunr: sorry.. you would need to remove that printf/fprintf, else it + gets too long. That's why I put a breakpoint. + a breakpoint ? + on the signal handler ? + braunr: yes, put a break after having entered the handler. Or edit + the pasted C code an remove that printf("got SIGHUP\n"); + i'm not sure that's correctly supported + and i can see why glibc would deadlock on the sigstate lock + don't do that :p + braunr: why does it deadlock? + because both the signal handler and messages from gdb will cause + common code paths to be taken + braunr: oh.. when I step instruction I'm inside an SIGTRAP handler + probably? + possible + i don't know the details but that's the kind of things i expect + and signals on the hurd are definitely buggy + i don't know if we support nesting them + i'd say we don't + braunr: I'll try to put a break beyond that xchg and continue + xhcg is probably the sigstate spinlock + xchg* + you'd need to reach the unlock instruction, which is probably + executed once the handler has finished running + braunr: yes :) ... one instruction beyond didn't help + braunr: thanks alot, putting a break in __sigreturn, after that + function has called _hurd_sigstate_unlock@plt works! + works ? + what did you want to do ? + braunr: I want to trace user code inside the signal handler, also how + we enter and how we leave. + well you can't do that inside, so no it doesn't work for you :/ + but that's a start i guess + braunr: I seem to do most normal things inside the handler, + step-instruction and put breaks. + ? + i thought that's what made the program deadlock + braunr: as you said earlier, the deadlock came when i "step + instruction" into the area between _hurd_sigstate_lock and + _hurd_sigstate_unlock. Otherwise I havn't had any issues. + but isn't the sigstate locked during the handler execution ? + braunr: no it locks and unlocks in __sigreturn which is done when + leaving the handler. + than how could it deadlock on handler entry ? + or perhaps the fact your handler was empty made the entry point + directly reach __sigreturn + hm no i don't buy it + the sigstate must also be locked on entry + braunr: there was never any problem with entering + then describe the problem with more details please + ah sorry + braunr: are you sure? there is minimal user-code run before the + signal is going into the handler. + you "step out of the handler" diff --git a/open_issues/glibc/0.4.mdwn b/open_issues/glibc/0.4.mdwn index ceb5ea21..f864469d 100644 --- a/open_issues/glibc/0.4.mdwn +++ b/open_issues/glibc/0.4.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,6 +10,8 @@ License|/fdl]]."]]"""]] [[!tag open_issue_glibc open_issue_libpthread]] +Things to consider doing when bumping the glibc SONAME. + # IRC, freenode, #hurd, 2012-12-14 diff --git a/open_issues/glibc/debian.mdwn b/open_issues/glibc/debian.mdwn index 331632f3..9886ec98 100644 --- a/open_issues/glibc/debian.mdwn +++ b/open_issues/glibc/debian.mdwn @@ -62,3 +62,70 @@ apter applying patches. If the Debian symbol versioning file is not up to date and the build of Debian packages fails due to this, putting `DPKG_GENSYMBOLS_CHECK_LEVEL=0` in the environment \`\`helps''; see `man dpkg-gensymbols`. + + +# IRC, freenode, #hurd, 2013-07-01 + + something seems to have changed with regard to patch handling in + eglibc 2.17 + pinotree: when i add a patch to series and use dpkg-buildpackage, + i'm told there are local modifications and the build stops :/ + any idea what i'm doing wrong ? + which steps do you do? + i extract the sources, copy the patch to debian/patches/hurd-i386, + add the appropriate line to debian/patches/series, call dch -i, then + dpkg-buildpackage + eglibc is a "3.0 (quilt)" format source package + this means its default patches are in a quilt-style system, and + they are applied on extraction + ok + and it can't detect new patches ? + so if you add a new patch to the global serie, you have to push + it manually + i have to revert them all ? + ok + how do i do that ? + quilt push -a + ok + thanks + remember to do that before starting the build, since the rest + assumes the quilt-style patches are fully applied + No push applies them, quilt pop -a reverts them + yeah, and he has to push the new over the dpkg-applied ones + Oh, aye + does quilt change series ? + no + ok + i mean, some commands do that + so i do everything i did, with an additional push, right ? + ok, screw me, i didn't get your question above :P + does that change your answer ? + does quilt change series ? + yes + if you import or create a new patch, it changes series indeed + ok + push or pop of patches does not + i'm doing it wron + g + btw, in a quilt patch stack you can easily import a new patch + using the import command + so for example you could do + apt-get source eglibc # or get it somehow else + cd eglibc-* + quilt import /location/of/my/patch + quilt push # now your patch is applied + ah thanks + dpkg-buildpackage as usual + that's what i was looking for + quilt new adds a new entry in series + y + or import, aye + braunr: if you want to learn quilt, a very good doc is its own, + eg /usr/share/doc/quilt/quilt.txt.gz + * bddebian has never actually used import + ok + it is basically a simple stack of patches + + braunr: yes, patch handling is a bit different + the arch-independant patches are applied by dpkg-source -x + and the arch-dependent patches are applied during build diff --git a/open_issues/glibc/debian/experimental.mdwn b/open_issues/glibc/debian/experimental.mdwn index 8d117e99..5168479d 100644 --- a/open_issues/glibc/debian/experimental.mdwn +++ b/open_issues/glibc/debian/experimental.mdwn @@ -11,6 +11,7 @@ License|/fdl]]."]]"""]] [[!tag open_issue_glibc]] Issues with the current 2.17 version of glibc/EGLIBC in Debian experimental. +Now in unstable. # IRC, OFTC, #debian-hurd, 2013-03-14 @@ -113,3 +114,62 @@ Issues with the current 2.17 version of glibc/EGLIBC in Debian experimental. eventually? (Some experimental package(s), but which?) that was libc0.3 packages which are indeed known to break the network + + +# IRC, freenode, #hurd, 2013-06-18 + + root@darnassus:~# dpkg-reconfigure locales + Generating locales (this might take a + while)... en_US.UTF-8...Segmentation fault + is it known ? + uh, no + + +## IRC, OFTC, #debian-hurd, 2013-06-19 + + btw i saw too the segmentation fault when generating locales + + +# IRC, OFTC, #debian-hurd, 2013-06-20 + + damn + hang at ext2fs boot + static linking issue, clearly + + +## IRC, freenode, #hurd, 2013-06-30 + + Mmm + __access ("/etc/ld.so.nohwcap", F_OK) at startup of ext2fs + deemed to fail.... + when does that happen? + at hwcap initialization + at least that's were ext2fs.static linked against libc 2.17 hangs + at startup + and this is indeed a very good culprit :) + ah, a debian patch + does anybody know a quick way to know whether one is the / ext2fs ? + :) + isn't the root fs given a special port? + I was thinking about something like this, yes + ok, boots + I'll build a 8~0 that includes the fix + so people can easily build the hurd package + Mmm, no, the bootstrap port is also NULL for normally-started + processes :/ + I don't understand why + ah, only translators get a bootstrap port :/ + perhaps CRDIR then + (which makes a lot of sense) + + +## IRC, freenode, #hurd, 2013-07-01 + + youpi: what is local-no-bootstrap-fs-access.diff supposed to fix ? + ext2fs.static linked againt debian glibc 2.17 + well, as long as you don't build & use ext2fs.static with it... + that's thing, i want to :) + +the + I'd warmly welcome a way to detect whether being the / translator + process btw + it seems far from trivial diff --git a/open_issues/glibc/t/tls-threadvar.mdwn b/open_issues/glibc/t/tls-threadvar.mdwn index 105a07c7..609d866b 100644 --- a/open_issues/glibc/t/tls-threadvar.mdwn +++ b/open_issues/glibc/t/tls-threadvar.mdwn @@ -64,3 +64,15 @@ dropped altogether, and `__thread` directly be used in glibc. I saw the mails, but didn't investigate at all [[!message-id "878vdyqht3.fsf@kepler.schwinge.homeip.net"]]. + + +# IRC, freenode, #hurd, 2013-07-08 + + tschwinge: apparently there were a lot of changes missing in the + threadvars branch I had commited long time ago + I'm gathering things + youpi: t/tls-threadvar you mean? + yes + tschwinge: yes, there were a lot other occurences of threadvars + stuff in various places + I'm building libc again, and will see what issue would remain diff --git a/open_issues/libc_variant_selection.mdwn b/open_issues/libc_variant_selection.mdwn index afcd9ae0..71370a43 100644 --- a/open_issues/libc_variant_selection.mdwn +++ b/open_issues/libc_variant_selection.mdwn @@ -1,4 +1,5 @@ -[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2010, 2011, 2013 Free Software Foundation, +Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -28,6 +29,28 @@ On Thu, Oct 07, 2010 at 11:22:46AM +0200, Samuel Thibault wrote: > Yes, you need to copy it by hand. Same for libc0.3-i686, we just need to > steal the cpuid code from the kfreebsd port of glibc. + +# IRC, freenode, #hurd, 2013-06-30 + + other than that, the hwcap system is not working for us yet, + right? + no but we'd like to use e.g. cpuid for that + like kfreebsd does + do they use cpuid for that? + i kind of lost myself in glibc's loading internals, trying to + find out where the hwcap bits come from + on linux it comes from the kernel + on kfreebsd aiui they use cpuid to figure it out from the process + itself + do you have any pointer to the kfreebsd way? iirc i had a look + in their sysdeps, but found nothing related to that + it's in local-sysdeps.diff aiui + +dl_platform_kfreebsd_i386_init + which fills dl_hwcap + called at _dl_sysdep_start + interesting + + --- Having working CPUID code inside [[glibc]] is also a prerequisite for proper diff --git a/open_issues/libnetfs_argument_parsing.mdwn b/open_issues/libnetfs_argument_parsing.mdwn new file mode 100644 index 00000000..e1e0d794 --- /dev/null +++ b/open_issues/libnetfs_argument_parsing.mdwn @@ -0,0 +1,62 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_hurd]] + +# IRC, freenode, #hurd, 2013-06-27 + + the arg parsing in libdiskfs and libnetfs differ :/ + afaics libdiskfs gets it right, libnetfs does not + what do you mean? + wrt to *_std_{runtime,startup}_argp + see eg netfs.h + libdiskfs/opts-std-runtime.c:const struct argp + diskfs_std_runtime_argp = + libdiskfs/opts-std-runtime.c-{ + libdiskfs/opts-std-runtime.c- std_runtime_options, parse_opt, + 0, 0, children + libdiskfs/opts-std-runtime.c-}; + but + libnetfs/std-runtime-argp.c:const struct argp + netfs_std_runtime_argp = { 0 }; + well there are no common startup/runtime options provided by + netfs + usually netfs-based translators put netfs_std_startup_argp as + child as their options, so if netfs starts providing options they would + work + ah + if you have a test showing issues, we can certainly look it :) + ok, m/b I was confused... + no worries, feel free to ask anytime + I thought about providing --update as common runtime flag, like + diskfs does, you think it's the right thing to do? + what would it do? + or should it be left for each translator to implement? + nothing by default I guess + options provided in libdiskfs are implemented and handled mostly + in libdiskfs itself + so imho a new option for libnetfs would be there because its + behaviour is implemented mostly within libnetfs itself + libdiskfs calls diskfs_reload_global_state + libnetfs could do the same, allowing translators to plug in + anything they wish + but I'll implement it in procfs for the time being + ah, its alias is remount + yes + I need that working for procfs + btw, I think I got your mount confusion thing figured out + but procfs has nothing to update/flush, all the information are + fetched at every rpc + yes + but we still need to ignore the flag + otherwise the set_options rpc fails + http://paste.debian.net/12938/ + whee, remounting proc works :) + :) diff --git a/open_issues/libpthread.mdwn b/open_issues/libpthread.mdwn index e2fda122..8e3fde71 100644 --- a/open_issues/libpthread.mdwn +++ b/open_issues/libpthread.mdwn @@ -1260,7 +1260,7 @@ Most of the issues raised on this page has been resolved, a few remain. i'll add traces to know which step causes the error -### IRC, freenode, #hurd, 2012-12-11 +#### IRC, freenode, #hurd, 2012-12-11 braunr: mktoolnix seems like a reproducer for the libports thread priority issue @@ -1273,7 +1273,7 @@ Most of the issues raised on this page has been resolved, a few remain. that's it, yes -### IRC, freenode, #hurd, 2013-03-01 +#### IRC, freenode, #hurd, 2013-03-01 braunr: btw, "unable to adjust libports thread priority: (ipc/send) invalid destination port" is actually not a sign of fatality @@ -1284,6 +1284,34 @@ Most of the issues raised on this page has been resolved, a few remain. weird sentence, agreed :p +#### IRC, freenode, #hurd, 2013-06-14 + + Hi, when running check for gccgo the following occurs (multiple + times) locking up the console + unable to adjust libports thread priority: (ipc/send) invalid + destination port + (not locking up the console, it was just completely filled with + messages)) + gnu_srs: are you running your translator as root ? + or, do you have a translator running as an unprivileged user ? + hm, invalid dest port + that's a bug :p + but i don't know why + i'll have to take some time to track it down + it might be a user ref overflow or something similarly tricky + gnu_srs: does it happen everytime you run gccgo checks or only + after the system has been alive for some time ? + (some time being at least a few hours, more probably days) + +#### IRC, freenode, #hurd, 2013-07-05 + + ok, found the bug about invalid ports when adjusting priorities + thhe hurd must be plagued with wrong deallocations :( + i have so many problems when trying to cleanly destroy threads + +[[libpthread/t/fix_have_kernel_resources]]. + + ### IRC, freenode, #hurd, 2013-03-11 youpi: oh btw, i noticed a problem with the priority adjustement @@ -1296,6 +1324,171 @@ Most of the issues raised on this page has been resolved, a few remain. uh indeed +### IRC, freenode, #hurd, 2013-07-01 + + braunr: it seems as if pfinet is not prioritized enough + I'm getting network connectivity issues when the system is quite + loaded + loaded with what ? + it could be ext2fs having a lot more threads than other servers + building packages + I'm talking about the buildds + ok + ironforge or others ? + they're having troubles uploading packages while building stuff + ironforge and others + that happened already in the past sometimes + but at the moment it's really pronounced + i don't think it's a priority issue + i think it's swapping + ah, that's not impossible indeed + but why would it swap? + there's a lot of available memory + a big file is enough + it pushes anonymous memory out + to fill 900MiB memory ? + i see 535M of swap on if + yes + ironforge is just building libc + and for some reason, swapping is orders of magnitude slower than + anything else + not linking it yet + i also see 1G of free memory on it + that's what I meant with 900MiB + so at some point, it needed a lot of memory, caused swapping + and from time to time it's probably swapping back + well, pfinet had all the time to swap back already + I don't see why it should be still suffering from it + swapping is a kernel activity + ok, but once you're back, you're back + unless something else pushes you out + if the kernel is busy waiting for the default pager, nothing makes + progress + (eccept the default pager hopefully) + sure but pfinet should be back already, since it does work + so I don't see why it should wait for something + the kernel is waiting + and the kernel isn't preemptibl + e + although i'm not sure preemption is the problem here + well + what I don't understand is what we have changed that could have so + much impact + the only culprit I can see is the priorities we have changed + recently + do you mean it happens a lot more frequently than before ? + yes + way + ok + ironforge is almost unusable while building glibc + I've never seen that + that's weird, i don't have these problems on darnassus + but i think i reboot it more often + could be a scalability issue then + combined with the increased priorities + if is indeed running full time on the host, whereas swapping + issues show the cpu being almost idle + loadavg is high too so i guess there are many threads + 0 971 3 -20 -20 1553 305358625 866485906 523M 63M * S 0 972 3 -20 -20 1434 125237556 719443981 483M 5.85M * S around 1k5 each + that's quite usual + could be the priorities then + but i'm afraid that if we lower them, the number of threads will + grow out of control + (good thing is i'm currently working on a way to make libpthread + actually remove kernel resources) + but the priorities should be the same in ext2fs and pfinet, + shouldn't they? + yes but ext2 has a lot more threads than pfinet + the scheduler only sees threads, we don't have a grouping feature + right + we also should remove priority depressing in glibc + (in sched_yield) + it affects spin locks + + youpi: is it normal to see priorities of 26 ? + braunr: we have changed the nice factor + ah, factor + Mm, I'm however realizing the gnumach kernel running these systems + hasn't been upgraded in a while + it may not even have the needed priority levels + ar euare you using top right now on if ? + hm no i don't see it any more + well yes, could be the priorities .. + I've rebooted with an upgraded kernel + no issue so far + package uploads will tell me on the long run + i bet it's also a scalability issue + but why would it appear now only? + until the cache and other data containers start to get filled, + processing is fast enough that we don't see it hapenning + sure, but I haven't seen that in the past + oh it's combined with the increased priorities + even after a week building packages + what i mean is, increased priorities don't affect much if threads + porcess things fast + things get longer with more data, and then increased prioritis + give more time to these threads + and that's when the problem appears + but increased priorities give more time to the pfinet threads too, + don't they? + yes + so what is different ? + but again, there are a lot more threads elsewhere + with a lot more data to process + sure, but that has alwasy been so + hm + really, 1k5 threads does not surprise me at all :) + 10k would + there aren't all active either + yes + but right, i don't know why pfinet would be given less time than + other threads .. + compared to before + particularly on xen-based buildds + libpthread is slower than cthreads + where it doesn't even have to wait for netdde + threads need more quanta to achieve the same ting + perhaps processing could usually be completed in one go before, + and not any more + we had a discussion about this with antrik + + youpi: concerning the buildd issue, i don't think pfinet is + affected actually + but the applications using the network may be + why using the network would be a difference ? + normal applications have a lower priority + what i mean is, pfinet simply has nothing to do, because normal + applications don't have enough cpu time + (what you said earlier seemed to imply pfinet had issues, i don't + think it has) + it should be easy to test by pinging the machine while under load + we should also check the priority of the special thread used to + handle packets, both in pfinet and netdde + this one isn't spawned by libports and is likely to have a lower + priority as well + + youpi: you're right, something very recent slowed things down a + lot + perhaps the new priority factor + well not the factor but i suppose the priority range has been + increased + +[[nice_vs_mach_thread_priorities]]. + + braunr: haven't had any upload issue so far + over 20 uploads + while it was usually 1 every 2 before... + so it was very probably the kernel missing the priorities levels + ok + i think i've had the same problem on another virtual machine + with a custom kernel i built a few weeks ago + same kind of issue i guess + it's fine now, and always was on darnassus + ## IRC, freenode, #hurd, 2012-12-05 diff --git a/open_issues/libpthread/t/fix_have_kernel_resources.mdwn b/open_issues/libpthread/t/fix_have_kernel_resources.mdwn index 10577c1e..4e35161f 100644 --- a/open_issues/libpthread/t/fix_have_kernel_resources.mdwn +++ b/open_issues/libpthread/t/fix_have_kernel_resources.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,7 +10,9 @@ License|/fdl]]."]]"""]] [[!tag open_issue_libpthread]] -`t/have_kernel_resources` +`t/fix_have_kernel_resources` + +Address problem mentioned in [[/libpthread]], *Threads' Death*. # IRC, freenode, #hurd, 2012-08-30 @@ -19,3 +21,178 @@ License|/fdl]]."]]"""]] tschwinge: i.e. the ability to tell the kernel where the stack is, so it's unmapped when the thread dies which requiring another thread to perform this deallocation + + +## IRC, freenode, #hurd, 2013-05-09 + + braunr: Speaking of which, didn't you say you had another "easy" + task? + bddebian: make a system call that both terminates a thread and + releases memory + (the memory released being the thread stack) + this way, a thread can completely terminates itself without the + assistance of a managing thread or deferring work + braunr: That's "easy" ? :) + bddebian: since it's just a thread_terminate+vm_deallocate, it is + something like thread_terminate_self + But a syscall not an RPC right? + in hurd terminology, we don't make the distinction + the only real syscalls are mach_msg (obviously) and some to get + well known port rights + e.g. mach_task_self + everything else should be an RPC but could be a system call for + performance + since mach was designed to support clusters, it was necessary that + anything not strictly machine-local was an RPC + and it also helps emulation a lot + so keep doing RPCs :p + + +## IRC, freenode, #hurd, 2013-05-10 + + i'm not sure it should only apply to self though + youpi: can we get a quick opinion on this please ? + i've suggested bddebian to work on a new RPC that both terminates + a thread and releases its stack to help fix libpthread + and initially, i thought of it as operating only on the calling + thread + do you see any reason to make it work on any thread ? + (e.g. a real thread_terminate + vm_deallocate) + (or any reason not to) + thread stack deallocation is always a burden indeed + I'd tend to think it'd be useful, but perhaps ask the list + + +## IRC, freenode, #hurd, 2013-06-26 + + looks like there is a port right leak in libpthread + grmbl, the port leak seems to come from mach_port_destroy being + buggy :/ + hum, apparently we're not the only ones to suffer from port leaks + wrt mach_port_destroy + ew, libpthread is leaking + memory or ports? + both + sounds great ;) + as it is, libpthread doesn't destroy threads + it queues them so they're recycled late + r + but there is confusion between the thread structure itself and its + internal resources + i.e. there is pthread_alloc which allocates a thread structure, + and pthread_create which allocates everything else + but on pthread_exit, nothing is destroyed + when a thread structure is reused, its internal resources are + replaced by new instances + oh + it's ok for joinable threads but most of our threads are detached + pinotree: as expected, it's bigger than expected :p + so i won't be able to write a quick fix + the true way to fix this is make it possible for threads to free + their own resources + let's do that :p + ok, got the new thread termination function, i'll build eglibc + package providing it, then experiment with libpthread + braunr: iirc there's also a tschwinge patch in the debian eglibc + about that + ah + libpthread_fix.diff + i see + thanks for the notice + bddebian: + http://www.sceen.net/~rbraun/0001-thread_terminate_deallocate.patch + bddebian: this is what it looks like + see, short and easy + Aye but didn't youpi say not to bother with it?? + he did ? + i don't remember + I thought that was the implication. Or maybe that was the one I + already did!? + i'd be interested in reading that + anyway, there still are problems in libpthread, and this call is + one building block to fix some of them + some important ones + (big leaks) + + +## IRC, freenode, #hurd, 2013-06-29 + + damn, i fix leaks in libpthread, only to find out leaks somewhere + else :( + bddebian: ok, actually it was a bit more complicated than what i + showed you + because in addition to the stack, the call must also release the + send right in the caller's ipc space + (it can't be released before since there would be no mean to + reference the thread to destroy) + or perhaps it should strictly be reserved to self termination + hmm + yes it would probably be simpler + but it should be a decent compromise + i'm close to having a libpthread that doesn't leak anything + and that properly destroys threads and their resources + + +## IRC, freenode, #hurd, 2013-06-30 + + bddebian: ok, it was even more tricky, because the kernel would + save the return value on the user stack (which is released by the call + and then invalid) before checking for asynchronous software traps (ASTs, + a kind of software interrupts in mach), and terminating the calling + thread is done by a deferred AST ... :) + hmm, making threads able to terminate themselves makes rpctrace a + bit useless :/ + well, more restricted + + ok so, tough question : + i have a small test program that creates a thread, and inspect its + state before any thread dies + i can see msg_report_wait requests when using ps + (one per thread) + one of these requests create a new receive right, apparently for + the second thread in the test program + each time i use ps, i can see the sequence numbers of two receive + rights increase + i guess these rights are related to proc and signal handling per + thread + but i can't find what create them + does anyone know ? + tschwing_: ^ :) + + again, too many things wrong elsewhere to cleanly destroy threads + .. + something is deeply wrong with controlling terminals .. + + +## IRC, freenode, #hurd, 2013-07-01 + + youpi: if you happen to notice what receive right is created for + each thread (beyond the obvious port used for blocking and waking up), + please let me know + it's the only port leak i have with thread destruction + and i think it's related to the proc server since i see the + sequence number increase every time i use ps + + pinotree: my change doesn't fix all the pthread leaks but it's a + lot better + bddebian: i've spent almost the whole week end trying to find the + last port leak without success + there is some weird bug related to the controlling tty that hits + me every time i try to change something + it's the same bug that prevents ttys from being correctly closed + when using ssh or screen + well maybe not the same, but it's close + some stale receive right kept around for no apparent reason + and i can't find its source + + +## IRC, freenode, #hurd, 2013-07-02 + + and btw, i don't think i can make my libpthread patch work + i'll just aim at avoiding leaks, but destroying threads and their + related resources depends on other changes i don't clearly see + + +## IRC, freenode, #hurd, 2013-07-03 + + grmbl, i don't want to give up thread destruction .. diff --git a/open_issues/libpthread_assertion_thread_prevp.mdwn b/open_issues/libpthread_assertion_thread_prevp.mdwn index e8160528..f93f07d6 100644 --- a/open_issues/libpthread_assertion_thread_prevp.mdwn +++ b/open_issues/libpthread_assertion_thread_prevp.mdwn @@ -87,3 +87,23 @@ failed"]] removing the libports_stability patch exposed bugs in libpthread, triggering assertions when queueing/dequeue threads from a queue (but i don't know which one / in which function) + + +## IRC, freenode, #hurd, 2013-06-25 + + braunr: + https://buildd.debian.org/status/fetch.php?pkg=libmemcached&ver=1.0.17-2&arch=hurd-i386&stamp=1372165732 + make: ./pthread/pt-internal.h:122: __pthread_enqueue: Assertion + `thread->prevp == 0' failed. \o/ + (it should rather be /o\, but better pretend not) + pinotree: yes, we regularly see it + pinotree: how long has the machine been running at this point ? + dunno, you should ask samuel about that + does it happen after N hours/days? + a few days of moderate to high activity yes + ah ok + and i actually see this error much more often when i disable the + libports stability patch in the hurd debian package + so i guess something is wrong with thread recycling + but i wanted to completely rewrite that part with the new kernel + call i asked bddebian to work on :) diff --git a/open_issues/mondriaan_memory_protection.mdwn b/open_issues/mondriaan_memory_protection.mdwn new file mode 100644 index 00000000..2c7b9ba1 --- /dev/null +++ b/open_issues/mondriaan_memory_protection.mdwn @@ -0,0 +1,85 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +. + + +# IRC, freenode, #hurd, 2013-07-02 + + in any case, what I wanted to check is if current hurd support + PIE + I just saw samuel posted some fixes to have PIE working in hurd + are those included in the official image? + sure + it's just a trivial fixup in some address calculation code + youpi: nice + and does anyone know how complex would it be to implement some + hackish support to have non-overlapping virtual addresses for + applications supporting PIE? + not too difficult + really? I didn't expect such an answer XD + I'd like to have something similar to a SASOS + (single address space os) + ? + you mean an sasos on top of mach ? + yes, but only for a few apps I want to evaluate + i see + the optimal would be to have all of hurd's servers on that mode + you'l probably need to implement a small allocator but other than + that it shouldn't be too hard, yes + uh ?? + but running on 32 bits can be a problem here + and not hurdish at all + what's not hurdish? + we do want address space separation + well, you can have multiple address spaces (page tables), but + without overlapping addresses between them + that's exactly what I'm looking for + sorry i don't see what you mean + if you run several servers in the same address space, they can + corrupt each other + we don't want that + it's that simple + yes, sorry, I didn't explain myself + I want a separate address space on each server + but I want all memory allocations to be on addresses unique to + the whole OS + that still doesn't make sense + well, it will still be secure + but I know it does not make sense per se + I want to do some experiments with a simulator + why do you want them non overlapping if they're separate ? + well, in my simulator I wouldn't need to change the page tables, + protection is provided through other means + segmentation ? + that's one possibility + (small address spaces) + what do you have in mind ? + it wouldn't be on top of mach anyway then + mach implements paging + what I'm simulating is something of the likes of Mondriaan + (http://www.cs.utexas.edu/~witchel/pubs/mmp-asplos2002.pdf) + paging is ok for me + 19:06 < xscript> well, in my simulator I wouldn't need to change + the page tables, protection is provided through other means + it didn't sound so + I meant switching page tables (cr3, etc) + mach does that + I know, I know, I can just ignore that part for the moment + ok + for now, I'd like to morph hurd into a SASOS using one page table + per process + I just wanted to know how hard that would be, without starting + with a full dive into the code + there are other options (OSes, microkernels), but none of them + provides as many readily-available applications as hurd + I suppose MINIX would also be easy to modify, but there's less + apps there, and I also would like to tamper with MIG + I just wonder how hard it would be to modify MIG diff --git a/open_issues/some_todo_list.mdwn b/open_issues/some_todo_list.mdwn index 80592abf..48c2944d 100644 --- a/open_issues/some_todo_list.mdwn +++ b/open_issues/some_todo_list.mdwn @@ -42,6 +42,21 @@ From Marcus, 2002: * Translators * Does settrans -g work? -fg? * Does fsysopts work? Does setting options with fsysopts work? + + IRC, freenode, #hurd, 2013-05-23: + + fsysopts /servers/socket/2 works by /1 gives Operation not + supported. + + [[!taglink open_issue_hurd]]. + + ah right, some servers don't implement that + work around this by using showtrans + fsysopts asks the server itself how it's running, usually giving + its command name and options + showtrans asks the parent how it starts a passive translator + attached to the node + Yes showtrans works :), thanks. * Does stat() work on all translated nodes and give proper data? * What about chown, chmod (some translators should pass this through to the underlying node, esp in /dev!) * Does statfs give correct data? diff --git a/open_issues/translator_stdout_stderr.mdwn b/open_issues/translator_stdout_stderr.mdwn index 14ea1c6d..89efd4e1 100644 --- a/open_issues/translator_stdout_stderr.mdwn +++ b/open_issues/translator_stdout_stderr.mdwn @@ -1,5 +1,5 @@ -[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation, -Inc."]] +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011, 2013 Free Software +Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -9,25 +9,69 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!tag open_issue_hurd]] +[[!toc]] -There have been several discussions and proposals already, about adding a -suitable logging mechanism to translators, for example. +# "Weird Issue" -Decide / implement / fix that (all?) running (passive?) translators' output -should show up on the (Mach / Hurd) console / syslog. +## IRC, freenode, #hurd, 2013-07-01 +[[!tag open_issue_hurd]] -[[!taglink open_issue_documentation]]: [[!message-id -"87oepj1wql.fsf@becket.becket.net"]] + oh, btw, there is this weird issue that I cannot figure out + I noticed that there is no newline printed by /hurd/init after + printing " proc" and " auth" + but there *is* a printf("\n"); fflush(stdout); in there + it's just not working + iirc a newline is supposed to be printed after all the essential + servers have been started + that one + yes + but this doesn't happen + for whatever reason printf("foo"); yields no output + how are proc and auth printed ? + neither does printf("%s", "foo"); + using printf + but printf("%i fooo", 4); works + uh + ?? + and does printf("loooooooooong line") worker? + no + uh + -er + and yes, the code is always fflushing stdout + perhaps trying to put a sleep(1); to check for timing issues? + yes, I suspect something like that + b/c *sometimes* my hurd just freezes at this point + ??? + after printing proc auth and not printing the newline + such horror stories . + .. + and I *think* that putting more printfs there for testing + purposes makes this worse, but I'm not sure about this + in case you need to debug using printf, there is the mach_print + system call + (in -dbg kernels) +[[microkernel/mach/gnumach/interface/syscall/mach_print]]. -[[!taglink open_issue_documentation]]: Neal once had written an email on this -topic. + what's wrong with using printf? + you need to write the little assembly call yourself, where you + intend to use it, because it's not exported by glibc + printf is an RPC + teythoon: RPCs are complicated stuff + in particular in core glibc parts like signal handling + and printf goes through the console translator + also, if you don't yet have a console server available, it comes + in handy + better direct output directly to the kernel -IRC, freenode, #hurd, 2011-11-06 +# `stderr` buffered + +## IRC, freenode, #hurd, 2011-11-06 + +[[!tag open_issue_hurd]] about CLI_DEBUG, you can use #define CLI_DEBUG(fmt, ...) { fprintf(stderr, fmt, ## __VA_ARGS__); fflush(stderr); } @@ -40,7 +84,24 @@ IRC, freenode, #hurd, 2011-11-06 That sounds wrong. -IRC, freenode, #hurd, 2011-11-23 +# Logging + +[[!tag open_issue_hurd]] + +There have been several discussions and proposals already, about adding a +suitable logging mechanism to translators, for example. + +Decide / implement / fix that (all?) running (passive?) translators' output +should show up on the (Mach / Hurd) console / syslog. + +[[!taglink open_issue_documentation]]: [[!message-id +"87oepj1wql.fsf@becket.becket.net"]] + +[[!taglink open_issue_documentation]]: Neal once had written an email on this +topic. + + +## IRC, freenode, #hurd, 2011-11-23 we'd need a special logging task for hurd servers if syslog would work, that could be used optionally diff --git a/open_issues/user-space_device_drivers.mdwn b/open_issues/user-space_device_drivers.mdwn index 8cde8281..7331bb54 100644 --- a/open_issues/user-space_device_drivers.mdwn +++ b/open_issues/user-space_device_drivers.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2009, 2011, 2012 Free Software Foundation, +[[!meta copyright="Copyright © 2009, 2011, 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -154,12 +154,100 @@ A similar problem is described in < braunr> s/disk/storage/ -### IRC, freenode, #hurd, 2012-04-25 +#### IRC, freenode, #hurd, 2012-04-25 btw, remember the initrd thing? I just came across task.c in libstore/ :) +#### IRC, freenode, #hurd, 2013-06-24 + + we added a new initrd command to gnumach, to expose a new mach + device, which ext2fs can open and unzip + we consider replacing that with simply putting the data in a dead + process + s/process/task + and let ext2fs read data from the task, and kill it when done + ok + alternatively, tmps would work with an initial .tar.gz payload + that would be best for memory usage + tmpfs* + can't we replace the initrd concept with sub/neighbourhood? + setting up tmpfs with an initial payload could be done with a + bootstrap subhurd + yes + but it seems to me that having tmpfs being able to have an initial + payload is interesting + is there any advantage of the tmpfs translator prefilled with a + tarball over ext2fs with copy & bunzip? + memory usage + ext2fs with copy&bunzip takes memory for zeroes + and we have to forecast how much data might be stored + (if writable) + ah sure + but why would it have to be in the tmpfs translator? I why not + start the translator and have tar extract stuff there? + with the livecd I had trouble replacing the root translator, but + when using subhurds that shouldn't be a prwoblem at all + I don't have a real opinion on this + except that people don't usually like initrd :) + 12:43 < teythoon> but why would it have to be in the tmpfs + translator? I why not start the translator and have tar extract stuff + there? + that sounds an awful lot like an initramfs + yes, exactly, without actually having an initramfs of course + yep + i actually prefer that way too + a system on a r/o isofs cannot do much, but it can do this + on the other hand, i wouldn't spend much time on a virtio disk + driver for now + the hurd as it is can't boot on a device that isn't managed by the + kernel + we'd need to change the boot protocol + + +#### IRC, freenode, #hurd, 2013-06-28 + + I'm tempted to redo a livecd, simpler and without the initrd + hack that youpi used for d-i + initrd hack ? + you mean more a la initramfs then ? + no, I thought about using a r/o isofs translator, but instead of + fixing that one up with a r/w overlay and lot's of firmlinks like I used + to, it would just start an ext2fs translator with copy on an image stored + on the iso and start a subhurd + why a subhurd ? + neighbourhurd even + b/c back in the days I had trouble replacing / + yes, that's hard + subhurd would take of that for free + are you sure ? + somewhat + i'm not, but this requires thorough thinking + and i'm not there yet + y would it not? + just start a subhurd and let that one take over the console and + let the user and d-i play nicely in that environment + no hacks involved + because it would require sharing things between the two system + instances, and that's not easy + no but the bootstrap system does nothing after launching the + subhurd + I mean yes, technically true, but why would it be hard to share + with someone who does nothing? + the context isn't well defined enough to clearly state anything + if you don't use the resources of the first hurd, that's ok + otherwise, it may be easy or not, i don't know yet + you think it's worth a shot and see what issues crop up? + sure + definitely + it doesn't sound complicated at all + it's easy enough to the point we see something goes wrong or works + completely + so worth testin + cool :) + + ### IRC, freenode, #hurd, 2012-07-17 OK, here is a stupid question I have always had. If you move -- cgit v1.2.3