summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
-rw-r--r--capability.mdwn20
-rw-r--r--community/gsoc/2013/hacklu.mdwn617
-rw-r--r--community/gsoc/2013/nlightnfotis.mdwn450
-rw-r--r--community/gsoc/project_ideas/mtab.mdwn98
-rw-r--r--community/gsoc/project_ideas/mtab/discussion.mdwn907
-rw-r--r--hurd.mdwn1
-rw-r--r--hurd/coding_style.mdwn59
-rw-r--r--hurd/console/discussion.mdwn10
-rw-r--r--hurd/debugging.mdwn27
-rw-r--r--hurd/debugging/rpctrace.mdwn4
-rw-r--r--hurd/translator.mdwn2
-rw-r--r--hurd/translator/auth.mdwn13
-rw-r--r--hurd/translator/firmlink.mdwn14
-rw-r--r--hurd/translator/netio.mdwn2
-rw-r--r--hurd/translator/proc.mdwn53
-rw-r--r--hurd/translator/procfs/jkoenig/discussion.mdwn20
-rw-r--r--hurd/translator/socketio.mdwn30
-rw-r--r--hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn129
-rw-r--r--libpthread.mdwn41
-rw-r--r--microkernel/mach/deficiencies.mdwn128
-rw-r--r--microkernel/mach/gnumach/debugging.mdwn5
-rw-r--r--microkernel/mach/gnumach/interface/syscall/mach_print.mdwn29
-rw-r--r--microkernel/mach/history.mdwn134
-rw-r--r--microkernel/mach/mig.mdwn6
-rw-r--r--microkernel/mach/mig/structured_data.mdwn119
-rw-r--r--open_issues/64-bit_port.mdwn20
-rw-r--r--open_issues/anatomy_of_a_hurd_system.mdwn189
-rw-r--r--open_issues/arm_port.mdwn1
-rw-r--r--open_issues/code_analysis.mdwn14
-rw-r--r--open_issues/dde.mdwn44
-rw-r--r--open_issues/gdb_signal_handler.mdwn403
-rw-r--r--open_issues/glibc/0.4.mdwn4
-rw-r--r--open_issues/glibc/debian.mdwn67
-rw-r--r--open_issues/glibc/debian/experimental.mdwn60
-rw-r--r--open_issues/glibc/t/tls-threadvar.mdwn12
-rw-r--r--open_issues/libc_variant_selection.mdwn25
-rw-r--r--open_issues/libnetfs_argument_parsing.mdwn62
-rw-r--r--open_issues/libpthread.mdwn197
-rw-r--r--open_issues/libpthread/t/fix_have_kernel_resources.mdwn181
-rw-r--r--open_issues/libpthread_assertion_thread_prevp.mdwn20
-rw-r--r--open_issues/mondriaan_memory_protection.mdwn85
-rw-r--r--open_issues/some_todo_list.mdwn15
-rw-r--r--open_issues/translator_stdout_stderr.mdwn87
-rw-r--r--open_issues/user-space_device_drivers.mdwn92
44 files changed, 4243 insertions, 253 deletions
diff --git a/capability.mdwn b/capability.mdwn
index 7219cdce..0ebe5cd4 100644
--- a/capability.mdwn
+++ b/capability.mdwn
@@ -1,5 +1,5 @@
-[[!meta copyright="Copyright © 2007, 2008, 2010, 2011 Free Software Foundation,
-Inc."]]
+[[!meta copyright="Copyright © 2007, 2008, 2010, 2011, 2013 Free Software
+Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -77,6 +77,22 @@ port|microkernel/mach/port]]. As in UNIX (see above), they are not
[[persistent|persistency]].
+## IRC, freenode, #hurd, 2013-07-01
+
+ <nlightnfotis> I have read plenty of documents, and wrapped my head around
+ most Hurd concepts, but I still have not understood well what
+ capabilities are.
+ <youpi> Mmm, which capabilities?
+ <youpi> AIUI, the Hurd doesn't really have a notion of capabilites, just a
+ notion of owning a port right
+ <nlightnfotis> From what I have understood (from the critique) they
+ reference ports so they objects can be referenced via them
+ <youpi> (which provides processes a way for doing things)
+ <youpi> ok, so we are talking about the same thing, I guess
+ <nlightnfotis> ahh, that's cool. I thought there was more to the story that
+ I couldn't understand
+
+
# Further Reading
* [[Mach port|microkernel/mach/port]]
diff --git a/community/gsoc/2013/hacklu.mdwn b/community/gsoc/2013/hacklu.mdwn
new file mode 100644
index 00000000..d0185c60
--- /dev/null
+++ b/community/gsoc/2013/hacklu.mdwn
@@ -0,0 +1,617 @@
+[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!toc]]
+
+
+# IRC, freenode, #hurd, 2013-06-23
+
+ <hacklu> braunr: sorry for the late reply. Honestly to say, the school
+ works had taken most of my time these days. I haven't got any
+ siginificant progress now. I am trying to write a little debugger demo on
+ Hurd.
+ <hacklu> braunr: things goes more hard than I think, these are some
+ differences between ptrace() on Hurd and Linux. I am trying to solve
+ this.
+
+
+# IRC, freenode, #hurd, 2013-06-24
+
+ <hacklu> this is my weekly report
+ http://hacklu.com/blog/gsoc-weekly-report1-117/.
+ <hacklu> and I have two main questions when I read the gdb source code.
+ <hacklu> 1/What is the S_exception_raise_request()? 2/what is the role of
+ ptrace in gdb port on Hurd?
+ <youpi> hacklu: where did you see S_exception_raise_request?
+ <hacklu> in gdb/gnu-nat.c
+ <youpi> ah, in gdb
+ <hacklu> yeah. and I have read the <The hurd hacking guide>. is says the S_
+ start means server stub.
+ <youpi> yes
+ <youpi> what happens is that gnu_wait keeps calling mach_msg
+ <youpi> to get a message
+ <youpi> then it passes that message to the various stubs servers
+ <youpi> see just below, it calls exc_server, among others
+ <youpi> and that's exc_server which ends up calling
+ S_exception_raise_request, if the message is an exception_raise request
+ <youpi> exc_server is a mere multiplexer, actually
+ <tschwinge> S_exception_raise_request is the implementation of the request
+ part (so one half of a typical RPC) of the Mach exception interface.
+ <tschwinge> See gdb/exc_request.defs in GDB and include/mach/exc.defs in
+ Mach.
+ <hacklu> youpi: how gnu_wait pass one message to exc_server? in which
+ function?
+ <youpi> in gnu_wait()
+ <youpi> && !exc_server (&msg.hdr, &reply.hdr)
+ <hacklu> oh, I see this.
+ <hacklu> firstly I think it is a type check simply.
+ <youpi> see the comment: "handle what we got"
+ <tschwinge> The Hurd's proc server also is involved in the exception
+ passing protocol (see its source code).
+ <hacklu> tschwinge: I will check the source code later. is the exception
+ take place in this way: 1. the inferior call ptrace(TRACE_ME). 2.the gdb
+ call task_set_exception_port. 3. mach send a notification to the
+ exception port set before. 4. gdb take some action.
+ <tschwinge> hacklu: Yes, that's it, roughly. The idea is that GDB replaces
+ a process' standard exception port, and replaces it "with itself", so
+ that when the process that is debugged receives and exception (by Mach
+ sending a exception_raise RPC), GDB can then catch that and act
+ accordingly.
+ <tschwinge> hacklu: As for your other questions, about ptrace: As you can
+ see in [glibc]/sysdeps/mach/hurd/ptrace.c, ptrace on Hurd is simply a
+ wrapper around vm_read/write and more interfaces.
+ <tschwinge> hacklu: As the GDB port for Hurd is specific to Hurd by
+ definition, you can also directly use these calls in GDB for Hurd.
+ <tschwinge> ..., as it is currently done.
+ <hacklu> and in detail, the part 3 mach send a notification to the
+ excetption port is like this: gnu_wait get the message in mach_msg, and
+ then pass it to exc_serer by exc_server(),then exc_server call
+ S_exception_raise_request()? ?
+ <hacklu> tschwinge: yeah, I have see the ptrace.c. I was wonder about
+ nobody use ptrace in Hurd except TRACEME...
+ <tschwinge> hacklu: Right about »and in detail, [...]«.
+ <tschwinge> hacklu: It would be very good (and required for your
+ understanding anyway), if you could write up a list of things that
+ happens when a process (both under the control of GDB as well as without
+ GDB) is sent an exception (due to a breakpoint instruction, for example).
+ <tschwinge> Let me look something up.
+ <hacklu> tschwinge: what's the function of exc_server? if I can get the
+ notification in mach_msg().
+ <youpi> to multiplex the message
+ <youpi> i.e. decoding it, etc. up to calling the S_ function with the
+ proper parameters
+ <youpi> exc_server being automatically generated, that saves a lot of code
+ <tschwinge> That is generated by MIG from the gdb/exc_request.defs file.
+ <tschwinge> You'll find the generated file in the GDB build directory.
+ <hacklu> I have wrote down the filenames. after this I will check that.
+ <tschwinge> hacklu: I suggest you also have a look at the Mach 3 Kernel
+ Principles book,
+ <http://www.gnu.org/software/hurd/microkernel/mach/documentation.html>.
+ <tschwinge> This also has some explanation of the thread/task's exception
+ mechanism.
+ <tschwinge> And of course, explains the RPC mechanism, which the exception
+ mechanism is built upon.
+ <tschwinge> And then, really make a step-by-step list of what happens; this
+ should help to better visualize what's going on.
+ <hacklu> ok. later I will update this list on my blog.
+ <tschwinge> hacklu: I cannot tell off-hand why GDB on Hurd is using
+ ptrace(PTRACE_TRACEME) instead of doing these calls manually. I will
+ have to look that up, too.
+ <hacklu> tschwinge: thanks.
+ <tschwinge> hacklu: Anyway -- you're asking sensible questions, so it seems
+ you're making progress/are on track. :-)
+ <hacklu> tschwinge: there is something harder than I had thought, I haven't
+ got any meaningful progress. sorry for this.
+ <tschwinge> hacklu: That is fine, and was expected. :-) (Also, you're
+ still busy with university.)
+ <hacklu> I will show more time and enthusiasm on this.
+ <tschwinge> hacklu: Oh, and one thing that may be confusing: as you may
+ have noticed, the names of the same RPC functions are sometimes slightly
+ different if different *.defs files. What is important is the subsystem
+ number, such as 2400 in [GDB]/gdb/exc_request.defs (and then incremented
+ per each routine/simpleroutine/skip directive).
+ <tschwinge> hacklu: Just for completeness, [hurd]/hurd/subsystems has a
+ list of RPC subsystems we're using.
+ <tschwinge> And the name given to routine 2400, for example, is just a
+ "friendly name" that is then used locally in the code where the *.defs
+ file has been processed by MIG.
+ <tschwinge> What a clumsy explanation of mine. But you'll get the idea, I
+ think. ;-)
+ <tschwinge> hacklu: And don't worry about your progress -- you're making a
+ lot of progress already (even if it doesn't look like it, because you're
+ not writing code), but the time spent on understanding these complex
+ issues (such as the RPC mechanism) definitely counts as progress, too.
+ <hacklu> tschwinge: not clearly to got it as I am not sensitive with the
+ MIG's grammer. But I know, the exc is the routine 2400's alias name?
+ <tschwinge> hacklu: I'd like to have you spend enough time to understand
+ these fundamental concepts now, and then switch to "hacking mode" (write
+ code) later, instead of now writing code but not understanding the
+ concepts behind it.
+ <hacklu> I have wrote a bit code to validate my understanding when I read
+ the soruce code. But the code not run. http://pastebin.com/r3wC5hUp
+ <tschwinge> The subsystem directive [...]. As well, let me just point you
+ to the documentation:
+ <http://www.gnu.org/software/hurd/microkernel/mach/mig/documentation.html>,
+ MIG - THE MACH INTERFACE GENERATOR, chapter 2.1 Subsystem identification.
+ <tschwinge> hacklu: Yes, writing such code for testing also is a good
+ approach. I will have to look at that in more detail, too.
+ * tschwinge guesses hacklu is probably laughing when seeing the years these
+ documents were written in (1989, etc.). ;-)
+ <hacklu> mach_msg make no sense in my code, and the process just hang. kill
+ -9 can't stop is either.
+ <braunr> hacklu: do you understand why kill -KILL might not work now ?
+ <hacklu> braunr: no, but I found I can use gdb to attach to that process,
+ then quit in gdb, the process quit too.
+ <hacklu> maybe that process was waiting a resume.
+ <braunr> something like that yes
+ <braunr> iirc it's related to a small design choice in the proc server
+ <braunr> something that put processes in an uninterruptible state when
+ being debugged
+ <hacklu> iirc ?
+ <braunr> if i recall cl=orrectly
+ <braunr> correctly*
+ <hacklu> like D status in linux?
+ <braunr> or T
+ <braunr> there has been a lot of improvements regarding signal handling in
+ linux over time so it's not really comparable now
+ <braunr> but that's the idea
+ <hacklu> in ps, i see the process STAT is THumx
+ <braunr> did you see that every process on the hurd has at least two
+ threads ?
+ <hacklu> no, but I have see that in hurd, the exception handler can't live
+ in the same context with the victim. so there must be at least two
+ threads. I think
+ <braunr> hacklu: yes
+ <braunr> that thread also handles regular signals
+ <braunr> in addition to mach exceptions
+ <braunr> (there are two levels of multiplexing in servers, first locating
+ the subsystem, then the server function)
+ <braunr> hacklu: if what i wrote is confusing, don't hesitate to ask for
+ clarifications (i really don't intend to make things confusing)
+ <hacklu> braunr: I don't know what you say about the "multiplexing in
+ servers". For instance, is it means how to pass message from mach_msg to
+ exc_server in gnu_wait()?
+ <braunr> hacklu: i said that the "message thread" handles both mach
+ exceptions and unix signals
+ <braunr> hacklu: these are two different interfaces (and in mach terms,
+ subsystems)
+ <braunr> hacklu: see hurd/msg.defs for the msg interface (which handles
+ unix signals)
+ <braunr> hacklu: to handle multiple interfaces in the same thread, servers
+ need to first find the right subsystem
+ <braunr> this is done by subsequently calling all demux functions until one
+ returns true
+ <braunr> (finding the right server function is done by these demux
+ functions)
+ <braunr> hacklu: see hurd/msgportdemux.c in glibc to see how it's done
+ there
+ <braunr> it's short actually, i'll past it here :
+ <braunr> return (_S_exc_server (inp, outp) ||
+ <braunr> _S_msg_server (inp, outp));
+ <braunr> hacklu: did that help ?
+ <hacklu> braunr: a bit more confusing. one "message thread" handles
+ exceptions and signals, means the message thread need to recive message
+ from two port. then pass the message to the right server which handle the
+ message. the server also should pick the right subsystem from a lot of
+ subsystems to handle the msg. is this ?
+ <braunr> the message thread is a server thread
+ <braunr> (which means every normal process is actually also a server,
+ receiving exceptions and signals)
+ <braunr> there may be only two ports, or more, it doesn't matter much, the
+ port set abstraction takes care of that
+ <hacklu> so the message thread directly pass the msg to the right
+ subsystem?
+ <braunr> not directly as you can see
+ <braunr> it tries them all until one is able to handle the incoming message
+ <braunr> i'm not sure it will help you with gdb, but it's important to
+ understand for a better general understanding of the system
+ <braunr> ugly sentence
+ <hacklu> ah, I see. like this in gnu-nat.c if(!notify_server(&msg.hdr,
+ &reply.hdr) && !exc_server(&msg.hdr...)
+ <braunr> yes
+ <hacklu> the thread just ask one by one.
+ <braunr> be careful about the wording
+ <braunr> the thread doesn't "send requests"
+ <braunr> it runs functions
+ <braunr> (one might be tempted to think there are other worker threads
+ waiting for a "main thread" to handle demultiplexing messages)
+ <hacklu> I got it.
+ <hacklu> the notify_server function is just run in the same context in
+ "message thread",and there is no RPC here.
+ <braunr> yes
+ <hacklu> and the notify_server code is generater by mig automatically.
+ <braunr> yes
+
+
+# IRC, freenode, #hurd, 2013-06-29
+
+[[!tag open_issue_documentation]]
+
+ <hacklu> I just failed to build the demo on
+ this. http://walfield.org/pub/people/neal/papers/hurd-misc/ipc-hello.c
+ <hacklu> or, example in machsys.doc called simp_ipc.c
+ <pinotree> we don't use cthreads anymore, but pthreads
+ <hacklu> pinotree: em.. and I also failed to find the <servers/env_mgr.h>
+ in example of <A programmer's guide to MACH system call>
+ <pinotree> that i don't know
+ <hacklu> maybe the code in that book out-of-date
+ <teythoon> hacklu: mig and mach ipc documentation is quite dated
+ unfortunately, and so are many examples floating around the net
+
+ <hacklu> btw, I have one more question. when I read <Mach 3 kernel
+ interface>. I find this state: When an exception occurs in a thread, the
+ thread sends an exception message to
+ <hacklu> its exception port, blocking in the kernel waiting for the receipt
+ of a reply. It is
+ <hacklu> assumed that some task is listening to this
+ <hacklu> port, using the exc_serverfunction to decode the messages and
+ then call the
+ <hacklu> linked in catch_exception_raise. It is the job of
+ catch_exception_raiseto handle the exception and decide the course of
+ action for thread.
+ <hacklu> that says, it assumed another task to recieve the msg send to one
+ thread's exception port. why another task?
+ <hacklu> I remmebered, there are at least two threads in one task, one is
+ handle the exception stuffs.
+ <braunr> there are various reasons
+ <braunr> first is, the thread causing the exception is usually not waiting
+ for a message
+ <braunr> next, it probably doesn't have all the info needed to resolve the
+ exception
+ <braunr> (depending on the system design)
+ <braunr> and yes, the second thread in every hurd process is the msg
+ thread, handling both mach exceptions and hurd signals
+ <hacklu> but in this state, I can't find any thing with the so called msg
+ thread
+ <braunr> ?
+ <hacklu> if exist a task to do the work, why we need this thread?
+ <braunr> this thread is the "task"
+ <hacklu> ?
+ <braunr> the msg thread is the thread handling exceptions for the other
+ threads in one task
+ <braunr> wording is important here
+ <braunr> a task is a collection of resources
+ <braunr> so i'm only talking about threads really
+ <braunr> 14:11 < hacklu> assumed that some task is listening to this
+ <braunr> this is wrong
+ <braunr> a task can't listen
+ <braunr> only a thread can
+ <hacklu> in you words, the two thread is in the same task?
+ <braunr> yes
+ <braunr> 14:32 < braunr> and yes, the second thread in every hurd process
+ is the msg thread, handling both mach exceptions and hurd signals
+ <braunr> process == task here
+ <hacklu> yeah, I always think the two thread stay in one task. but I found
+ that state in <mach 3 kernel interface>. so I confuzed
+ <hacklu> s/confuzed/confused
+ <braunr> statement you mean
+ <hacklu> if two thread stay in the same task. and the main thread throw a
+ exception, the other thread to handle it?
+ <braunr> depends on how it's configured
+ <braunr> the thread receiving the exceptions might not be in the same task
+ at all
+ <braunr> on the hurd, only the second thread of a task receives exception
+ <braunr> s
+ <hacklu> I just wonder how can the second thread catch the exception from
+ its containning task
+ <braunr> forget about tasks
+ <braunr> tasks are resource containers
+ <braunr> they don't generate or catch exceptions
+ <braunr> only threads do
+ <braunr> for each thread, there is an exception port
+ <braunr> that is, one receive right, and potentially many send rights
+ <braunr> the kernel uses a send right to send exceptions
+ <braunr> the msg thread waits for messages on the receive right
+ <braunr> that's all
+ <hacklu> ok. if I divide zero in main thread, the kernel will send a msg to
+ the main thread's exception port. and then, the second thread(in the same
+ task) is waiting on that port. so he get the msg. is it right?
+ <braunr> don't focus on main versus msg thread
+ <braunr> it applies to all other threads
+ <braunr> as well
+ <braunr> otherwise, you're right
+ <hacklu> ok, just s/main/first
+ <braunr> no
+ <braunr> main *and* all others except msg
+ <hacklu> main *and* all others except msg ?
+ <braunr> the msg thread gets exception messages for all other threads in
+ its task
+ <braunr> (at least, that's how the hurd configures things)
+ <hacklu> got it.
+ <hacklu> if the msg thread throw exception either, who server for himself?
+ <braunr> i'm not sure but i guess it's simply forbidden
+ <hacklu> i used gdb to attach a little progrom which just contains a divide
+ zero. and I only found the msg thread is in the glibc.
+ <braunr> yes
+ <hacklu> where is the msg thread located in.
+ <braunr> it's created by glibc
+ <hacklu> is it glibc/hurd/catch-exc.c?
+ <braunr> that's the exception handling code, yes
+ <hacklu> there are some differences between the code and the state in <mach
+ 3 system interface>.
+ <braunr> state or statement ?
+ <hacklu> staement
+ <braunr> which one ?
+ <hacklu> http://pastebin.com/ZTBrUAsV
+ When an exception occurs in a thread, the thread sends an exception
+ message to
+ its exception port, blocking in the kernel waiting for the receipt of a
+ reply. It is
+ assumed that some task is listening (most likely with mach_msg_server)
+ to this
+ port, using the exc_serverfunction to decode the messages and then
+ call the
+ linked in catch_exception_raise. It is the job of
+ catch_exception_raiseto handle the exception and decide the course of
+ action for thread. The state of the
+ blocked thread can be examined with thread_get_state.
+ <braunr> what difference ?
+ <hacklu> in the code, I can't find things like exc_server,mach_msg_server
+ <braunr> uh
+ <braunr> ok it's a little tangled
+ <braunr> but not that much
+ <braunr> you found the exception handling code, and now you're looking for
+ what calls it
+ <braunr> simple
+ <braunr> see _hurdsig_fault_init
+ <hacklu> from that statemnet I thought there are another _task_ do the
+ exception things for all of the systems thread before you have told me
+ the task means the msg thread.
+ <braunr> again
+ <braunr> 14:47 < braunr> forget about tasks
+ <braunr> 14:47 < braunr> tasks are resource containers
+ <braunr> 14:47 < braunr> they don't generate or catch exceptions
+ <braunr> 14:47 < braunr> only threads do
+ <hacklu> yeah, I think that document need update.
+ <braunr> no
+ <braunr> it's a common misnomer
+ <braunr> once you're used to mach concepts, the statement is obvious
+ <hacklu> braunr: so I need read more :)
+ <hacklu> _hurdsig_fault_init send exceptions for the signal thread to the
+ proc server?
+ <hacklu> why come about _proc_ server?
+ <braunr> no it gives the proc server a send right for signals
+ <braunr> exceptions are a mach thing, signals are a hurd thing
+ <braunr> the important part is
+ <braunr> err = __thread_set_special_port (_hurd_msgport_thread,
+ <braunr> THREAD_EXCEPTION_PORT, sigexc);
+ <hacklu> this one set the exception port?
+ <braunr> yes
+ <braunr> hm wait
+ <braunr> actually no, wrong part :)
+ <braunr> this sets the excpetion port for the msg thread (which i will call
+ the signal thread as mentioned in glibc)
+ <hacklu> but the comment above this line, Direct signal thread exceptions
+ to the proc server means what?
+ <braunr> that the proc server handles exceptions on the signal thread
+ <hacklu> the term signal thread equals the term msg thread?
+ <braunr> yes
+ <hacklu> so, the proc server handles the exceptions throwed by the msg
+ thread?
+ <braunr> looks that way
+ <hacklu> feels a little strange.
+ <braunr> why ?
+ <braunr> this thread isn't supposed to cause exceptions
+ <braunr> if it does, something is deeply wrong, and something must clean
+ that task up
+ <braunr> and the proc server seems to be the most appropriate place from
+ where to do it
+ <hacklu> why need a special server to just work the msg thread? I don't
+ think that thread will throw exception frequentlly
+ <braunr> what does frequency have to do with anything here ?
+ <braunr> ok the appropriate code is _hurdsig_init
+ <braunr> the port for receiving exceptions is _hurd_msgport
+ <braunr> the body of the signal thread is _hurd_msgport_receive
+ <hacklu> aha, in the _hurd_msgport_receive I have finally found the
+ while(1) loop mach_msg_server().
+ <hacklu> so the code is conform with the documents.
+ <hacklu> braunr: [21:18] <braunr> what does frequency have to do with
+ anything here ? yes, I have totally understood your words now. thank you
+ very much.
+ <braunr> :)
+
+
+# IRC, freenode, #hurd, 2013-07-01
+
+ <hacklu> hi. this is my weekly
+ report. http://hacklu.com/blog/gsoc-weekly-report2-124/ welcome to any
+ comment
+ <hacklu> teythoon: I only get clear about the rpc stuff. seems a lot behind
+ my plan
+ <youpi> good progress :)
+ <hacklu> I have wrote the details of the exception handle which was asked
+ by tschwing_ last week. Am I all right in my post?
+ <youpi> hacklu: as far as I understand signals, yes :)
+ <hacklu> youpi: thanks for god, I am on the right way finally... :)
+ <hacklu> the mig book says simpleroutine is the one use to implement asyn
+ RPCs which doesn't expect an reply. But I have found a place to pass an
+ reply port to the RPC interface which has been declared as simpleroutine
+ <youpi> hacklu: probably the simpleroutine hardcodes a reply port?
+
+ <youpi> hacklu: about _hurd_internal_post_signal, this is the hairiest part
+ of GNU/Hurd, signal handling
+ <youpi> simply because it's the hairiest part of POSIX :)
+ <youpi> you probably want to just understand that it implements the
+ POSIXity of signal delivering
+ <youpi> i.e. deliver/kill/suspend the process as appropriate
+ <youpi> I don't think you'll need to dive more
+ <hacklu> aha.
+ <hacklu> it will save a lot of time.
+ <hacklu> it seems like the wait_for_inferior() in gdb. which also has too
+ many lines and too many goto
+ <youpi> hacklu: btw, which simpleroutine were you talking about ?
+ <hacklu> I forget where it is, I am finding it now.
+ <youpi> which version of gdb are you looking the source of?
+ <youpi> (in mine, wait_for_inferior is only 45 lines long)
+ <hacklu> I dont know how to pick the verison, I just use the git
+ version. maybe I give a wrong name.
+ <youpi> ok
+ <hacklu> youpi:I remembered, my experience comes from here
+ http://www.aosabook.org/en/gdb.html. (All of this activity is managed by
+ wait_for_inferior. Originally this was a simple loop, waiting for the
+ target to stop and then deciding what to do about it, but as ports to
+ various systems needed special handling, it grew to a thousand lines,
+ with goto statements criss-crossing it for poorly understood
+ <hacklu> reasons.)
+ <hacklu> youpi: the simpleroutine is gdb/gdb/exc_request.defs
+ <youpi> so there is indeed an explicit reply port
+ <hacklu> but simpleroutine is for no-reply use. why use reply port here?
+ <youpi> AIUI, it's simply a way to make the request asynchronous, but still
+ permit an answer
+ <hacklu> ok, I will read the mig book carefully.
+ <braunr> hacklu: as youpi says
+ <braunr> a routine can be broken into two simpleroutines
+ <braunr> that's why some interfaces have interface.defs,
+ interface_request.defs and interface_reply.defs files
+ <braunr> nlightnfotis: in mach terminology, a right *is* a capability
+ <braunr> the only thing mach doesn't easily provide is a way to revoke them
+ individually
+ <nlightnfotis> braunr: Right. And ports are associated with the process
+ server and the kernel right? I mean, from what I have understood, if a
+ process wants to send a signal to another one, it has to do so via the
+ ports to that process held by the process server
+ <nlightnfotis> and it has to establish its identity before doing so, so
+ that it can be checked if it has the right to send to that port.
+ <braunr> yes
+ <nlightnfotis> do process own any ports? or are all their ports associated
+ with the process server?
+ <nlightnfotis> *processes
+ <braunr> mach ports were intended for a lot of different uses
+ <braunr> but in the hurd, they mostly act as object references
+ <braunr> the process owning the receive right (one at most per port)
+ implements the object
+ <braunr> processes owning send rights invoke methods on the object
+ <braunr> use portinfo to find out about the rights in a task
+ <braunr> (process is the unix terminology, task is the mach terminologyà
+ <braunr> )
+ <braunr> i use them almost interchangeably
+ <nlightnfotis> ahh yes, I remember about the last bit. And mach tasks have
+ a 1 to 1 association with user level processes (the ones associated with
+ the process server)
+ <braunr> the proc server is a bit special because it has to know about all
+ processes
+ <braunr> yes
+
+In context of [[open_issues/libpthread/t/fix_have_kernel_resources]]:
+
+ <braunr> hacklu: if you ever find out about either glibc or the proc server
+ creating one receive right for each thread, please let me know
+
+
+# IRC, freenode, #hurd, 2013-07-07
+
+ <hacklu> how fork() goes?
+ <pinotree> see sysdeps/mach/hurd/fork.c in glibc' sources
+ <hacklu> when the father has two thread( main thread and the signal thead),
+ if the father call fork, then the child inmediatelly call exev() to
+ change the excute file. how many thread in the children?
+ <hacklu> For instance, the new execute file also have two thread.
+ <hacklu> will the exev() destroyed two threads and then create two new?
+ <hacklu> s/exev()/excv()
+ <hacklu> s/exev()/exec() :)
+
+ <hacklu> what libhurduser-2.13.so does?
+ <hacklu> where can I find this source?
+ <pinotree> contains all the client stubs for hurd-specific RPCs
+ <pinotree> it is generated and built automatically within the glibc build
+ process
+
+ <hacklu> and what is the "proc" server?
+ <pinotree> what handles in user spaces the processes
+ <hacklu> so if I call proc_wait_request(), I will go into the
+ S_proc_wait_reply?
+ <hacklu> thanks, I have found that.
+
+
+# IRC, freenode, #hurd, 2013-07-08
+
+ <hacklu> hi, this is my weekly
+ report. http://hacklu.com/blog/gsoc-weekly-report3-137/
+ <hacklu> this week I have met a lot of obstacles. And I am quite desired to
+ participate in this meeting.
+ <tschwinge> hacklu: So from your report, the short version is: you've been
+ able to figure out how the things work that you were looking at (good!),
+ and now there are some new open questions that you're working on now.
+ <tschwinge> hacklu: That sounds good. We can of course try to help with
+ your open questions, if you're stuck figuring them out on your own.
+ <hacklu> tschwinge: the most question is: what is the proc server? why need
+ to call proc_get_reqeust() before the mach_msg()?
+ <hacklu> and Is there exist any specific running sequence between father
+ and child task after fork()? And I found the inferior always call the
+ trace_me() in the same time(the trace me printf always in the same line
+ of the output log). which I have post in my report.
+ <tschwinge> hacklu: The fork man-page can provide a high-level answer to
+ your Q3: »The child process is created with a single thread—the one that
+ called fork(). The entire virtual address space of the parent is
+ replicated in the child, including the states of mutexes, condition
+ variables, and other pthreads objects [...]«
+ <tschwinge> hacklu: What happens in GNU Hurd is that the signal thread is
+ also "cloned" (additionally to the thread which called fork), but then it
+ (the signal thread) is re-started from the beginning. (So this is very
+ much equivalent to creating a new signal thread.)
+ <tschwinge> hacklu: Then, upon exec, a new memory image is created/loaded,
+ replacing the previous one. [glibc]/sysdeps/mach/hurd/execve.c. What
+ actually happens with the existing thread (in particular, the signal
+ thread) I don't know off-hand. Then answer is probably found in
+ [glibc]/hurd/hurdexec.c -- and perhaps some code of the exec server
+ ([hurd]/exec/).
+ <hacklu> I have checked the status of my regiter mail to FSF. it says it
+ had arrived in USA.
+ <tschwinge> hacklu: OK, good.
+ <tschwinge> hacklu: This is some basic information about the observer_*
+ functions is GDB:
+ http://sourceware.org/gdb/current/onlinedocs/gdbint/Algorithms.html#index-notifications-about-changes-in-internals-57
+ »3.10 Observing changes in gdb internals«.
+ <hacklu> tschwinge: not too clear. I will think this latter. and what is
+ the proc server?
+ <teythoon> hacklu: /hurd/proc, maps unix processes to mach threads afaiui
+ <hacklu> teythoon: question is, the mach_msg() will never return unless I
+ called proc_wait_request() first.
+ <teythoon> hacklu: sorry, I've no idea ;)
+ <hacklu> teythoon: :)
+ <tschwinge> hacklu: I will have to look into that myself, too; don't know
+ the answer off-hand.
+ <tschwinge> hacklu: In your blog you write proc_get_request -- but such a
+ functions doesn't seems to exist?
+ <hacklu> tschwinge: s/proc_get_request/proc_wait_request called in
+ gun_wait() [gnu-nat.c]
+ <tschwinge> hacklu: Perhaps the wait man-page's description of WUNTRACED
+ gives a clue: »also return if a child has stopped [...]«. But it also to
+ me is not yet clear, how this relates to the mach_mag call, and how the
+ proc server exactly is involved in it.
+ <tschwinge> I'm reading various source code files.
+ <tschwinge> At least, I don't undestand why it is required for an exception
+ to be forwarded.
+ <hacklu> if I need to read the proc server source code?
+ <tschwinge> I can see how it to become relevant for the case that GDB has
+ to be informed that the debugee has exited normally.
+ <tschwinge> hacklu: Yeah, probably you should spend some time with that, as
+ it will likely help to get a clearer picture of the situation, and is
+ relevant for other interactions in GDB, too.
+ <tschwinge> hacklu: By the way, if you find that pieces of the GDB source
+ code (especially the Hurd files of it) are insufficiently documented,
+ it's a very good idea, once you have figured out something, to add more
+ source code comments to the existing code. Or writed these down
+ separately, if that is easier.
+ <hacklu> which is the proc server? hurd/exec ?
+ <hacklu> that ok, I already comment things on my notes.
+ <tschwinge> hacklu: [Hurd]/proc/
+ <tschwinge> hacklu: And [Hurd]/hurd/process*.defs
+ <hacklu> got it
+ <tschwinge> hacklu: I'll have to experiment a bit with your HDebugger
+ example, but I'm out of time right now, sorry. Will continue later.
+ <hacklu> tschwinge: yep, the HDebugger has a problem, if you put the
+ sleep() after the printf in the just_print(), thing will hang.
+ <hacklu> tschwinge: and I am a little curious about how do you find my
+ code? I dont't remember I have mentioned that :)
+ <hacklu> tschwinge: I have post my gihub link in the last week report, I
+ found that.
+ <tschwinge> hacklu: That's how I found it, yes.
+ <hacklu> tschwinge: :)
diff --git a/community/gsoc/2013/nlightnfotis.mdwn b/community/gsoc/2013/nlightnfotis.mdwn
new file mode 100644
index 00000000..43f9b14c
--- /dev/null
+++ b/community/gsoc/2013/nlightnfotis.mdwn
@@ -0,0 +1,450 @@
+[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!toc]]
+
+
+# IRC, freenode, #hurd, 2013-06-29
+
+ <teythoon> so, how is your golang port going?
+ <nlightnfotis> I just started working on it. I had been reading
+ documentation so far. Maybe over reading as people told me when I asked
+ for their feedback
+ <nlightnfotis> but I will report on what I have done (technically tomorrow,
+ and post it in the mailing list too.
+
+ <nlightnfotis> Hey guys, what could possibly cause the following error
+ message when executing a program in the Hurd? "./dumper: Could not open
+ note: (system server) error with unknown subsystem"
+ <nlightnfotis> My program is one that opens a file and dumps it into stdout
+ <nlightnfotis> pinotree: the code I am using is the one present here
+ http://www.gnu.org/software/hurd/hacking-guide/hhg.html under paragraph
+ 6.1
+ <nlightnfotis> I investigated it a bit but can not find a lead. I seem to
+ have all the rights to open the file that I want to dump to stdout
+ <pinotree> what if you reset errno to 0 just after all the declarations in
+ main, before the instructions?
+ <nlightnfotis> will check this out and get back to you.
+ <pinotree> sure :)
+ <nlightnfotis> pinotree: Now it suggests that it can't get the number of
+ readable files, which the source suggests that is normal behavior.
+ Thanks for your assistance.
+
+
+# IRC, freenode, #hurd, 2013-07-01
+
+ <nlightnfotis> youpi: from my part I can report that I have started working
+ with the code, and doing as Thomas suggested. I was about to write my
+ report yesterday, but I am facing some build errors on the HURD, which I
+ would like to investigate further before I write my report.
+ <nlightnfotis> that's why I decided to write it later in the day.
+ <youpi> I don't think you have to wait
+ <youpi> you can simply write in your report that you are having build
+ errors
+ <nlightnfotis> ok. I will have it written and delivered later in the day.
+ <nlightnfotis> braunr: that's cool. I think my reading has paid for
+ itself. And you may be pleased to know that I have gotten my hands dirty
+ with the code. I was about to write report yesterday, but some build
+ errors with the gcc (that I am investigating atm) are holding me
+ off. Will have that written later in the day.
+ <braunr> don't hesitate to ask help about build errors
+ <braunr> don't wait too much
+ <braunr> you need to progress on what matters, and not be blocked by
+ secondary problems
+ <nlightnfotis> I will see myself asking for help rather sooner than later,
+ but I would like to investigate it myself, and attempt to solve the
+ issues that occur to me before resort to bugging you guys.
+ <braunr> sure
+ <braunr> just not too long
+ <braunr> too long being a day or so
+ <nlightnfotis> these were my build_results on the hurd
+ <nlightnfotis> they were linker errors
+ <nlightnfotis>
+ https://gist.github.com/NlightNFotis/5896188#file-build_results
+ <nlightnfotis> I am trying to build gcc on a linux 32 bit environment. It
+ also has some issues but not linker errors
+ <nlightnfotis> will resolve them to see if the linker errors are
+ reproducible on linux
+ <braunr> oh, lex stuff
+ <braunr> should be easy enough
+
+
+# IRC, freenode, #hurd, 2013-07-05
+
+ <nlightnfotis> I have not made much progress, but I see myself working with
+ it.
+ <nlightnfotis> I have managed to build gcc go on Linux
+ <nlightnfotis> but Hurd seems to have some issues
+ <nlightnfotis> it seems to randomly crash
+ <teythoon> the build process?
+ <nlightnfotis> not quite randomly it seems to be though
+ <nlightnfotis> yeah
+ <nlightnfotis> I have noticed that there is a pattern
+ <nlightnfotis> it does crash after some time
+ <teythoon> ^^
+ <nlightnfotis> but it doesn't crash at specific files
+ <braunr> define crash
+ <nlightnfotis> at some times it may crash during compiling insn-emit.c
+ <braunr> (hello guys)
+ <teythoon> hi braunr :)
+ <nlightnfotis> braunr: hey there! It does seem to keep on compiling this
+ file for a very long time (I have let it do so for 10, 20, 30 minutes)
+ but the result is the same
+ <nlightnfotis> and it does so for different files for different build
+ options
+ <braunr> ok so it doesn't crash
+ <braunr> it just doesn't complete
+ <braunr> is the virtual machine eating 100% cpu during that time ?
+ <nlightnfotis> I can still type at the terminal, but I can't send a term
+ signal
+ <nlightnfotis> I can report that QEMU does hold 100% of one core at that
+ time, (like it keeps processing) but there is no output on the terminal
+ <braunr> ok
+ <nlightnfotis> of course I can type at the terminal
+ <nlightnfotis> but nothing happens
+ <braunr> any idea of the size of the files involved ?
+ <nlightnfotis> I am checking it out right now
+ <nlightnfotis> before this goes any further, let me report on my
+ investigation
+ <braunr> i expect that to be our classic writeback thread storm issue
+ <nlightnfotis> initially, I thought it might be that it run out of memory
+ <nlightnfotis> even though I know that compilation is not memory intensive,
+ rather, cpu intensive
+ <nlightnfotis> anyway I increased the size of ram available to the vm
+ <nlightnfotis> from 1024 mb to 1536
+ <nlightnfotis> that didn't seem to have any effect. The "crash" still
+ happens at the same time, at the same files
+ <braunr> use freeze
+ <braunr> not crash
+ <braunr> crash is very misleading here
+ <nlightnfotis> freeze it is then.
+ <nlightnfotis> anyway
+ <nlightnfotis> then it striked me that it might be that the hard disk size
+ (3gb) might be too small (considering the gcc git repo is 1gb+)
+ <nlightnfotis> so I resized the qemu image to 8gb of hdd size
+ <nlightnfotis> the new size is acknowledged by the vm
+ <pinotree> for gcc in debug mode? might still not be enough
+ <nlightnfotis> but still it has no effect - it seems to follow its freezing
+ patterns
+ <pinotree> giving your work, i'd have not less than 15-20
+ <braunr> i'd use 32
+ <pinotree> *given
+ <braunr> but that's because i like power of twos
+ <nlightnfotis> pinotree: thanks for the advice. Right now I was gonna
+ increase the swap size
+ <nlightnfotis> according to vmstat in the hurd
+ <nlightnfotis> swap size is 173 mb
+ <nlightnfotis> don't know if it does have an impact
+ <braunr> it may but before rushing
+ <braunr> if you need swap, you're doomed anyway
+ <braunr> consider swap highly unreliable on the hurd
+ <braunr> please show the output of df -h on the file system you're using to
+ build
+ <braunr> ideally, i'd recommend using separate / and /home file systems
+ <braunr> it really improves reliability
+ <nlightnfotis> I don't think it swaps to be honest; however that's
+ something that my mentor thomas had suggested (increasing swap size) so I
+ am gonna try it at some time.
+ <pinotree> or have a separate file system in a subdi and work on it
+ <braunr> yes, /home or whatever suits you
+ <braunr> just not /
+ <nlightnfotis> braunr: pinotree: thanks both for your advice. Will do now,
+ and report on the results.
+ <braunr> that's not all
+ <braunr> 11:17 < braunr> please show the output of df -h on the file system
+ you're using to build
+ <nlightnfotis> braunr: I am on it. Oh and btw, everytime I am forced to
+ close the vm (due to the freezes) when I restart it ext2 reports that the
+ file system was not cleanly unmounted and does some repair to some
+ files. I am trying to find an explanation for that, but I can think of
+ many things
+ <braunr> well obviously
+ <pinotree> ext2 has no journaling
+ <braunr> the file system was not cleanly unmounted since you restarted it
+ with a cold reset
+ <nlightnfotis> braunr: df -h comes out with this: "df: cannot read table of
+ mounted file systems"
+ <pinotree> also, even if you manage to always shut down correctly, when
+ fsck runs because of the maximum mount count it'd find errors anyway (so
+ we have some bug)
+ <braunr> nlightnfotis: df -h /path/to/build/dir
+ <braunr> pinotree: not really bugs but it could be cleaned up
+ <nlightnfotis> filesystem: - Size 2.8G Used 2.8G Avail 0 Use% 100% Mounted
+ on /
+ <nlightnfotis> wow
+ <braunr> nlightnfotis: see
+ <nlightnfotis> that seems to explain many things
+ <teythoon> ^^
+ <nlightnfotis> thanks for that braunr!
+ <braunr> you resized the disk, but not the partition and the file system
+ <pinotree> braunr: well, if something in ext2 (or its libs) leaves issues
+ in the fs, i'd call that a bug :>
+ <nlightnfotis> yeah, that was utterly stupid of me
+ <braunr> pinotree: they're not issues
+ <braunr> nlightnfotis: be careful, mach needs a reboot every time you
+ change a partition table
+ <teythoon> nlightnfotis: important thing is that you found the issue :)
+ <braunr> then only, you can use resize2fs
+ <teythoon> braunr: weird, I thought mach nowadays can reload the partition
+ tables?
+ <teythoon> braunr: doesn't d-i need that?
+ <braunr> maybe a recent change i forgot
+ <braunr> or maybe fdisk still reports the error although it's fine
+ <braunr> in doubt, rebooting is still safe :p
+ <teythoon> or maybe youpi hacked it into d-is gnumach
+ <braunr> i doubt it would be there for the installer only :)
+ <braunr> if it's there, it's there
+ <braunr> i just don't know it
+ <nlightnfotis> braunr: teythoon: and everyone else that helped me. Thanks
+ you all guys. This was something that was driving me crazy. Will do all
+ that you suggested and report back on my status
+
+
+# IRC, freenode, #hurd, 2013-07-08
+
+ <nlightnfotis> tschwinge, I have managed to overcome most of the obstacles
+ I had initially faced with my project
+ <nlightnfotis> but I still had some build errors, that's why I have not
+ reported yet. Wanna try to see if I can resolve them today, and write my
+ report in the afternoon.
+ <tschwinge> nlightnfotis: So, from a quick look into the IRC backlog, it
+ was a "simple" out of disk space problem? %-) That happens.
+ <tschwinge> nlightnfotis: And yes, GCC needs a lot of disk space.
+ <tschwinge> nlightnfotis: What kind of build errors are you seeing now?
+ <nlightnfotis> tschwinge, yeah I felt stupid at the time, but it didn't
+ actually strike me that the file system didn't see the extra space. Also
+ it took me some time to figure out that in order to mount the new
+ partition, I only had to edit /etc/fstab
+ <nlightnfotis> always tried to mount it with the ext2 translator
+ <nlightnfotis> and the translator kept dying
+ <nlightnfotis> but it's all figured out now
+ <nlightnfotis> the latest build errors I am seeing are these
+ <teythoon> nlightnfotis: o_O you used fstab and it worked?
+ <nlightnfotis> yeah
+ <teythoon> nlightnfotis: that's unexpected from my perspective...
+ <nlightnfotis> I only had to add the new partition into fstab
+ <nlightnfotis> teythoon: I can pastebin my fstab if you wanna take a look
+ at it
+ <nlightnfotis> tschwinge: these were my latest build errors
+ https://www.dropbox.com/s/b0pssdnfa22ajbp/build_results
+ <teythoon> nlightnfotis: I'm pretty sure that mount -a isn't done on hurd
+ w/o pinos runsystem.sysv
+ <teythoon> weird
+ <nlightnfotis> tschwinge: I have also tried to build gcc with "make -w"
+ which from what I know supresses the errors that stopped compilation
+ <nlightnfotis> but the weird thing is that gcc nearly took forever to build
+ <teythoon> nlightnfotis: could you do a showtrans /your/mountpoint?
+ <nlightnfotis> teythoon: /hurd/ext2fs /dev/hd0s3
+ <teythoon> nlightnfotis: ok, so you've set a passive translator and an
+ active is started on demand
+ <nlightnfotis> it must be a passive translator
+ <teythoon> nlightnfotis: this is the hurd way of doing things, fstab is
+ unrelated
+ <nlightnfotis> it seems to persist during reboots
+ <teythoon> yes, exactly
+ <nlightnfotis> teythoon: my fstab if you wanna take a look
+ http://pastebin.com/ef94JPhG
+ <nlightnfotis> after I added /dev/hd0s3 to fstab along with its mountpoint,
+ and restarting the hurd, only then I did manage to use that partition
+ <nlightnfotis> before doing so I tried pretty much anything involving
+ mounting the partition and setting the ext2fs translator for it, but it
+ kept dying
+ <nlightnfotis> of course it was a ext2 filesystem
+ <youpi> err, perhaps adding to fstab simply triggered an fsck at reboot?
+ <teythoon> nlightnfotis: might have been that you needed to reboot mach so
+ that it picks up the new partition table
+ <teythoon> youpi: I thought this was fixed, the partition reloading I mean?
+ <youpi> that is needed, yes
+ <youpi> let me check
+ <nlightnfotis> youpi: it could be, though, to be honest, my hurd system
+ does an fsck all the time at boot
+ <teythoon> how do you manage to do that w/o rebooting for d-i?
+ <youpi> (I don't remember whether device busy is detected)
+ <youpi> teythoon: by making all translators go away, iirc
+ <teythoon> nlightnfotis: btw, you have ~/gcc_new as mountpoint in your
+ fstab, pretty sure that this cannot work, the path has to be absolute and
+ no ~ expansion is done
+ <nlightnfotis> tbh it does work, and it's weird
+ <teythoon> nlightnfotis: it works b/c of the passive translator you set,
+ not b/c of the fstab entry
+ <nlightnfotis> teythoon: should I change it?
+ <teythoon> probably, yes
+ <tschwinge> Well, that is probably not used anywhere.
+ <teythoon> tschwinge: not yet but soon ;)
+ <tschwinge> Isn't /etc/fstab only consulted for fsck.
+ <youpi> atm yes
+ <tschwinge> Anyway, it is definitely a very good idea to have a partition
+ separate from the rootfs for doing actual work.
+ <tschwinge> I think I described that in one of the first GSoC coodridation
+ emails. In the long one.
+ <nlightnfotis> teythoon: Oh it struck me now! Is it because tilde expansion
+ is only happening in bash, but /etc/fstab is read before bash is
+ initialized?
+ <tschwinge> nlightnfotis: Instead of fumbling around with partitioning of
+ disk images, it may be easier in your KVM/QEMU setup to simply add a new
+ disk using -hdb [file] (or similar).
+ <tschwinge> nlightnfotis: Basically, yes.
+ <youpi> nlightnfotis: fstab is not related with bash in any way
+ <nlightnfotis> anyway, it shouldn't matter now, it seems to be working, and
+ I wouldn't like fiddling around with it and messing it up now. I will
+ continue with resolving the gcc issues.
+ <tschwinge> But /etc/fstab has its very own "language" (layout), so tilde
+ expansion will never be done there.
+ <tschwinge> nlightnfotis: df -h ~/gcc_new/
+ <nlightnfotis> tschwinge: size 24G Used: 4.2G Avail 18G
+ <tschwinge> OK, that's fine.
+ <tschwinge> As you can see on
+ <http://darnassus.sceen.net/~hurd-web/open_issues/gcc/#index4h1>, GCC
+ will easily need some GiB.
+ <nlightnfotis> tschwinge: I have some questions about GCC: out of curiosity
+ how much time does it take to compile it on your machine? Because
+ yesterday I tried a -w (suppress warnings) build and it seemed to take
+ forever
+ <nlightnfotis> mind you the vm has 1536 ram available (I have read
+ somewhere that it can utilise such an amount) and the vm is KVM enabled
+ <youpi> without disabling g++, it can easily take hours
+ <tschwinge> nlightnfotis: The build error is unexpected, because I had
+ addressed that issue in a recent patch. :-)
+ <tschwinge> nlightnfotis: This is wrong: »checking whether setcontext
+ clobbers TLS variables... [...] yes«. Please check your sources, that
+ they correspond to the current version of the upstream
+ tschwinge/t/hurd/go branch.
+ <tschwinge> nlightnfotis: Quoting from that wiki page: »This takes up
+ around 3.5 GiB, and needs roughly 3.5 h on kepler.SCHWINGE and 15 h on
+ coulomb.SCHWINGE.« The latter is my Hurd machine.
+ <tschwinge> That's however with Java and Ada enabled, and a full
+ three-stages bootstrap.
+ <youpi> ah, right, there's java & ada too
+ <nlightnfotis> tschwinge: git branch (in the repo): master,
+ *tschwinge/t/hurd/go
+ <youpi> in debian they are built separately
+ <tschwinge> What I asked you to do is configure »--disable-bootstrap
+ --enable-languages=go«.
+ <tschwinge> So that should be a lot quicker.
+ <nlightnfotis> tschwinge: oh yes, everytime I have tried to compile gcc I
+ have done with these configurations
+ <tschwinge> But still a few hours perhaps.
+ <nlightnfotis> that's what I did yesterday too.
+ <tschwinge> OK, good. :-)
+ <tschwinge> A bootstrap build is a good way to check the just-built GCC for
+ sanity, but we expect that it is fine, as we concentrate on the GCC Go
+ port.
+ <nlightnfotis> the only "extra" configuration yesterday was my "-w" flag to
+ make, because those errors were actually triggered by -Werror
+ <tschwinge> Let me read up what make -w does. ;-)
+ <nlightnfotis> ah, yes, d/w I have read and understood what the bootstrap
+ build is. Seems like we don't need it atm
+ <nlightnfotis> afaik it suppresses all warnings
+ <pinotree> youpi: gcj no more
+ <nlightnfotis> the way gcc builds, it does convert (some) warnings to
+ errors
+ <tschwinge> Hmm. -w, --print-directory Print a message containing the
+ working directory before and after other processing.
+ <pinotree> youpi: doko folded gcj and gdc into gcc-4.8 to "workaround"
+ Built-Using
+ <tschwinge> nlightnfotis: Ah, that'S configure --enable-werror or something
+ like that.
+ <youpi> pinotree: right
+ <nlightnfotis> yep, and -w suppresses it
+ <nlightnfotis> (from what I have understood)
+ <tschwinge> nlightnfotis: Are you thinking about make -k?
+ <tschwinge> Yeah, I guess.
+ <nlightnfotis> let me see what -k does
+ <pinotree> youpi: (just to make builds even more lightweight, eh</irony>)
+ <nlightnfotis> yeah, -k should do too, I shall try it
+ <tschwinge> But: if gcc -Werror fails, even with make -k, the build will
+ not be able to come to a successful end, because that one complation
+ artefact that failed will be missing.
+ <nlightnfotis> so I shall try again with -w (supressed warnings)
+ <tschwinge> Configureing with --disable-werror (or similar) will "help" if
+ -Werror is the default, and the build fails due to that.
+ <nlightnfotis> from what I have understood these "errors" are not something
+ critical: it's only that function prototypes for these functions are
+ missing
+ <nlightnfotis> I have seen the code there, and even "default" gcc generated
+ prototypes (from the first usage of the function) should do, so I can't
+ understand why it might be a serious problem if I tell gcc to skip that
+ point
+ <tschwinge> nlightnfotis: Ah, now I see. You don't mean make -w, but
+ rather gcc -w: »-w Inhibit all warning messages.«
+ <tschwinge> But really, there shouldn't be such warnings/errors that make
+ the build fail.
+ <nlightnfotis> yeah
+ <tschwinge> nlightnfotis: In your GCC sources directory, what does this
+ tell: git rev-parse HEAD
+ <tschwinge> And, is the checkout clean: git status
+ <tschwinge> The latter will take some time.
+ <nlightnfotis> git status takes an awful amount of time
+ <nlightnfotis> last I checked
+ <nlightnfotis> but git rev-parse HEAD
+ <nlightnfotis> produces this result:
+ <nlightnfotis> 91840dfb3942a8d241cc4f0e573e5a9956011532
+ <tschwinge> OK, that's correct. So probably some of the checked out files
+ are not in a pristine state?
+ <nlightnfotis> I shall run a git clean and see. If that doesn't work too,
+ maybe I shall reclone the repository?
+ <nlightnfotis> there's nothing foreign to the repo that I have added, only
+ lib gmp, lib mpc and lib mpfr (and they are in their own folders inside
+ my gcc working directory)
+ <tschwinge> nlightnfotis: You shouldn't need to do the latter if you
+ instead run: apt-get build-dep gcc-4.8
+ <nlightnfotis> I remember having done that inside the Hurd, but it always
+ resulted in an error from what I can recall
+ <nlightnfotis> let me check this out
+ <nlightnfotis> yes
+ <tschwinge> nlightnfotis: Whenever you use Git on Hurd, pass the --quiet
+ flag, to avoid the rare but possible corruption issue described on
+ <http://darnassus.sceen.net/~hurd-web/open_issues/git_duplicated_content/>
+ and <http://darnassus.sceen.net/~hurd-web/open_issues/git-core-2/>.
+ <nlightnfotis> tschwinge: Forgive me for that. I will set up an alias
+ immediately.
+ <tschwinge> nlightnfotis: I don't know if an alias is possible, because --
+ I think -- you'll need to do things like: git fetch --quiet
+ <tschwinge> So pass --quiet to subcommands.
+ <nlightnfotis> oh. ok.
+ <tschwinge> nlightnfotis: What you can also do, is shut down your Hurd VM,
+ and mount the disk image on GNU/Linux (mount with offset to get the right
+ partition), and then run a diff -ru against a Git clone done on
+ GNU/Linux, and see whether there are any unexpected differences outside
+ of the .git/ directory.
+ <nlightnfotis> sounds like a plan. I will check this out today then :)
+ <nlightnfotis> tschwinge: if all else fails, then recloning the repo with
+ --quiet passed should work, right?
+ <tschwinge> Yes, that's probably the most straight-forward check to do.
+ <tschwinge> Heh, yes to both these questions. :-)
+ <tschwinge> nlightnfotis: Oh, you don't even have to re-clone, but rather
+ re-check-out the branch.
+ <nlightnfotis> I was thinking of recloning just to bring the whole
+ repository to a pristine state
+ <tschwinge> So something like (inside the source directory): rm -rf ./*
+ (remove any files, but leave .* in place, in particular the .git/
+ directory), followd by git checkout -f HEAD --quiet
+ <tschwinge> nlightnfotis: But before doing that, please do the diff first,
+ so that we know (hopefully) where the erroneous build results were coming
+ from.
+ <nlightnfotis> considering the Copyright assignment files, I have sent them
+ from day 1 (that is the 20th of June). I have not heard anything about
+ those documents to date (sadly)
+ <nlightnfotis> what's worst is that although I have a reference number to
+ track those documents, their (greek postal office) tracking service sucks
+ so badly, that one day it's offline, the next it suggests it can't find
+ the object in their database, the next it says it is still in the local
+ post office
+ <nlightnfotis> let me check it out now
+ <nlightnfotis> still nothing from their online service
+ <nlightnfotis> let me call them
+ <nlightnfotis> tschwinge: I called the post office regarding the copyright
+ papers. They told me that the same day (the 20th of June) it left from
+ Herakleion, Crete to Athens and the same day it must have left the
+ country heading towards the US. They also told me it takes about 1 week
+ for it to arrive.
+ <tschwinge> nlightnfotis: OK, so probably waiting at the FSF office to be
+ processed. Let's allow for some more time. After all, this is not
+ critical for your progress.
diff --git a/community/gsoc/project_ideas/mtab.mdwn b/community/gsoc/project_ideas/mtab.mdwn
index d6f04385..0c0bb87b 100644
--- a/community/gsoc/project_ideas/mtab.mdwn
+++ b/community/gsoc/project_ideas/mtab.mdwn
@@ -11,6 +11,8 @@ License|/fdl]]."]]"""]]
[[!meta title="mtab"]]
+[[!tag open_issue_hurd]]
+
In traditional monolithic system, the kernel keeps track of all mounts; the
information is available through `/proc/mounts` (on Linux at least), and in a
very similar form in `/etc/mtab`.
@@ -73,99 +75,3 @@ Possible mentors: Olaf Buddenhagen (antrik), Carl Fredrik Hammar (cfhammar)
Exercise: Make some improvement to any of the existing Hurd translators.
Especially those in [hurdextras](http://www.nongnu.org/hurdextras/) are often
quite rudimentary, and it shouldn't be hard to find something to improve.
-
-
-# Related Discussion
-
-## IRC, freenode, #hurd, 2013-04-17
-
- <kuldeepdhaka> thinking how to get the listing. traversing would be
- ineffecient, trying to come up with something better
- <braunr> what listing ?
- <braunr> and traversing what ?
- <kuldeepdhaka> mtab
- <braunr> well i assumed so
- <braunr> be more precise please
- <kuldeepdhaka> when the translator is done initalized <translation
- info> are written to /etc/mtab <translation info> will be provided
- by the translator, and when some one want to read the info just read it
- this way if their is some credentials like ftp sites pass username can be
- masked by the translator
- <kuldeepdhaka> if some trans dont want to list them, no need to write to
- file | while unmounting (sorry i couldnt find the right word) , it
- will pass the mount node address | <translation info> will have special
- structure to remove/add mounts example "a /mount-to /mount-from" = add
- , "r /mount-to" = remove here "/mount-to" will be unique for every
- mount
- <kuldeepdhaka> this have a draw back , we would have to trust trans for the
- listed data | also "/mount-to" + "/mount-from" could be used a
- combination for making sure that other trans unable remove others trans
- mount data
- <kuldeepdhaka> sorry but "also "/mount-to" + "/mount-from" could be used a
- combination for making sure that other trans unable remove others trans
- mount data" this is a bad idea if we had to print the whole thing
- <kuldeepdhaka> braunr, whats ur opinion?
- <pinotree> you don't need a mtab to "unmount" things on hurd
- <braunr> kuldeepdhaka: hum, have you read the project idea ?
- <braunr>
- http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/mtab/
- <braunr> A more promising approach is to have mtab exported by a special
- translator, which gathers the necessary information on demand. This could
- work by traversing the tree of translators, asking each one for mount
- points attached to it.
- <kuldeepdhaka> pinotree, not to unmount, i mean is to remove the
- <translation data>
- <braunr> for a first implementation, i'd suggest a recursive traversal of
- root-owned translators
- <kuldeepdhaka> braunr, hum, but it did stated it as inefficient
- <braunr> where ?
- <kuldeepdhaka> para 5 , line 3
- <kuldeepdhaka> and line 6
- <braunr> no
- <braunr> traversing "all" nodes would be inefficient
- <braunr> translators which host the nodes of other translators could
- maintain a simple list of active translators
- <braunr> ext2fs, e.g. (if that's not already the case) could keep the list
- of the translators it started
- <braunr> we can already see that list with pstree for example
- <braunr> but this new list would only retain those relevant for mtab
- <braunr> i.e. root-owned ones
- <pinotree> i would not limit to those though
- <braunr> and then filter on their type (e.g. file system ones)
- <braunr> pinotree: why ?
- <pinotree> this way you could have proper per-user /proc/$pid/mounts info
- <braunr> we could also very easily have a denial of service
- <kuldeepdhaka> but how will the mount point and source point will be
- listed?
- <braunr> they're returned by the translator
- <kuldeepdhaka> k
- <braunr> you ask /, it returns its store and its options, and asks its
- children recursively
- <braunr> a /home translator would return its store and its options
- <braunr> etc..
- <braunr> each translator would build the complete path before returning it
- <braunr> sort of, it's very basic
- <braunr> but that would be a very hurdish way to do it
- <kuldeepdhaka> shall /etc/mtab should be made seek-able and what should be
- the filesize? content are generated on demand so, it could arise problem
- (fsize:0 , seek-able:no), ur opinions?
- <braunr> kuldeepdhaka: it should have all the properties of a regular file
- <braunr> the filesize would be determined after it's generated
- <braunr> being empty doesn't imply it's not seekable
- <kuldeepdhaka> content is generated on demand so, could cause problem while
- seeking and filesize, shall i still program as regular file?
- <kuldeepdhaka> in two different read, it could generate different content,
- though same seek pos is used...
- <braunr> what ?
- <braunr> the content is generated on open
- <kuldeepdhaka> ooh, ok
-
-
-## IRC, freenode, #hurd, 2013-06-04
-
- <safinaskar> how to see list of all connected translators?
- <braunr> you can't directly
- <braunr> you can use ps to list processes and guess which are translators
- <braunr> (e.g. everything starting with /hurd/)
- <braunr> a recursive call to obtain such a list would be useful
- <braunr> similar to what's needed to implement /proc/mounts
diff --git a/community/gsoc/project_ideas/mtab/discussion.mdwn b/community/gsoc/project_ideas/mtab/discussion.mdwn
new file mode 100644
index 00000000..0e322c11
--- /dev/null
+++ b/community/gsoc/project_ideas/mtab/discussion.mdwn
@@ -0,0 +1,907 @@
+[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_hurd]]
+
+# IRC, freenode, #hurd, 2013-04-17
+
+ <kuldeepdhaka> thinking how to get the listing. traversing would be
+ ineffecient, trying to come up with something better
+ <braunr> what listing ?
+ <braunr> and traversing what ?
+ <kuldeepdhaka> mtab
+ <braunr> well i assumed so
+ <braunr> be more precise please
+ <kuldeepdhaka> when the translator is done initalized <translation
+ info> are written to /etc/mtab <translation info> will be provided
+ by the translator, and when some one want to read the info just read it
+ this way if their is some credentials like ftp sites pass username can be
+ masked by the translator
+ <kuldeepdhaka> if some trans dont want to list them, no need to write to
+ file | while unmounting (sorry i couldnt find the right word) , it
+ will pass the mount node address | <translation info> will have special
+ structure to remove/add mounts example "a /mount-to /mount-from" = add
+ , "r /mount-to" = remove here "/mount-to" will be unique for every
+ mount
+ <kuldeepdhaka> this have a draw back , we would have to trust trans for the
+ listed data | also "/mount-to" + "/mount-from" could be used a
+ combination for making sure that other trans unable remove others trans
+ mount data
+ <kuldeepdhaka> sorry but "also "/mount-to" + "/mount-from" could be used a
+ combination for making sure that other trans unable remove others trans
+ mount data" this is a bad idea if we had to print the whole thing
+ <kuldeepdhaka> braunr, whats ur opinion?
+ <pinotree> you don't need a mtab to "unmount" things on hurd
+ <braunr> kuldeepdhaka: hum, have you read the project idea ?
+ <braunr>
+ http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/mtab/
+ <braunr> A more promising approach is to have mtab exported by a special
+ translator, which gathers the necessary information on demand. This could
+ work by traversing the tree of translators, asking each one for mount
+ points attached to it.
+ <kuldeepdhaka> pinotree, not to unmount, i mean is to remove the
+ <translation data>
+ <braunr> for a first implementation, i'd suggest a recursive traversal of
+ root-owned translators
+ <kuldeepdhaka> braunr, hum, but it did stated it as inefficient
+ <braunr> where ?
+ <kuldeepdhaka> para 5 , line 3
+ <kuldeepdhaka> and line 6
+ <braunr> no
+ <braunr> traversing "all" nodes would be inefficient
+ <braunr> translators which host the nodes of other translators could
+ maintain a simple list of active translators
+ <braunr> ext2fs, e.g. (if that's not already the case) could keep the list
+ of the translators it started
+ <braunr> we can already see that list with pstree for example
+ <braunr> but this new list would only retain those relevant for mtab
+ <braunr> i.e. root-owned ones
+ <pinotree> i would not limit to those though
+ <braunr> and then filter on their type (e.g. file system ones)
+ <braunr> pinotree: why ?
+ <pinotree> this way you could have proper per-user /proc/$pid/mounts info
+ <braunr> we could also very easily have a denial of service
+ <kuldeepdhaka> but how will the mount point and source point will be
+ listed?
+ <braunr> they're returned by the translator
+ <kuldeepdhaka> k
+ <braunr> you ask /, it returns its store and its options, and asks its
+ children recursively
+ <braunr> a /home translator would return its store and its options
+ <braunr> etc..
+ <braunr> each translator would build the complete path before returning it
+ <braunr> sort of, it's very basic
+ <braunr> but that would be a very hurdish way to do it
+ <kuldeepdhaka> shall /etc/mtab should be made seek-able and what should be
+ the filesize? content are generated on demand so, it could arise problem
+ (fsize:0 , seek-able:no), ur opinions?
+ <braunr> kuldeepdhaka: it should have all the properties of a regular file
+ <braunr> the filesize would be determined after it's generated
+ <braunr> being empty doesn't imply it's not seekable
+ <kuldeepdhaka> content is generated on demand so, could cause problem while
+ seeking and filesize, shall i still program as regular file?
+ <kuldeepdhaka> in two different read, it could generate different content,
+ though same seek pos is used...
+ <braunr> what ?
+ <braunr> the content is generated on open
+ <kuldeepdhaka> ooh, ok
+
+
+# IRC, freenode, #hurd, 2013-06-04
+
+ <safinaskar> how to see list of all connected translators?
+ <braunr> you can't directly
+ <braunr> you can use ps to list processes and guess which are translators
+ <braunr> (e.g. everything starting with /hurd/)
+ <braunr> a recursive call to obtain such a list would be useful
+ <braunr> similar to what's needed to implement /proc/mounts
+
+
+# IRC, freenode, #hurd, 2013-06-25
+
+In context of [[microkernel/mach/mig/documentation/structured_data]].
+
+ <teythoon> should I go for an iterator like interface instead?
+ <teythoon> btw, what's the expected roundtrip time?
+ <braunr> don't think that way
+ <braunr> consider the round trip delay as varying
+ <teythoon> y, is it that bad?
+ <braunr> no
+ <braunr> but the less there is the better
+ <braunr> we think the same with system calls even if they're faster
+ <braunr> the delay itself isn't the real issue
+ <braunr> look at how proc provides information
+ <braunr> (in procfs for example)
+
+
+## IRC, freenode, #hurd, 2013-06-26
+
+ <teythoon> so tell me about the more hurdish way of dealing with that issue
+ <teythoon> creating a specialized translator for this?
+ <braunr> 11:45 < pinotree> there's also
+ http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/mtab/
+ about that topic
+ <braunr> you need to avoid thinking with centralization in mind
+ <braunr> the hurd is a distributed system in practice
+ <braunr> i think proc is the only centralized component in there
+ <teythoon> braunr: would having an mtab translator and having fs
+ translators register to thae be acceptable?
+ <teythoon> that*
+ <braunr> teythoon: why do you want to centralize it ?
+ <braunr> translators already register themselves when they get attached to
+ a node
+ <braunr> we don't want an additional registration
+ <braunr> have you read the link we gave you ?
+ <teythoon> I did and I got the message, but isn't the concept of
+ /proc/mounts or a mtab file already a centralized one?
+ <braunr> that doesn't mean the implementation has to be
+ <braunr> and no, i don't think it's centralized actually
+ <braunr> it's just a file
+ <braunr> you can build a file from many sources
+ <teythoon> or if we do it your way recursing on fs translators *but*
+ restricting this to root owned translators also suffering from
+ centralization wrt to the root user? I mean the concept of all mounted
+ filesystems does not apply cleanly to the hurd
+ <braunr> i don't understand
+ <braunr> restricting to the root user doesn't mean it's centralized
+ <braunr> trust has nothing to do with being centralized
+ <teythoon> I guess I'm not used to thinking this way
+ <braunr> teythoon: i guess that's the main reason why so few developers
+ work on the hurd
+ <teythoon> also the way fs notification is done is also centralized, that
+ could also be done recursively
+ <braunr> what doyou call fs notification ?
+ <teythoon> and the information I need could just be stuffed into the same
+ mechanism
+ <teythoon> fs translators being notified of system shutdown
+ <braunr> right
+ <braunr> that gets a bit complicated because the kernel is also a
+ centralized component
+ <braunr> it knows every memory object and their pagers
+ <braunr> it manages all virtual memory
+ <braunr> there are two different issues here
+ <braunr> syncing memory and shutting down file systems
+ <braunr> the latter could be done recursively, yes
+ <braunr> i wonder if the former could be delegated to external pagers as
+ well
+ <braunr> teythoon: but that's not the focus of your work aiui, it would
+ take much time
+ <teythoon> sure, but missing an mtab file or better yet /proc/mounts could
+ be an issue for me, at least a cosmetic one, if not a functional one
+ <braunr> i understand
+ <teythoon> and hacking up a quick solution for that seemed like a good
+ exercise
+ <braunr> i suggest you discuss it with your mentors
+ <braunr> they might agree to a temporary centralized solution
+ <braunr> although i don't think it's much simpler than the recursive one
+ <teythoon> braunr: would that be implemented in libdiskfs and friends?
+ <braunr> teythoon: i'm not sure, it might be a generic fs operation
+ <braunr> libnetfs etc.. are also mount points
+ <teythoon> so where would it go if it was generic?
+ <braunr> libfshelp perhaps
+ <teythoon> translator startup is handled in start-translator-long.c, so in
+ case a startup is successful, I'd add it to a list?
+ <braunr> i'd say so, yes
+ <teythoon> would that cover all cases, passive and active translators?
+ <braunr> that's another question
+ <braunr> do we consider passive translators as mounted ?
+ <teythoon> ah, that was not what i meant
+ <braunr> i know
+ <braunr> but it's related
+ <teythoon> start b/c of accessing a passive one vs. starting an active one
+ using settrans
+ <braunr> start_translator_xxx only spawn active translators
+ <braunr> it's the same
+ <teythoon> ok
+ <braunr> the definition of a passive translator is that it starts the
+ active translator on access
+ <teythoon> yeah I can see how that wouldn't be hard to implement
+ <braunr> i think we want to include passive translators in the mount table
+ <braunr> so registration must happen before starting the active one
+ <teythoon> so it's a) keeping a list of active translators and b) add an
+ interface to query fs translators for this list and c) an interface to
+ query mtab style information?
+ <braunr> keeping a list of all translators attached
+ <braunr> and yes
+ <braunr> well
+ <braunr> a is easy
+ <braunr> b is the real work
+ <braunr> c would be procfs using b
+ <teythoon> oh? I thought recursing on the translators and querying info
+ would be separate operations?
+ <braunr> why so ?
+ <braunr> the point is querying recursively :)
+ <braunr> and when i say recursively, it's only a logical view
+ <teythoon> ok, yes, it can be implemented this way, so we construct the
+ list while recursing on the translators
+ <braunr> i think it would be better to implement it the way looking up a
+ node is done
+ <teythoon> in a loop, using a stack?
+ <braunr> iteratively
+ <braunr> a translator would provide information about itself (if
+ supported), and referrences to translators locally registered to it
+ <teythoon> could you point me to the node lookup?
+ <teythoon> ah, yes
+ <braunr> eg., you ask /, it tells you it's on /dev/hd0, read-write, with
+ options, and send rights to /home, /proc, etc..
+ <braunr> well rights, references
+ <braunr> it could be the path itself
+ <teythoon> rights as in a port to the translators?
+ <braunr> i think the path would be better but i'm not sure
+ <braunr> it would also allow you to check the permissions of the node
+ before querying
+ <teythoon> path would be nicer in the presence of stacked translators
+ <braunr> and obviously you'd have the path right away, no need to provide
+ it in the reply
+ <teythoon> true
+
+ <teythoon> braunr: if we want to list passive translators (and I agree, we
+ should), it isn't sufficient to touch libfshelp, as setting a passive
+ translator is not handled there, only the startup
+ <braunr> teythoon: doesn't mean you can't add something there that other
+ libraries will use
+ <braunr> so yes, not sufficient
+
+
+## IRC, freenode, #hurd, 2013-06-29
+
+ <teythoon> braunr: diskfs_S_fsys_set_options uses diskfs_node_iterate to
+ recurse on active translators if do_children is given
+ <teythoon> braunr: I wonder how fast that is in practice
+ <teythoon> braunr: if it's fast enough, there might not even be a need for
+ a new function in fsys.defs
+ <teythoon> and no need to keep a list of translators for that reason
+ <teythoon> braunr: if it's not fast enough, then diskfs_S_fsys_set_options
+ could use the list to speed this up
+ <braunr> teythoon: on all nodes ?
+ <teythoon> braunr: i believe so, yes, see libdiskfs/fsys-options.c
+ <braunr> teythoon: well, if it really is all node, you clearly don't want
+ that
+
+
+## IRC, freenode, #hurd, 2013-07-01
+
+ <teythoon> I've ment to ask, the shiny new fsys_get_translators interface,
+ should it return the options for the queried translator or not?
+ <braunr> i don't think it should
+ <teythoon> ok
+ <braunr> let's walk through why it shouldn't
+ <teythoon> may I assume that the last argument returned by fsys_get_options
+ is the "source"?
+ <braunr> how would you know these options ?
+ <braunr> the source ?
+ <teythoon> I wouldn't actually
+ <braunr> yes, you wouldn't
+ <braunr> you'd have to ask the translators for that
+ <braunr> so the only thing you can do is point to them
+ <teythoon> well, the device to include in the mtab file
+ <braunr> and the client asks
+ <braunr> i don't know fsys_get_options tbh
+ <teythoon> well, both tmpfs and ext2fs print an appropriate value for
+ "device" as last argument
+ <braunr> looks like a bad interface to me
+ <braunr> options should be options
+ <braunr> there should be a specific call for the device
+ <braunr> but if everyone agrees with the options order, you can do it that
+ way for now i guess
+ <teythoon> one that could be used to recreate the "mount" using either
+ mount or settrans
+ <braunr> just comment it where appropriate
+ <teythoon> I thought that'd be the point?
+ <braunr> ?
+ <teythoon> % fsysopts tmp
+ <teythoon> /hurd/tmpfs --writable --no-inherit-dir-group --no-sync 48K
+ <braunr> where is the device ?
+ <teythoon> % settrans -ca tmp $(fsysopts tmp)
+ <braunr> 15:56 < teythoon> well, both tmpfs and ext2fs print an appropriate
+ value for "device" as last argument
+ <teythoon> 48K
+ <braunr> i don't see it
+ <braunr> really ?
+ <teythoon> yes
+ <braunr> what about ext2fs ?
+ <braunr> hm ok i see
+ <teythoon> % fsysopts /
+ <teythoon> ext2fs --writable --no-inherit-dir-group --sync=10
+ --store-type=typed device:hd0s1
+ <braunr> i don't think you should consider that as devices
+ <braunr> but really translator specific options
+ <pinotree> agree
+ <teythoon> I don't ;)
+ <teythoon> b/c the translator calling convention is hardcoded in the mount
+ utility
+ <braunr> ?
+ <teythoon> I think it's reasonable to assume that this mapping can be
+ reversed
+ <pinotree> theorically you can write a translator that takes no arguments,
+ but just options
+ <braunr> the 48K string for tmpfs is completely meaningless
+ <braunr> in fstab, it should be none
+ <pinotree> "tmpfs"
+ <braunr> the linux equivalent is the size option
+ <braunr> no, none
+ <braunr> it's totally ignored
+ <braunr> and it's recommended to set none rather than the type to avoid
+ confusion
+ <teythoon> u sure?
+ <teythoon> % settrans -cga tmp /hurd/tmpfs --mode=666 6M
+ <teythoon> % settrans -cga tmp /hurd/tmpfs --mode=666 6M
+ <teythoon> % fsysopts tmp
+ <teythoon> /hurd/tmpfs --writable --no-inherit-dir-group --no-sync 6M
+ <braunr> i've not explained myself clearly
+ <braunr> it's not ignored by the translator
+ <braunr> but in fstab, it should be in the options field
+ <braunr> it's not the source
+ <braunr> clearly not
+ <teythoon> ah
+ <braunr> now i'm talking about fstab, but iirc the format is similar in
+ mtab/mounts
+ <pinotree> close, but not the same
+ <braunr> yes, close
+ <teythoon> ok, so I'll put a method into libfshelp so that translators can
+ explicitly set a device and patch all existing translators to do so?
+ <braunr> teythoon: what i meant is that, for virtual vile systems (actually
+ file systems with no underlying devices), the device field is normally
+ ignored
+ <braunr> teythoon: why do you need that for exactly
+ <teythoon> right
+ <pinotree> do they even have a "device" field?
+ <braunr> (i can see why but i'd like more visibility)
+ <braunr> pinotree: not yet
+ <braunr> pinotree: that's what he wants to add
+ <braunr> but i'd like to see if there is another way to get the information
+ <braunr> 16:05 < braunr> teythoon: why do you need that for exactly
+ <teythoon> well if I'm constructing a mtab entry I need a value for the
+ device field
+ <braunr> do we actually need it to be valid ?
+ <teythoon> not necessarily I guess
+ <braunr> discuss it with your mentors then
+ <youpi> it has to be valid for e2fsck checks etc.
+ <braunr> doesn't e2fsck check fstab actually ?
+ <youpi> i.e. actually for the cases where it's trivial
+ <youpi> fstab doesn't tell it whether it's mounted
+ <youpi> I mean fsck checking whether it's mounted
+ <youpi> not fsck -a
+ <braunr> oh
+ <braunr> couldn't we ask the device instead ?
+ <braunr> looks twisted too
+ <youpi> that'd mean patching a lot of applications which do similar checks
+ <braunr> yes
+ <braunr> teythoon: propose an interface for that with your mentors then
+ <teythoon> yeah, but couldn't you lay it out a little, I mean would it be
+ one procedure or like three?
+ <braunr> 16:04 < teythoon> ok, so I'll put a method into libfshelp so that
+ translators can explicitly set a device and patch all existing
+ translators to do so?
+ <teythoon> ok
+ <braunr> why three ?
+ <teythoon> no, I mean when adding stuff to fsys.defs
+ <braunr> i understood that
+ <braunr> but why three ? :)
+ <teythoon> it'd be more generic
+ <braunr> really ?
+ <braunr> please show a quick example of what you have in mind
+ <teythoon> i honestly don't know, thus I'm asking ;)
+ <braunr> well first, this device thing bothers me
+ <braunr> when you look at how we set up our ext2fs translators, you can see
+ they use device:xxx
+ <braunr> and not /dev/xxx
+ <braunr> but ok, let's assume it's harmless
+ <teythoon> ok, but isn't the first way actually better?
+ <braunr> i think it ends up being the same
+ <braunr> ideally, that's what we want to use as device path
+ <teythoon> but you can recreate a storeio translator using the device:xxx
+ info, the node is useless for that
+ <braunr> so that we don't need to explicitely set it
+ <braunr> ?
+ <braunr> what do you mean ?
+ <teythoon> well, fsysopts / tells currently tells me device:hd0s1
+ <braunr> for /, there isn't much choice
+ <braunr> /dev isn't there yet
+ <teythoon> ah, got it
+ <teythoon> that's why it differs...
+ <braunr> differs ?
+ <braunr> from what ?
+ <braunr> other ext2fs translators are set the same way by the debian
+ installer for example
+ <teythoon> % fsysopts /media/scratch
+ <teythoon> /hurd/ext2fs --writable --no-inherit-dir-group /dev/hd1s1
+ <teythoon> here it uses the path to the node
+ <braunr> that's weird
+ <braunr> was that done by the debian installer ?
+ <teythoon> ah no, that was me
+ <braunr> :p
+ <braunr> $ fsysopts /home
+ <braunr> /hurd/ext2fs --writable --no-inherit-dir-group --store-type=device
+ hd0s6
+ <braunr> so as you can see, it's not that simple to infer the device path
+ <teythoon> oho, yet another way ;)
+ <teythoon> right then
+ <pinotree> isn't device:hd0s1 as shortcut for specifying the store type, as
+ done with --store-type=device hd0s1?
+ <braunr> but perhaps we don't need to
+ <braunr> yes it is
+ <pinotree> iirc it's something libstore does, per-store prefixes
+ <braunr> ah that sucks
+ <braunr> teythoon: you may need to normalize those strings
+ <braunr> so that they match what's in fstab
+ <braunr> i.e. unix /dev paths
+ <braunr> otherwise e2fsck still won't be able to find the translators
+ mounting the device
+ <braunr> well, if it's mounted actually
+ <braunr> it just needs to find the matching line in mtab aiui
+ <braunr> so perhaps a libfshelp function for that, yes
+ <teythoon> braunr: so you suggest adding a normalizing function to
+ libfshelp that creates a /dev/path?
+ <braunr> yes
+ <braunr> used by the call you intend to add, which returns that device
+ string as found in fstab
+ <teythoon> found in fstab? so this would only work for translators managed
+ by fstab?
+ <braunr> no
+ <teythoon> ah
+ <teythoon> a string like the ones found in fstab?
+ <braunr> yes
+ <braunr> so that fsck and friends are able to know whether a device is
+ mounted or not
+ <braunr> i don't see any other purpose for that string in mtab
+ <braunr> you'd take regular paths as they are, convert device:xxx to
+ /dev/xxx, and return "none" for the rest i suppose
+ <teythoon> ok
+ <braunr> i'm not even sure it's right
+ <braunr> youpi: are you sure it's required ?
+ <teythoon> well it's a start and I think it's not too much work
+ <braunr> aiui, e2fsck may simply find the mount point in fstab, and ask the
+ translator if it's mounted
+ <teythoon> we can refine this later on maybe?
+ <braunr> or rather, init scripts, using mountpoint, before starting e2fsck
+ <braunr> teythoon: sure
+ <teythoon> there's this mountpoint issue... I need to run fsysopts /
+ --update early in the boot process
+ <teythoon> otherwise the device ids returned by stat(2)ing / are wrong and
+ mountpoint misbehaves
+ <teythoon> i guess b/c it's the rootfs
+ <braunr> device ids ?
+ <teythoon> % stat / | grep Device
+ <teythoon> Device: 3h/3d Inode: 2 Links: 22
+ <braunr> do you mean the major/minor identifiers ?
+ <teythoon> I do. if I don't do the --update i get seemingly random values
+ <braunr> i guess that's expected
+ <braunr> we don't have major/minor values
+ <braunr> well, they're emulated
+ <teythoon> well, if that's fixable, that'd be really nice ;)
+ <braunr> we'll never have major/minor values
+ <teythoon> yeah, I understand that
+ <braunr> but they could be fixed by MAKEDEV when creating device nodes
+ <teythoon> but not having to call fsys_set_options on the rootfs to get the
+ emulation up to speed
+ <braunr> try doing it from grub
+ <braunr> not sure it's possible
+ <braunr> but worth checking
+ <teythoon> by means of an ext2fs flag?
+ <braunr> yes
+ <braunr> if there is one
+ <braunr> i don't know the --update flag, is it new from your work ?
+ <teythoon> braunr: no, it's been there before. -oremount gets mapped to
+ that
+ <braunr> it's documented by fsysopts, but not by the ext2fs translators
+ <teythoon> libdiskfs source says something about flushing buffers iirc
+ <braunr> -s
+ <braunr> what does it do ?
+ <braunr> teythoon: ok
+ <teythoon> braunr: so the plan is to automatically generate a device path
+ from the translators argz vector but to provide the functionality so
+ translators can set a more appropriate value? did I get the last part of
+ the discussion right?
+ <braunr> not set, return
+ <teythoon> yeah return from the procedure but settable using libfshelp?
+ <braunr> why settable ?
+ <braunr> you'd have a fsys call to obtain the dev string, and the server
+ side would call libfshelp on the fly to obtain a normalized value and
+ return it
+ <teythoon> ah, make a function overrideable that returns an appropriate
+ response?
+ <braunr> overrideable ?
+ <teythoon> like netfs_append_args
+ <braunr> you wouldn't change the command line, no
+ <teythoon> isn't that done using weak references or something?
+ <teythoon> no I know
+ <braunr> sorry i'm lost then
+ <teythoon> never mind, I'll propose a patch early to get your feedback
+ <youpi> braunr: am I sure that _what_ is required?
+ <youpi> the device?
+ <youpi> e2fsck surely needs it, yes
+ <braunr> a valid device path, yes
+ <youpi> it can't rely only on fstab
+ <braunr> yes
+ <youpi> since users may mount things by hand
+ <braunr> i've used strace on it and it does perform lookups there
+ <braunr> (although i also saw uuid magic that i guess wouldn't work yet on
+ the hurd)
+
+
+## IRC, freenode, #hurd, 2013-07-03
+
+ <teythoon> I added a procedure to fsys.defs, added a server stub to my
+ tmpfs translator and wrote a simple client, but something hasn't picked
+ up the new message yet
+ <teythoon> % ./mtab tmp
+ <teythoon> ./mtab: get_translator_info: (ipc/mig) bad request message ID
+ <teythoon> I guess it's libhurduser.so from glibc, not sure though...
+ <braunr> glibc would only have the client calls
+ <braunr> what is "% ./mtab tmp" ?
+ <teythoon> mtab is my mtab tool/soon to be a translator testing thing, tmp
+ is an active tmpfs with the appropriate server stub
+ <braunr> so mtab has the client call, right ?
+ <teythoon> yes
+ <braunr> then tmpfs doesn't
+ <teythoon> so what to do about it?
+ <teythoon> i set LD_LIBRARY_PATH to my hurd builds lib dir, is that
+ preserved by settrans -a?
+ <pinotree> not really
+ <braunr> not at all
+ <braunr> there is a wiki entry about that iirc
+ <pinotree> http://darnassus.sceen.net/~hurd-web/hurd/debugging/translator/
+ <teythoon> yeah, I read it too once
+ <teythoon> ah
+ <braunr> on the other hand, using export to set the environment should do
+ the work
+ <teythoon> yes, that did the trick, thanks :)
+ * teythoon got his EOPNOPSUPP... *nomnomnom
+ <braunr> ?
+ <braunr> same error ?
+ <teythoon> well I stubbed it out
+ <braunr> oh
+ <teythoon> no, that's what I've been expecting ;)
+ <pinotree> great
+ <braunr> :)
+ <braunr> yes that's better than "mig can't find it"
+ <teythoon> braunr: in that list of active and passive translators that will
+ have to be maintained, do you expect it should carry more information
+ other than the relative path to that translator?
+ <braunr> like what ?
+ <teythoon> dunno, maybe like a port to any active translator there
+ <teythoon> should we care if any active translator dies and remove the
+ entry if there's no passive translator that could restart it again?
+ <braunr> don't add anything until you see it's necessary or really useful
+ <braunr> yes
+ <braunr> think of something like sshfs
+ <braunr> when you kill it, it's not reported by mount any more
+ <teythoon> well, for a dynamically allocated list of strings I could use
+ the argz stuff, but if we'd ever add anything else, we'd need a linked
+ list or something, maybe a hash table
+ <teythoon> yes, I thought that'd be useful
+ <braunr> use libihash for no
+ <braunr> now
+ <teythoon> braunr: but what would I use as keys? the relative path should
+ be unique (unless translators are stacked... hmmm), but that's the value
+ I'd like to store and ihash keys are pointers
+ <teythoon> stacked translators are an kinda interesting case for mtab
+ anyways...
+ <braunr> why not store the string address ?
+ <braunr> i suppose that, for stacked translators, the code querying
+ information would only return the topmost translator
+ <braunr> since this is the one which matters for regular clients (if i'm
+ right)
+ <teythoon> wouldn't that map strings that are equal but stored at different
+ locations to different values?
+ <teythoon> that'd defeat the point
+ <teythoon> I suppose so, yes
+ <braunr> then add a layer that looks for existing strings before adding
+ <braunr> the list should normally be small so a linear lookup is fine
+ <teythoon> yeah sure, but then there's little advantage of using ihash in
+ the first place, isn't it?
+ <braunr> over what ?
+ <teythoon> over not using it at all
+ <braunr> how would you store the list then ?
+ <teythoon> it's either ll or ll+ihash
+ <braunr> uh no
+ <braunr> let me check
+ <braunr> there is ihash_iterate
+ <braunr> so you don't need a linked list
+ <teythoon> so how do I store my list of strings to deduplicate the keys?
+ <braunr> you store pointers
+ <braunr> and on addition, you iterate over all entries, making sure none
+ matches the new one
+ <braunr> and if it does, you replace it i guess
+ <braunr> depending on how you design the rest
+ <teythoon> in an dynamically allocated region of memory?
+ <braunr> i don't understand
+ <braunr> your strings should be dynmaically allocate, yes
+ <teythoon> no the array of char *
+ <braunr> your data structure being managed by libihash, you don't care
+ about allocation
+ <braunr> what array ?
+ <teythoon> ah, got it...
+ <teythoon> right.
+ <braunr> there is only one structure here, an ihash of char *
+ <teythoon> yes, I got the picture ;)
+ <braunr> goo
+ <braunr> d
+ <braunr> actually, the lookup wouldn't be linear since usually, hash tables
+ have stale entries
+ <teythoon> heh... what forest?!?
+ <braunr> but that's ok
+ <braunr> teythoon: ?
+ <teythoon> the one I couldn't make out b/c of all the trees...
+ <braunr> ?
+ <teythoon> ah, it's not important. there is this saying over here, not sure
+ if there's an english equivalent
+ <braunr> ok got it
+ <braunr> we have the same in french
+ <teythoon> I ran into a problem with my prototype
+ <teythoon> if an translator is set in e. g. diskfs_S_file_set_translator,
+ how do I get the path to that node?
+ <teythoon> I believe there cannot be a way to do that, b/c the mapping is
+ not bijective
+ <braunr> it doesn't have to be
+ <teythoon> ok, so how do I get *a* path for this node?
+ <braunr> that's another question
+ <braunr> do you see how the node is obtained ?
+ <braunr> np = cred->po->np;
+ <teythoon> yes
+ <braunr> the translation occurred earlier
+ <braunr> you need to find where
+ <braunr> then perhaps, you'll need to carry the path along
+ <braunr> or if you're lucky, it will still be there somewhere
+ <teythoon> the translation from path to node?
+ <braunr> yes
+ <teythoon> doesn't that happen in the client? and the client hands a file_t
+ to the file_set_translator routine?
+ <braunr> the relative lookup can't happen in the client
+ <braunr> the server can (and often does) retain information between two
+ RPCs
+ <teythoon> uh, I can access information from a previous rpc? is that
+ considered safe?
+ <braunr> think of open() then read()
+ <braunr> a simple int doesn't carry enough information
+ <braunr> that's why it's a descriptor
+ <teythoon> ah, the server retains some state, sure
+ <braunr> what it refers to is the state retained between several calls
+ <braunr> the object being invoked by clients
+ <braunr> teythoon: what is the "passive" parameter passed to
+ diskfs_S_file_set_translator ?
+ <teythoon> braunr: argz vector of the passive translator
+ <braunr> so it is a name
+ <braunr> but we also want active translators
+ <braunr> and what is active ?
+ <teythoon> not the name of the node though
+ <teythoon> active is the port (?) to the active translator
+ <teythoon> I guess
+ <braunr> fsys_t, looks that way yes
+ <braunr> i suppose you could add the path to the peropen structure
+ <teythoon> ok
+ <braunr> see diskfs_make_peropen
+ <teythoon> braunr: but translation happens in dir_lookup
+ <teythoon> in all places I've seen diskfs_make_peropen used, the path is
+ not available
+ <teythoon> why did you point me to diskfs_make_peropen?
+ <teythoon> s/dir_lookup/diskfs_lookup/
+ <teythoon> diskfs_lookup operates on struct node, so the path would have to
+ be stored there, right?
+ <braunr> teythoon: dir_lookup should call diskfs_make_peropen
+ <braunr> at least diskfs_S_dir_lookup does
+ <braunr> and the path is present there
+ <teythoon> braunr: right
+
+ <teythoon> hrm... I added a path field to struct peropen and initialize it
+ properly in diskfs_make_peropen, but some bogus values keep creeping in
+ :/
+ <braunr> first of all, make it a dynamically allocated string
+ <teythoon> it is
+ <braunr> not a fixed sized embedded array
+ <braunr> good
+ <teythoon> yes
+ <braunr> if you really need help debugging what's happening, feel free to
+ post your current changes somewhere
+ <teythoon> there is a struct literal in fsys-getroot.c, but i fixed that as
+ well
+ <teythoon> % ./mtab tmp
+ <teythoon> none tmp ../tmpfs/tmpfs writable,no-inherit-dir-group,no-sync 0
+ 0
+ <teythoon> none tmp/bar ../tmpfs/tmpfs
+ writable,no-inherit-dir-group,no-sync 0 0
+ <teythoon> none tmp/foo ../tmpfs/tmpfs
+ writable,no-inherit-dir-group,no-sync 0 0
+ <teythoon> none tmp/foo/bar ../tmpfs/tmpfs
+ writable,no-inherit-dir-group,no-sync 0 0
+ <teythoon> :)
+
+
+## IRC, freenode, #hurd, 2013-07-10
+
+ <teythoon> btw, I read getcwd.c and got the idea
+ <teythoon> however this situation is different afaict
+ <teythoon> getcwd has a port to the current working directory, right?
+ <teythoon> so they can do open_dir with .. as relative path
+ <teythoon> but all I've got is a port referencing the node the translator
+ is being attached to
+ <teythoon> s/open_dir/dir_lookup/
+ <teythoon> and that is not necessarily a directory, so dir_lookup fails
+ with not a directory
+ <teythoon> as far as I can see it is not possible to get the directory a
+ node is in from a port referencing that node
+ <teythoon> dir_lookup has to be handled by all nodes, not just directories
+ <teythoon> but file nodes only support "looking up" the empty string
+ <teythoon> not empty, but null:
+ <teythoon> This call is required to be supported by all files (even
+ non-directories) if the filename is null, and should function in that
+ case as a re-open of the file. */
+ <braunr> why do you want the directory ?
+ <braunr> 10:40 < teythoon> as far as I can see it is not possible to get
+ the directory a node is in from a port referencing that node
+ <teythoon> to readdir(3) it and figure out the name of the node the
+ translator is bound to
+ <teythoon> similar to what getcwd does
+ <braunr> that's out of the question
+ <teythoon> wasn't that was youpi was suggesting?
+ <braunr> you may have a lot of nodes in there, such a lookup shouldn't be
+ done
+ <braunr> i didn't see that detail
+ <teythoon> "│ Concerning storing the path, it's a bit sad to have to do
+ that, and
+ <teythoon> │ it'll become wrong if one moves the mount points. Another
+ way would
+ <teythoon> │ be to make the client figure it out by itself from a port to
+ the mount
+ <teythoon> │ point, much like glibc/sysdeps/mach/hurd/getcwd.c. It'll be
+ slower, but
+ <teythoon> │ should be safer. The RPC would thus return an array of
+ ports to the
+ <teythoon> │ mount points instead of an array of strings.
+ <braunr> yes i remember that
+ <braunr> but i didn't understand well how getcwd work
+ <braunr> s
+ <braunr> another scalability issue
+ <braunr> not a big one though, we rarely have translators in directories
+ with thousands of nodes
+ <braunr> so why not
+ <braunr> teythoon: do it as youpi suggested
+ <braunr> well if you can
+ <braunr> eh
+ <braunr> if not, i don't know
+ <braunr> 10:47 < teythoon> │ it'll become wrong if one moves the mount
+ points. Another way would
+ <teythoon> yes, I know... :/
+ <teythoon> well, I'm not even sure it is possible to get the directory a
+ node is in from the port referencing the node
+ <teythoon> as in, I'm not sure if the information is even there
+ <teythoon> b/c a filesystem is a tree, directories are nodes and files are
+ leafs
+ <teythoon> all non-leaf nodes reference their parent to allow traversing
+ the tree starting from any directory
+ <teythoon> but why would a leaf reference its parent(s - in case of
+ hardlinks)?
+ <braunr> uh, for the same reason ?
+ <teythoon> sure, it would be nice to do that, but I dont think this is
+ possible on unixy systems
+ <braunr> ?
+ <teythoon> you cannot say fchdir(2) to a fd that references a file
+ <braunr> do you mean /path/to/file/../ ?
+ <teythoon> yes
+ <teythoon> only that /path/to/file is given as fd or port
+ <braunr> when i pasted
+ <braunr> 10:49 < braunr> 10:47 < teythoon> │ it'll become wrong if one
+ moves the mount points. Another way would
+ <braunr> i was actually wondering if it was true
+ <teythoon> ah
+ <braunr> why can't the path be updated at the same time ?
+ <braunr> it's a relative path anyway
+ <braunr> completely managed by the parent translator
+ <teythoon> ah
+ <teythoon> right
+ <teythoon> it's still kind of hacky, but I cannot see how to do this
+ properly
+ <braunr> hacky ?
+ <teythoon> but yes, updating the path should work I guess
+ <teythoon> or sad
+ <braunr> what i find hacky is to set translators in two passes
+ <braunr> otherwise we'd only keep the translator paths
+ <braunr> not all paths
+ <teythoon> true
+ <braunr> but then, it only concerns open nodes
+ <braunr> and again, there shouldn't be too many of them
+ <braunr> so actually it's ok
+ <teythoon> braunr: I understand the struct nodes are cached in libdiskfs,
+ so wouldn't it be easier to attach the path to that struct instead of
+ struct peropen so that all peropen objects reference the same node
+ object?
+ <teythoon> so that the path can be updated if anyone dir_renames it
+ <teythoon> *all peropen objects derived from the same file name that is
+ <braunr> teythoon: i'm not sure
+ <braunr> nodes could be real nodes (i.e. inodes)
+ <braunr> there can be several paths for the same inode
+ <teythoon> braunr: I'm aware of that, but didn't we agree the other day
+ that any path would do?
+ <braunr> i don't remember we did
+ <braunr> i don't know the details well, but i don't think setting a
+ translator on a hard link should set the translator at the inode level
+ <braunr> on the other hand, if a new inode is created to replace the
+ previous one (or stack over it), then storing the path there should be
+ fine
+ <teythoon> braunr: I don't think I can update the paths if they're stored
+ in the peropen struct
+ <teythoon> how would I get a reference to all those peropen objects?
+ <braunr> ?
+ <braunr> first, what's the context when you talkb about updating paths ?
+ <teythoon> well, youpi was concerned about renaming a mount point
+ <teythoon> and you implied that this could be managed
+ <braunr> can we actually do that btw ?
+ <teythoon> what?
+ <braunr> renaming a mount point
+ <teythoon> yep, just tried
+ <braunr> i mean, on a regular unix system like linux
+ <braunr> $ mv test blah
+ <braunr> mv: cannot move `test' to `blah': Device or resource busy
+ <braunr> (using sshfs so YMMV)
+ <pinotree> do you have anything (shells, open files, etc) inside it?
+ <braunr> no
+ <braunr> i'll try with an empty loop-mounted ext4
+ <teythoon> I was testing on the Hurd, worked fine there even with a shell
+ inside
+ <braunr> same thing
+ <braunr> i consider it a bug
+ <braunr> we may want to check what posix says about it
+ <teythoon> o_O
+ <braunr> and decide not to support renaming
+ <teythoon> why?
+ <pinotree> start a discussion in ml, maybe roland can chime in
+ <braunr> it complicates things
+ <braunr> ah yes
+ <teythoon> sure, but I can move or rename a directory, why should it be
+ different with a mount point?
+ <braunr> because it's two of them
+ <braunr> they're stacked
+ <braunr> if we do want to support that, we must be very careful about
+ atomically updating all the stack
+ <teythoon> ok
+ <teythoon> braunr: I'm trying to detect dying translators to remove them
+ from the list of translators
+ <teythoon> what port can I use for that purpose?
+ <teythoon> if I use the bootstrap port, can I then use the same method as
+ init/init.c uses? just defining a do_mach_notify_dead_name function and
+ the multiplexer will call this?
+ <braunr> teythoon: possibly
+ <teythoon> braunr: we'll see shortly...
+ <teythoon> I get KERN_INVALID_CAPABILITY indicating that my bootstrap port
+ is invalid
+ <teythoon> when calling mach_port_request_notification to get the dead name
+ notification I mean
+ <braunr> is the translator already started when you do that ?
+ <teythoon> yes, at least I think so, I'm hooking into
+ diskfs_S_file_set_translator and that gets an active translators port
+ <teythoon> also the mach docs suggests that the notification port is
+ invalid, not the name port referencing the translator
+ <braunr> i guess it shouldn't
+ <braunr> oh
+ <braunr> please show the code
+ <braunr> but beware, if the translator is started, assume it could die
+ immediately
+ <teythoon> braunr: http://paste.debian.net/15371/ line 87
+ <braunr> teythoon: notify can't be bootstrap
+ <braunr> what do you have in mind when writing this ?
+ <braunr> i'm not sure i follow
+ <teythoon> I want to be notified if an active translator goes away to
+ remove it from the list of translators
+ <braunr> ok but then
+ <braunr> create a send-once right
+ <braunr> and wait on it
+ <braunr> also, why do you want to be notified ?
+ <braunr> isn't this already done ?
+ <braunr> or can't do it lazily on access attempt ?
+ <braunr> +you
+ <teythoon> in the client?
+ <braunr> in the parent server
+ <braunr> what happens currently when a translator dies
+ <braunr> is the parent notified ?
+ <braunr> or does it give an invalid right ?
+ <teythoon> ah, i think so
+ <braunr> then you don't need to do it again
+ <teythoon> right, I overlooked that
diff --git a/hurd.mdwn b/hurd.mdwn
index ed88f09f..77401278 100644
--- a/hurd.mdwn
+++ b/hurd.mdwn
@@ -77,6 +77,7 @@ in the *unstable* branch of the Debian archive.
# Developer References
+* [[Coding_Style]]
* [[Rules]]
* [[Trackers]]
* [[Building]]
diff --git a/hurd/coding_style.mdwn b/hurd/coding_style.mdwn
new file mode 100644
index 00000000..cc1e3cf3
--- /dev/null
+++ b/hurd/coding_style.mdwn
@@ -0,0 +1,59 @@
+[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_documentation]]
+
+Some coding style comments that are specific to Hurd systems.
+
+[[!toc]]
+
+
+# Freeing Port Rights
+
+## IRC, freenode, #hurd, 2013-07-01
+
+ <teythoon> do I have to explicitly free ports in a short lived process like
+ mount?
+ <pinotree> better take the habit of doing that anyway
+ <teythoon> how do I recognize that I have to free something? mig spec?
+ <braunr> i'd say no
+ <braunr> mig does it for you
+ <braunr> gnumach reference manual
+ <teythoon> not memory, like port rights
+ <braunr> but no, really, for short lived processes it's ok
+ <braunr> yes, port rights
+ <braunr> like memory, you don't free stuff in short lived processes :p
+ <braunr> mach does it correctly when the task is destroyed
+ <braunr> but there are two use cases for rights
+ <braunr> those you create manually
+ <braunr> and those mig creates for its own purpose
+ <braunr> ignore those used by mig, they matter only in very specific parts
+ of glibc and other very low level stuff
+ <braunr> teythoon: keep in mind that there are two flavours of resources
+ with port rights
+ <teythoon> but how do I *know* from looking at say fs.defs that I have to
+ free anything I get?
+ <braunr> rights themselves, and the user reference count per right
+ <braunr> eh, that's complicated
+ <braunr> in a complete RPC call, you must watch two things usually
+ <braunr> out of line memory
+ <braunr> and right references
+ <braunr> except otherwise mentioned, you don't have to free anything
+ <braunr> freeing passed memory should be obvious (e.g. "out" keyword on a
+ memory range)
+ <braunr> for right references, it's less obvious
+ <braunr> refer to the mach server writing guide i guess
+ <teythoon> what does the dealloc qualifier do in mig defs?
+ <braunr> basically, send rights can be created from a receive right
+ (make_send), or another send right (copy_send)
+ <braunr> it tells mig which function to call once an RPC has returned
+ <braunr> all this is described in the mach server writing guide
+ <braunr> and it's tricky
+ <braunr> quite error-prone so check with portinfo
diff --git a/hurd/console/discussion.mdwn b/hurd/console/discussion.mdwn
index f887d826..0022ec23 100644
--- a/hurd/console/discussion.mdwn
+++ b/hurd/console/discussion.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -10,6 +10,8 @@ License|/fdl]]."]]"""]]
[[!tag open_issue_documentation]]
+[[!toc]]
+
# IRC, OFTC, #debian-hurd, 2012-09-24
@@ -40,3 +42,9 @@ License|/fdl]]."]]"""]]
hacking?
<allesa> to fix that is…
<youpi> some hacking yes
+
+
+# IRC, freenode, #hurd, 2013-07-10
+
+ <pinotree> http://xkbcommon.org/ ‘¡û sounds interesting for our console
+ translator
diff --git a/hurd/debugging.mdwn b/hurd/debugging.mdwn
index f4b5eba5..ae9b7bef 100644
--- a/hurd/debugging.mdwn
+++ b/hurd/debugging.mdwn
@@ -26,3 +26,30 @@ License|/fdl]]."]]"""]]
* [[glibc]]
* [[translator]]s
* [[trap_in_the_kernel]]
+
+
+# IRC, freenode, #hurd, 2013-06-30
+
+ <hacklu> braunr: I don't understand your question totally, but I want to
+ know how do you do this inspecting? <braunr> i have a small test program
+ that creates a thread, and inspect its state before any thread dies
+ <braunr> i use portinfo
+ <braunr> and rpctrace
+ <braunr> (there is also vminfo but you're not likely to need it for what
+ you're doing right now)
+ <hacklu> I have used rpctrace before, but portinfo, I will try it.
+ <hacklu> is portinfo show a process's all port use log?
+ <braunr> not log
+ <braunr> current state
+ <hacklu> dump the port name space?
+ <braunr> yes
+ <hacklu> I found some names are not continuous. how this come out?
+ <braunr> continuous ?
+ <hacklu> 101:send 103:send
+ <hacklu> missing 102
+ <braunr> some are freed
+ <braunr> a lot actually
+ <braunr> every RPC needs a reply port
+ <braunr> a temporary receive right to get replies from servers
+ <hacklu> so we can reuse the name which are freed before
+ <braunr> of course
diff --git a/hurd/debugging/rpctrace.mdwn b/hurd/debugging/rpctrace.mdwn
index a5c1a6e9..1570df4c 100644
--- a/hurd/debugging/rpctrace.mdwn
+++ b/hurd/debugging/rpctrace.mdwn
@@ -182,6 +182,10 @@ See `rpctrace --help` about how to use it.
<youpi> uhu, there's a TODO just above that assertion :)
+* IRC, freenode, #hurd, 2013-07-05
+
+ <pinotree> wish: make rpctrace decode the results of io_stat rpcs
+
# See Also
diff --git a/hurd/translator.mdwn b/hurd/translator.mdwn
index d4eaf950..37dac794 100644
--- a/hurd/translator.mdwn
+++ b/hurd/translator.mdwn
@@ -90,6 +90,7 @@ The [[concept|concepts]] of translators creates its own problems, too:
* [[hello]]
* [[auth]]
* [[exec]]
+* [[proc]]
* [[pfinet]]
* [[pflocal]]
* [[hostmux]]
@@ -112,6 +113,7 @@ The [[concept|concepts]] of translators creates its own problems, too:
* [[procfs]]
* [[nsmux]]
* [[netio]]
+* [[socketio]]
* [[tarfs]]
* [[gopherfs]]
* [[smbfs]]
diff --git a/hurd/translator/auth.mdwn b/hurd/translator/auth.mdwn
index d9e70ec2..7fd4832c 100644
--- a/hurd/translator/auth.mdwn
+++ b/hurd/translator/auth.mdwn
@@ -1,12 +1,19 @@
-[[!meta copyright="Copyright © 2008 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2008, 2013 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
-is included in the section entitled
-[[GNU Free Documentation License|/fdl]]."]]"""]]
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+The *auth server* (or, *authentification server*).
+
+It is stated by `/hurd/init`.
+
+
+# Documentation
[[*The_Authentication_Server*|documentation/auth]], the transcript of a talk
about the details of the authentication mechanisms in the Hurd by Wolfgang
diff --git a/hurd/translator/firmlink.mdwn b/hurd/translator/firmlink.mdwn
index 038879db..b53396b0 100644
--- a/hurd/translator/firmlink.mdwn
+++ b/hurd/translator/firmlink.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -20,3 +20,15 @@ License|/fdl]]."]]"""]]
<braunr> infinity0: firmlinks
<infinity0> ah thanks i'll look that up
<kilobug> braunr: oh, true, I forgot about that one
+
+
+# Open Issues
+
+ * [[!GNU_Savannah_bug 29809]]
+
+ * IRC, freenode, #hurd, 2013-07-07
+
+ <youpi> ok, found the firmlink-mv issue
+ <youpi> will commit that
+ <pinotree> \o/
+ <youpi> (bit of mach_print in translators, took just a few hours)
diff --git a/hurd/translator/netio.mdwn b/hurd/translator/netio.mdwn
index 44c35cf1..429072d4 100644
--- a/hurd/translator/netio.mdwn
+++ b/hurd/translator/netio.mdwn
@@ -11,6 +11,8 @@ License|/fdl]]."]]"""]]
`netio` is a translator designed for creating socket ports through the
filesystem.
+See also [[socketio]].
+
# Source
diff --git a/hurd/translator/proc.mdwn b/hurd/translator/proc.mdwn
new file mode 100644
index 00000000..98940f87
--- /dev/null
+++ b/hurd/translator/proc.mdwn
@@ -0,0 +1,53 @@
+[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+The *proc server* (or, *process server*) implements some aspects of [[Unix]]
+processes.
+
+It is stated by `/hurd/init`.
+
+
+# "Unusual" PIDs
+
+[[!tag open_issue_hurd]]
+
+
+## IRC, freenode, #hurd, 2012-08-10
+
+ <braunr> too bad the proc server has pid 0
+ <braunr> top & co won't show it
+
+
+## IRC, OFTC, #debian-hurd, 2012-09-18
+
+ <pinotree> youpi: did you see
+ https://enc.com.au/2012/09/careful-with-pids/'
+ <pinotree> ?
+ <youpi> nope
+
+
+## IRC, OFTC, #debian-hurd, 2013-06-23
+
+ <teythoon> I've got this idea about the pid 1 issue you mentioned
+ <teythoon> can't we just make init pid 1?
+ <teythoon> I mean the mapping is rather arbitrary, we could make our init
+ pid 2 or something and start sysvs init as pid 1
+ <pinotree> not totally sure it is that arbitrary up to the first 4-5 pids
+ <teythoon> y is that?
+ <pinotree> at least i see in hurd's code that /hurd/init is assumed as
+ pid=1
+ <teythoon> hurds init that has to stick around for the fs translator sync?
+ <pinotree> hurd's init does the basic server startup
+ <pinotree> iirc it also takes care of shutdown/reboot
+ <teythoon> that's what I meant
+ <teythoon> and if it wouldn't have to stick around for the translator sync
+ it could just exec sysvinit
+ <teythoon> I just think it's easier to patch hurd than to remove the
+ assumption that init is pid 1 from sysvinit
diff --git a/hurd/translator/procfs/jkoenig/discussion.mdwn b/hurd/translator/procfs/jkoenig/discussion.mdwn
index 197461b8..fcda453e 100644
--- a/hurd/translator/procfs/jkoenig/discussion.mdwn
+++ b/hurd/translator/procfs/jkoenig/discussion.mdwn
@@ -441,26 +441,6 @@ Needed by glibc's `pldd` tool (commit
on "internal" (but exported) glibc functions
-# "Unusual" PIDs
-
-Not actually related to procfs, but here seems to be a convenient place for
-filing these:
-
-
-## IRC, freenode, #hurd, 2012-08-10
-
- <braunr> too bad the proc server has pid 0
- <braunr> top & co won't show it
-
-
-## IRC, OFTC, #debian-hurd, 2012-09-18
-
- <pinotree> youpi: did you see
- https://enc.com.au/2012/09/careful-with-pids/'
- <pinotree> ?
- <youpi> nope
-
-
# CPU Usage
## IRC, freenode, #hurd, 2013-01-30
diff --git a/hurd/translator/socketio.mdwn b/hurd/translator/socketio.mdwn
new file mode 100644
index 00000000..479d5749
--- /dev/null
+++ b/hurd/translator/socketio.mdwn
@@ -0,0 +1,30 @@
+[[!meta copyright="Copyright © 2010, 2013 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+`socketio` is a translator designed for creating socket ports through the
+filesystem.
+
+See also [[netio]].
+
+
+# IRC, freenode, #hurd, 2013-06-30
+
+ <youpi> http://lists.gnu.org/archive/html/bug-hurd/2003-05/msg00069.html
+ <youpi> this was supposed to be much better than our current netio
+ <youpi> (which doesn't really have any documentation btw)
+ <teythoon>
+ http://web.archive.org/web/20060117085538/http://duesseldorf.ccc.de/~moritz/files/socketio.c.gz
+ <teythoon> youpi: socketio looks nice. any reason in particular why you are
+ working on it?
+ <youpi> teythoon: I was looking at the firewall stuff, and wondering about
+ Zheng Da's work, and seen netio, thus wondered "what is it about
+ already?" and found there was no documentation, so dug into the mailing
+ list archives, only to find it was supposed to be precated in favour of
+ socketio, etc. :)
diff --git a/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn
index 5228515f..16a23405 100644
--- a/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn
+++ b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn
@@ -1,4 +1,5 @@
-[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2010, 2011, 2013 Free Software Foundation,
+Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -270,3 +271,129 @@ See also: [[open_issues/resource_management_problems/pagers]].
<antrik> only problem is that the current defpager implementation can't
really handle that...
<antrik> at least that's my understanding of the situation
+
+
+# IRC, freenode, #hurd, 2013-07-05
+
+ <teythoon> btw, why does the tmpfs translator have to talk to the pager?
+ <teythoon> to get more control about how the memory is paged out?
+ <teythoon> read lot's of irc logs about tmpfs on the wiki, but I couldn't
+ find the answer to that
+ <mcsim> teythoon: did you read this?
+ http://www.gnu.org/software/hurd/hurd/translator/tmpfs/tmpfs_vs_defpager.html
+ <teythoon> mcsim: I did
+ <mcsim> teythoon: Last discussion, i think has very good point.
+ <mcsim> To provide memory objects you should implement pager interface
+ <mcsim> And if you implement pager interface you are the one who is asked
+ to write data to backing storage to evict them
+ <mcsim> But tmpfs doesn't do this
+ <teythoon> mmm, clients doing mmap...
+ <mcsim> teythoon: You don't have mmap
+ <mcsim> teythoon: mmap is implemented on top of mach interface
+ <mcsim> teythoon: I mean you don't have mmap at this level
+ <teythoon> mcsim: sure, but that's close enough for me at this point
+ <mcsim> teythoon: diskfs interface requires implementor to provide a memory
+ object port (send right)
+ <mcsim> Guest8183: Why tmpfs requires defpager
+ <Guest8183> how did you get to talk about that ?
+ <mcsim> I was just asked
+ <teythoon> Guest8183: it's just so unsettling that tmpfs has to be started
+ as root :/
+ <Guest8183> teythoon: why ?
+ *** Guest8183 (~rbraun@dalaran.sceen.net) is now known as braunr_
+ <teythoon> braunr_: b/c starting translators isn't a privileged operation,
+ and starting a tmpfs translator that doesn't even access any device but
+ "just" memory shouldn't require any special privileges as well imho
+ <teythoon> so why is tmpfs not based on say libnetfs? b/c it is used for
+ d-i and someone (apt?) mmaps stuff?
+ <pinotree> being libdiskfs-based isn't much the issue, iirc
+ <pinotree> http://lists.gnu.org/archive/html/bug-hurd/2013-03/msg00014.html
+ too
+ <kilobug> teythoon: AFAIK apt uses mmap, yes
+ <braunr_> teythoon: right
+ <braunr_> a ramfs is actually tricky to implement well
+ <mcsim> braunr_: What do you mean under "to implement well"?
+ <braunr_> as efficiently as possible
+ <braunr_> i.e. being as close as possible to the page cache for minimum
+ overhead
+ <mcsim> braunr: AFAIK ramfs should not use swap partition, so page cache
+ shouldn't be relevant for it.
+ <braunr> i'm talking about a ramfs in general
+ <braunr> not the specific linux ramfs
+ <braunr> in linux, what they call ramfs is the tiny version of tmpfs that
+ doesn't use swap
+ <braunr> i actually don't like "tmpfs" much
+ <braunr> memfs may be more appropriate
+ <braunr> anyway
+ <mcsim> braunr: I see. And do you consider defpager variant as "close as
+ possible to the page cache"?
+ <braunr> not far at least
+ <braunr> if we were able to use it for memory obects, it would be nice
+ <braunr> but defpager only gets attached to objects when they're evicted
+ <braunr> before that, anonymous (or temporary, in mach terminology) objects
+ have no backing store
+ <braunr> this was probably designed without having tmpfs in mind
+ <braunr> i wonder if it's possible to create a memory object without a
+ backing store
+ <mcsim> what should happen to it if kernel decides to evict it?
+ <braunr> it sets the default pager as its backing store and pushes it out
+ <mcsim> that's how it works now, but you said "create a memory object
+ without a backing store"
+ <braunr> mach can do that
+ <braunr> i was wondering if we could do that too from userspace
+ <mcsim> mach does not evict such objects, unless it bound a defpager to
+ them
+ <mcsim> but how can you handle this in userspace?
+ <braunr> i mean, create a memory object with a null control port
+ <braunr> mcsim: is that clearer ?
+ <mcsim> suppose you create such object, how kernel will evict it if kernel
+ does not know who is responsible for eviction of this object?
+ <braunr> it does
+ <braunr> 16:41 < braunr> it sets the default pager as its backing store and
+ pushes it out
+ <braunr> that's how i intend to do it on x15 at least
+ <braunr> but it's much simpler there because uvm provides better separation
+ between anonymous and file memory
+ <braunr> whereas they're much too similar in mach vm
+ <mcsim> than what the difference between current situation, when you
+ explicitly invoke defpager to create object and implicit method you
+ propose?
+ <braunr> you don't need a true defpager unless you actually have swap
+ <mcsim> ok
+ <mcsim> now I see
+ <braunr> it also saves the communication overhead when initializing the
+ object
+ <mcsim> thank you
+ <braunr> which may be important since we use ramfs for speed mostly
+ <mcsim> agree
+ <braunr> it should also simplify the defpager implementation, since it
+ would only have a single client, the kernel
+ <braunr> which may also be important with regard to global design
+ <braunr> one thing which is in my opinion very wrong with mach is that it
+ may be a client
+ <braunr> a well designed distributed system should normally not allow on
+ component to act as both client and server toward another
+ <braunr> i.e. the kernel should only be a server, not a client
+ <braunr> and there should be a well designed server hierarchy to avoid
+ deadlocks
+ <braunr> (such as the one we had in libpager because of that)
+ <mcsim> And how about filesystem? It acts both as server and as client
+ <braunr> yes
+ <braunr> but not towards the same other component
+ <braunr> application -> file system -> kernel
+ <braunr> no "<->"
+ <braunr> the qnx documentation explains that quite well
+ <braunr> let me see if i can find the related description
+ <mcsim> Basically, I've got your point. And I would rather agree that
+ kernel should not act as client
+ <braunr> mcsim:
+ http://www.qnx.com/developers/docs/6.4.0/neutrino/sys_arch/ipc.html#Robust
+ <braunr> one way to implement that (and qnx does that too) is to make
+ pagers act as client only
+ <braunr> they sleep in the kernel, waiting for a reply
+ <braunr> and when the kernel needs to evict something, a reply is sent
+ <braunr> (qnx doesn't actually do that for paging, but it's a general idea)
+ <mcsim> braunr: how hierarchy of senders is enforced?
+ <braunr> it's not
+ <braunr> developers must take care
+ <braunr> same as locking, be careful about it
diff --git a/libpthread.mdwn b/libpthread.mdwn
index 0a518996..fc5c0974 100644
--- a/libpthread.mdwn
+++ b/libpthread.mdwn
@@ -60,45 +60,8 @@ even if the current number of threads is lower.
The same issue exists in [[hurd/libthreads]].
-
-### IRC, freenode, #hurd, 2013-05-09
-
- <bddebian> braunr: Speaking of which, didn't you say you had another "easy"
- task?
- <braunr> bddebian: make a system call that both terminates a thread and
- releases memory
- <braunr> (the memory released being the thread stack)
- <braunr> this way, a thread can completely terminates itself without the
- assistance of a managing thread or deferring work
- <bddebian> braunr: That's "easy" ? :)
- <braunr> bddebian: since it's just a thread_terminate+vm_deallocate, it is
- <braunr> something like thread_terminate_self
- <bddebian> But a syscall not an RPC right?
- <braunr> in hurd terminology, we don't make the distinction
- <braunr> the only real syscalls are mach_msg (obviously) and some to get
- well known port rights
- <braunr> e.g. mach_task_self
- <braunr> everything else should be an RPC but could be a system call for
- performance
- <braunr> since mach was designed to support clusters, it was necessary that
- anything not strictly machine-local was an RPC
- <braunr> and it also helps emulation a lot
- <braunr> so keep doing RPCs :p
-
-
-#### IRC, freenode, #hurd, 2013-05-10
-
- <braunr> i'm not sure it should only apply to self though
- <braunr> youpi: can we get a quick opinion on this please ?
- <braunr> i've suggested bddebian to work on a new RPC that both terminates
- a thread and releases its stack to help fix libpthread
- <braunr> and initially, i thought of it as operating only on the calling
- thread
- <braunr> do you see any reason to make it work on any thread ?
- <braunr> (e.g. a real thread_terminate + vm_deallocate)
- <braunr> (or any reason not to)
- <youpi> thread stack deallocation is always a burden indeed
- <youpi> I'd tend to think it'd be useful, but perhaps ask the list
+The current implementation in libpthread is
+[[buggy|libpthread/t/fix_have_kernel_resources]].
# Open Issues
diff --git a/microkernel/mach/deficiencies.mdwn b/microkernel/mach/deficiencies.mdwn
index d1cdeb54..03e4a8b0 100644
--- a/microkernel/mach/deficiencies.mdwn
+++ b/microkernel/mach/deficiencies.mdwn
@@ -2190,3 +2190,131 @@ In context of [[open_issues/multithreading]] and later [[open_issues/select]].
<core-ix> i really need to dig into this code at some point :)
<braunr> well you may but you may not see that property from the code
itself
+
+
+## IRC, freenode, #hurd, 2013-06-28
+
+ <teythoon> so tell me about x15 if you are in the mood to talk about that
+ <braunr> what do you want to know ?
+ <teythoon> well, the high level stuff first
+ <teythoon> like what's the big picture
+ <braunr> the big picture is that x15 is intended to be a "better mach for
+ the hurd
+ <braunr> "
+ <braunr> mach is too general purpose
+ <braunr> its ipc mechanism too powerful
+ <braunr> too complicated, error prone, and slow
+ <braunr> so i intend to build something a lot simpler and faster :p
+ <teythoon> so your big picture includes actually porting hurd? i thought i
+ read somewhere that you have a rewrite in mind
+ <braunr> it's a clone, yes
+ <braunr> x15 will feature mostly sync ipc, and no high level types inside
+ messages
+ <braunr> the ipc system call will look like what qnx does
+ <braunr> send-recv from the client, recv/reply/reply-recv from the server
+ <teythoon> but doesn't sync mean that your context switch will have to be
+ quite fast?
+ <braunr> how does that differ from the async approach ?
+ <braunr> (keep in mind that almost all hurd RPCs are synchronous)
+ <teythoon> yes, I know, and it also affects async mode, but a slow switch
+ is worse for the sync case, isn't it?
+ <teythoon> ok so your ipc will be more agnostic wrt to what it transports?
+ unlike mig I presume?
+ <braunr> no it's the same
+ <braunr> yes
+ <braunr> input will be an array, each entry denoting either memory or port
+ rights
+ <braunr> (or directly one entry for fast ipcs)
+ <teythoon> memory as in pointers?
+ <braunr> (well fast ipc when there is only one entry to avoid hitting a
+ table)
+ <braunr> pointer/size yes
+ <teythoon> hm, surely you want a way to avoid copying that, right?
+ <braunr> the only operation will be copy (i.e. unlike mach which allows
+ sharing)
+ <braunr> why ?
+ <braunr> copy doesn't exclude zero copy
+ <braunr> (zero copy being adjusting page tables with copy on write
+ techniques)
+ <teythoon> right
+ <teythoon> but isn't that too coarse, like in cow a whole page?
+ <braunr> depends on the message size
+ <braunr> or options provided by the caller, i don't know yet
+ <teythoon> oh, you are going to pack the memory anyway?
+ <braunr> depends on the caller
+ <braunr> i'm not yet sure about these details
+ <braunr> ideally, i'd like to avoid serialization altogether
+ <teythoon> wouldn't that be like cheating b/c it's the first copy?
+ <braunr> directly pass pointers/sizes from the sender address space, and
+ either really copy or use zero copy
+ <teythoon> right, but then you're back at the page size issue
+ <braunr> yes
+ <braunr> it's not a real issue
+ <braunr> the kernel must support both ways
+ <braunr> the minor issue is determining which way to choose
+ <braunr> it's not a critical issue
+ <braunr> my current plan is to always copy, unless the caller has
+ explicitely set a flag and is passing properly aligned buffers
+ <teythoon> u sure? I mean the caller is free to arange the stuff he intends
+ to send anyway he likes, how are you going to cow that then?
+ <teythoon> ok
+ <teythoon> right
+ <braunr> properly aligned buffers :)
+ <braunr> otherwise the kernel rejects the request
+ <teythoon> that's reasonable, yes
+ <braunr> in addition to being synchronous, ipc will also take a special
+ path in the scheduler to directly use the client scheduling context
+ <braunr> avoiding the sleep/wakeup overhead, and providing priority
+ inheritence by side effect
+ <teythoon> uh, but wouldn't dropping serialization create security and
+ reliability issues? if the receiver isn't doing a proper job sanitizing
+ its stuff
+ <braunr> why would the client not sanitize ?
+ <braunr> err
+ <braunr> server
+ <braunr> it has to anyway
+ <teythoon> sure, but a proper parser written once might be more robust,
+ even if it adds overhead
+ <teythoon> the serialization i mean
+ <braunr> it's just a layer
+ <braunr> even with high level types, you still need to sanitize
+ <braunr> the real downside is loosing cross architecture portability
+ <braunr> making the potential implementation of a single system image a lot
+ more restricted or difficult
+ <braunr> but i don't care about that much
+ <braunr> mach was built with this in mind though
+ <teythoon> it's a nice idea, but i don't believe anyone does ssi anymore
+ <braunr> i don't know
+ <teythoon> and certainly not across architectures
+ <braunr> there are few projects
+ <braunr> anyway it's irrelevant currently
+ <braunr> and my interface just restricts it, it doesn't prevent it
+ <braunr> so i consider it an acceptable compromise
+ <teythoon> so, does it run? what does it do?
+ <teythoon> it certainly is, yes
+ <braunr> for now, it manages memory (physical, virtual, kernel, and soon,
+ anonymous)
+ <braunr> support multiple processors with the required posix scheduling
+ policies
+ <braunr> (it uses a cute proportionally fair time sharing algorithm)
+ <braunr> there are locks (spin locks, mutexes, condition variables) and
+ lockless stuff (à la rcu)
+ <braunr> both x86 and x86_64 are supported
+ <braunr> (even pae)
+ <braunr> work queues
+ <teythoon> sounds impressive :)
+ <braunr> :)
+ <braunr> i also added basic debugging
+ <braunr> stack trace (including getting the symbol table) handling
+ <braunr> so yes, it's much much better than what i previously did
+ <braunr> and on the right track
+ <braunr> it already scales a lot better than mach for what it does
+ <braunr> there are generic data structures (linked list, red-black tree,
+ radix tree)
+ <braunr> the radix tree supports lockless lookups, so looking up both the
+ page cache and the ipc spaces is lockless)
+ <teythoon> that's nice :)
+ <braunr> there are a few things using global locks, but there are TODOs
+ about them
+ <braunr> even with that, it should be scalable enough for a start
+ <braunr> and improving those parts shouldn't be too difficult
diff --git a/microkernel/mach/gnumach/debugging.mdwn b/microkernel/mach/gnumach/debugging.mdwn
index 71e92459..7e7cfb4e 100644
--- a/microkernel/mach/gnumach/debugging.mdwn
+++ b/microkernel/mach/gnumach/debugging.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2007, 2008, 2009, 2011, 2012 Free Software
+[[!meta copyright="Copyright © 2007, 2008, 2009, 2011, 2012, 2013 Free Software
Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
@@ -75,6 +75,9 @@ When you're [[running_a_system_in_QEMU|hurd/running/qemu]] you can directly
kernel](http://www.nongnu.org/qemu/qemu-doc.html#SEC48).
+## [[open_issues/debugging_gnumach_startup_qemu_gdb]]
+
+
# Code Inside the Kernel
Alternatively you can use an approach like this one: add the following code
diff --git a/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn b/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn
index ca52dca5..a169e92e 100644
--- a/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn
+++ b/microkernel/mach/gnumach/interface/syscall/mach_print.mdwn
@@ -59,3 +59,32 @@ License|/fdl]]."]]"""]]
it
[[Makefile]], [[mach_print.S]], [[main.c]].
+
+
+## IRC, freenode, #hurd, 2013-07-01
+
+ <youpi> braunr: btw, we are missing the symbol in mach/Versions
+ <braunr> youpi: what symbol ?
+ <youpi> so the libc-provided RPC stub is not available
+ <youpi> mach_printf
+ <youpi> -f
+ <braunr> it's a system calll
+ <braunr> not exported
+ <youpi> s/RPC/system call/
+ <braunr> that's expected
+ <youpi> libc does provide stubs for system calls too
+ <braunr> yes but not for this one
+ <youpi> I don't see why we wouldn't want to include it
+ <youpi> ?! it does
+ <braunr> it's temporary
+ <braunr> oh
+ <braunr> there must be automatic parsing during build
+ <youpi> sure
+ <braunr> nice
+
+ <braunr> youpi: if we're going to make this system call exported by glibc,
+ i should change its interface first
+ <braunr> it was meant as a very quick-and-dirty hack and directly accesses
+ the caller's address space without going through a special copy-from-user
+ function
+ <braunr> not very portable
diff --git a/microkernel/mach/history.mdwn b/microkernel/mach/history.mdwn
index 776bb1d7..c22ea739 100644
--- a/microkernel/mach/history.mdwn
+++ b/microkernel/mach/history.mdwn
@@ -78,3 +78,137 @@ IRC, freenode, #hurd, 2012-08-29:
https://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/About/About.html
<pavlx> can't be anymore
http://developer.apple.com/techpubs/macosx/Darwin/General/KernelProgramming/About/index.html
+
+IRC, freenode, #hurd, 2013-07-03:
+
+ *** natsukao (~natsukao@dynamic-adsl-94-37-184-109.clienti.tiscali.it) has
+ joined channel #hurd
+ <natsukao> hi
+ <natsukao> on 2012-08-29: i wrote a part of messages that then were posted
+ on http://www.gnu.org/software/hurd/microkernel/mach/history.html
+ <natsukao> i am sorry to inform you that apple computer cuèertino.inc, has
+ moved the URL: https://ssl.apple.com/science/profiles/cornell
+ <natsukao> and i have not found nothing on the source code of that page,
+ <natsukao> i used lftp without any success
+ <natsukao> and then wget, nothing to do
+ <natsukao> i have not found a copy cache of
+ https://ssl.apple.com/science/profiles/cornell
+ <natsukao> next time we save the documents and we provide to do our
+ archive/s
+ <natsukao> so that will be always available the infos
+ *** natsukao (~natsukao@dynamic-adsl-94-37-184-109.clienti.tiscali.it) is
+ now known as pavlx
+ <pavlx> happy hacking !!!!
+ <pavlx> "paolo del bene" <W3C_Freedom@riseup.net>
+ <pavlx> p.s: i'll turn back as soon as possible
+
+ <pavlx> i found the page of Darwin History, removed from apple compter
+ cupertino.inc
+ <pavlx> "Cached_ http___developer.apple.com_darwin_history.html"
+ <pavlx> the page http://developer.apple.com/darwin/history.html was moved
+ and now is available on:
+ <pavlx>
+ http://www.google.it/url?q=http://www.macmark.de/files/darwin_evolution.pdf%3FPHPSESSID%3D8b8683a81445f69d510734baa13aabfc&sa=U&ei=wMzTUb-NBIeFOOL4gNgE&ved=0CCQQFjAD&usg=AFQjCNFlLwC24nB5t14VUmedK4EmeE7Ohw
+ <pavlx> or simply: http://www.macmark.de/files/darwin_evolution.pdf
+ <pavlx> slides on: "Travel - Computer Science and Software Engineering"
+ <pavlx> www.csse.uwa.edu.au/~peterj/personal/PDFs/WWDC2004-6UP.pdf
+ <pavlx> about apple computer cupertino.inc, but there are many interesting
+ news
+ <teythoon> pavlx: uh, lot's of marketing noise from apple >,<
+ <pavlx> i found better material just now:
+ http://www.pcs.cnu.edu/~mzhang/CPSC450_550/2003CaseStudy/Mach_Presentation_DavidRamsey.ppt
+ <pavlx> teythoon, sorry, i turn back to sleep, see you later, paolo
+ W3C_Freedom@riseup.net
+ <pavlx> i'll charge of that page only things dedicated to GNU/HURD, but
+ slides are not mine, i found on internet
+ <teythoon> pavlx: sure, I didn't ment to offend you in any way
+
+IRC, freenode, #hurd, 2013-07-04:
+
+ <pavlx> there are few problems:
+ <pavlx> http://www.gnu.org/software/hurd/microkernel/mach/history.html
+ <pavlx> on the page GrantBow wrote: Apple's Macintosh OSX (OS 10.x) is
+ based on Darwin. "Darwin uses a monolithic kernel based on ?FreeBSD 4.4
+ and the OSF/mk Mach 3." Darwin also has a Kernel Programming Book.
+ <pavlx> the link to Darwin was moved, is not anymore
+ http://www.apple.com/macosx/technologies/darwin.html
+ <pavlx> then it's not FreeBSD 4.4 but BSD
+ <pavlx> and the link to Kernel Programming was moved is not
+ http://developer.apple.com/techpubs/macosx/Darwin/General/KernelProgramming/About/index.html
+ but
+ https://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/About/About.html
+ <pavlx> apple has moved the URL:
+ https://ssl.apple.com/science/profiles/cornell
+ <pavlx> apple has moved the URL:
+ http://www.apple.com/macosx/technologies/darwin.html
+ <pavlx> so on the website you can left few things about my old post:
+ <pavlx> from IRC, freenode, #hurd, 2012-08-29: needs to remove
+ <pavlx> http://dpaste.com/1286266/
+ <pavlx> the new one will be: http://pastebin.com/yMm8tcTN
+ IRC, freenode, #hurd, 2013-07-04:
+
+ <pavlx> was moved the page from apple.com about darwin kernel programming
+ as described on the https://www.gnu.org/software/hurd/microkernel/mach/history.html
+
+ <pavlx> the link to Kernel Programming:
+ https://developer.apple.com/library/mac/#documentation/Darwin/Conceptual/KernelProgramming/About/About.html
+ <pavlx> (anyway i searching with any key the things moved from apple)
+ <pavlx> about Darwin type http://apple.com/darwin
+ <pavlx> on the right side, towards the end of the website it says: Darwin
+ Technologies
+ <pavlx> click on it, or copy the URL in an other tab of your own browser,
+ and read:
+ https://developer.apple.com/library/mac/#documentation/MacOSX/Conceptual/OSX_Technology_Overview/SystemTechnology/SystemTechnology.html
+ <pavlx> and something is related to Darwin
+ <pavlx> and again : http://pastebin.com/DHkJDxy8
+ # Mac OS X Server
+
+ ... This kernel, known as Darwin, provides a stable, high-performance platform
+ for developing groundbreaking applications and system technologies. ...
+ http://www.apple.com/server/docs/MacOSX_Server_TO_300195.pdf
+
+ # Mac OS X Server Command-Line Administration
+
+ Page 1. Mac OS X Server Command-Line Administration For Version 10.3
+ http://www.apple.com/server/docs/Command_Line.pdf
+
+ # Press Info - Apple “Open Sources” Rendezvous
+
+ ... Rendezvous is part of a broader open source release today from Apple at
+ http://developer.apple.com/darwin which includes the Darwin 6.0.1 ...
+ http://www.apple.com/pr/library/2002/09/25Apple-Open-Sources-Rendezvous.html
+
+ # Press Info - Apple Releases Darwin 1.0 Open Source
+
+ ... Apple Releases Darwin 1.0 Open Source. New ... modules. Darwin 1.0 gives
+ developers access to essential Mac OS X source code. ...
+ http://www.apple.com/pr/library/2000/04/05Apple-Releases-Darwin-1-0-Open-Source.html
+
+ # Press Info - Apple's Mac OS X to Ship on March 24
+
+ ... Mac OS X is built upon an incredibly stable, open source, UNIX based
+ foundation called Darwin and features true memory protection, preemptive ...
+ http://www.apple.com/pr/library/2001/01/09Apples-Mac-OS-X-to-Ship-on-March-24.html
+
+ # Press Info - Mac OS X “Gold Master” Released To ...
+
+ ... Mac OS X is built upon an incredibly stable, open source, UNIX
+ basedfoundation called Darwin and features true memory protection ...
+ http://www.apple.com/pr/library/2001/03/07Mac-OS-X-Gold-Master-Released-To-Manufacturing.html
+
+ * Press Info - Apple Announces Mac OS X “Jaguar” ...
+
+ ... As an active member of the Open Source community, Apple has distributed
+ Open Directory technology through the Darwin Open Source Project. ...
+ http://www.apple.com/pr/library/2002/07/17Apple-Announces-Mac-OS-X-Jaguar-Server-Worlds-Easiest-to-Manage-UNIX-Based-Server-Software.html
+ <pavlx> and:
+ http://lists.apple.com/archives/darwinos-users/2005/Apr/msg00021.html
+ <youpi> pavlx: it's hard to follow the changes you are talking
+ about. Perhaps you could simply edit these wiki pages?
+ <pavlx> anyway i am saying to you that i found a mailing list where are
+ availables the sources codes of darwin ppc-801 and x86
+ <pavlx> and as last thing mac os x 10.4
+ <braunr> pavlx: what's all this about ?
+ <pavlx> i am sorry, i did changes on the wiki
+ <pavlx> but after the preview and after to have saved, it show again the
+ old chat of 2012
diff --git a/microkernel/mach/mig.mdwn b/microkernel/mach/mig.mdwn
index 331b3bf4..d6340574 100644
--- a/microkernel/mach/mig.mdwn
+++ b/microkernel/mach/mig.mdwn
@@ -1,5 +1,5 @@
-[[!meta copyright="Copyright © 2001, 2002, 2003, 2006, 2007, 2008, 2010 Free
-Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2001, 2002, 2003, 2006, 2007, 2008, 2010, 2013
+Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -22,9 +22,9 @@ wait for a result on a newly created [[reply port|port]], decode return
arguments from the reply message (*demarshalling*, or *unmarshalling*) and pass
them to the client program. Similar actions are provided in the skeletons that
are linked to server programs.
-
MIG allows very precise semantics to be specified about what the arguments are
and how to be passed.
+It has its problems with [[structured_data]], however.
* [[Documentation]]
diff --git a/microkernel/mach/mig/structured_data.mdwn b/microkernel/mach/mig/structured_data.mdwn
new file mode 100644
index 00000000..1c8abe08
--- /dev/null
+++ b/microkernel/mach/mig/structured_data.mdwn
@@ -0,0 +1,119 @@
+[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_mig]]
+
+# IRC, freenode, #hurd, 2013-06-25
+
+ <teythoon> is there a nice way to get structured data through mig that I
+ haven't found yet?
+ <teythoon> say an array of string triples
+ <braunr> no
+ <teythoon> :/
+ <braunr> but you shouldn't need that
+ <teythoon> my use case is getting info about fs translators from init to
+ procfs
+
+[[community/gsoc/project_ideas/mtab]].
+
+ <teythoon> should I go for an iterator like interface instead?
+ <braunr> depends
+ <braunr> how many do you need ?
+ <braunr> you could go for a variable sized array too
+ <braunr> have a look at what already exists
+ <teythoon> records, maybe 10-15, depends on many fs translators are running
+ <braunr> a variable sized array is ok if the size isn't too big (and when i
+ say too big, i mean hundreds of MiB)
+ <braunr> an iterator is ok too if there aren't too many items
+ <braunr> you may want to combine both (i think that's what proc does)
+ <braunr> be aware that the maximum size of a message is limited to 512 MiB
+ <teythoon> yeah I saw the array[] of stuff stuff, but array[] of string_t
+ does not work, I guess b/c string_t is also an array
+ <teythoon> how would I send an array of variable length strings?
+ <braunr> i'm not sure you can
+ <braunr> or maybe out of line
+ <teythoon> somehow I expected mig to serialize arbitrary data structures,
+ maybe it's to old for that?
+ <teythoon> yeah, I read about uot of line, but that seems overkill
+ <braunr> it is old yes
+ <braunr> and not very user friendly in the end
+ <braunr> let me check
+ <teythoon> we could stuff json into mig...
+ <braunr> see proc_getallpids for example
+ <braunr> we could get rid of low level serialization altogether :p
+ <teythoon> hah, exactly what I was looking at
+ <braunr> (which is what i'll do in x15)
+ <braunr> type pidarray_t = array[] of pid_t;
+ <teythoon> but that is trivial b/c its array[] of pid_t
+ <braunr> and always have the server writing guide near you
+ <teythoon> yes
+ <braunr> well, make one big string and an array of lengths :p
+ <teythoon> thought about that and said to myself, there must be a better
+ way that I haven't found yet
+ <braunr> or one big string filled with real null-terminated c strings that
+ you keep parsing until you ate all input bytes
+ <braunr> i'm almost certain there isn't
+ <braunr> type string_t = c_string[1024]; /* XXX */
+ <teythoon> yes
+ <braunr> even that isn't really variable sized
+ <teythoon> you think anyone would object to me putting a json encoder in
+ /hurd/init? it is probably better than me at serializing stuff...
+ <braunr> try with mig anyway
+ <braunr> the less dependencies we have for core stuff, the simpler it is
+ <braunr> but i agree, mig is painful
+ <teythoon> would it be too hacky if I abused the argz functions? they do
+ exactly what I'd need
+
+
+## IRC, freenode, #hurd, 2013-06-26
+
+ <teythoon> there is https://code.google.com/p/protobuf-c/ and it has a rpc
+ mechanism and I believe one could plug arbitrary transports easily
+ <braunr> please don't think about it
+ <braunr> we really don't want to add another layer of serialization
+ <braunr> it's better to completely redesign mach ipc anyway
+ <braunr> and there is a project for that :p
+ <teythoon> ive seen x15
+ <teythoon> just food for thought
+ <braunr> i've studied google protocol buffers
+ <braunr> and fyi, no, it wouldn't be easy to plug arbitrary transports on
+ top of mach
+ <braunr> there is a lot of knowledge about mach ports in mig
+
+[[community/gsoc/project_ideas/mtab]].
+
+ <teythoon> but again I face the challenge of serializing a arbitrary sized
+ list of arbitrary sized strings
+ <braunr> yes
+ <teythoon> list of ports is easier ;) but I think its worthwile
+ <teythoon> so what about abusing argz* for this? you think it's too bad a
+ hack?
+ <braunr> no since it's in glibc
+ <teythoon> awesome :)
+ <braunr> but i don't remember the details well and i'm not sure the way you
+ use it is safe
+ <teythoon> yeah, I might have got the details wrong, I hadn't had the
+ chance to test it ;)
+
+ <braunr> about this dynamic size problem
+ <braunr> a "simple" varying size array should do
+ <braunr> you can easily put all your strings in there
+ <teythoon> seperated by 0?
+ <braunr> yes
+ <teythoon> that's exactly what the argz stuff does
+ <braunr> you'll get the size of the array anyway, and consume it until
+ there is no byte left
+ <braunr> good
+ <braunr> but be careful with this too
+ <braunr> since translators can be run by users, they somtimes can't be
+ trusted
+ <braunr> and even a translator running as root may behave badly
+ <braunr> so careful with parsing
+ <teythoon> noted
diff --git a/open_issues/64-bit_port.mdwn b/open_issues/64-bit_port.mdwn
index 66da44b9..b0c95612 100644
--- a/open_issues/64-bit_port.mdwn
+++ b/open_issues/64-bit_port.mdwn
@@ -1,4 +1,5 @@
-[[!meta copyright="Copyright © 2011, 2012 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2011, 2012, 2013 Free Software Foundation,
+Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -137,3 +138,20 @@ See also [[microkernel/mach/gnumach/memory_management]].
<braunr> hm actually no, it would require mcmodel=large
<braunr> hum, that's stupid, we can make the kernel run at -2g, and use 3g
up to the sign extension hole for the kernel map
+
+
+# IRC, freenode, #hurd, 2013-07-02
+
+In context of [[mondriaan_memory_protection]].
+
+ <xscript> BTW, it's not like I have an infinite amount of time for this,
+ but having 64-bit support would be valuable for me, so I might contribute
+ that back if it's not a too monumental task
+ <xscript> I saw some discussions about 32bit apps on top of 64bit mach, but
+ I'd like a full 64bit system
+ <xscript> any clues?
+ <xscript> I suppose the compiler support is all there already
+ <xscript> is MIG (and mach) the only piece missing?
+ <braunr> the problem is the interfaces themselves
+ <braunr> type widths
+ <braunr> as passed between userspace and kernel
diff --git a/open_issues/anatomy_of_a_hurd_system.mdwn b/open_issues/anatomy_of_a_hurd_system.mdwn
index 11a1f754..75a62535 100644
--- a/open_issues/anatomy_of_a_hurd_system.mdwn
+++ b/open_issues/anatomy_of_a_hurd_system.mdwn
@@ -323,7 +323,9 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l
<braunr> etc..
-# IRC, freenode, #hurd, 2012-12-06
+# Service Directory
+
+## IRC, freenode, #hurd, 2012-12-06
<spiderweb> what is the #1 feature that distinguished hurd from other
operating systems. the concept of translators. (will read more when I get
@@ -333,56 +335,7 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l
<braunr> and the VFS permissions to control access to those services
-# IRC, freenode, #hurd, 2012-12-10
-
- <spiderweb> I want to work on hurd, but I think I'm going to start with
- minix, I own the minix book 3rd ed. it seems like a good intro to
- operating systems in general. like I don't even know what a semaphore is
- yet.
- <braunr> well, enjoy learning :)
- <spiderweb> once I finish that book, what reading do you guys recommend?
- <spiderweb> other than the wiki
- <braunr> i wouldn't recommend starting with a book that focuses on one
- operating system anyway
- <braunr> you tend to think in terms of what is done in that specific
- implementation and compare everything else to that
- <braunr> tannenbaum is not only the main author or minix, but also the one
- of the book http://en.wikipedia.org/wiki/Modern_Operating_Systems
- <braunr>
- http://en.wikipedia.org/wiki/List_of_important_publications_in_computer_science#Operating_systems
- should be a pretty good list :)
-
-
-# IRC, freenode, #hurd, 2013-03-12
-
- <mjjc> i have a question regarding ipc in hurd. if a task is created, does
- it contain any default port rights in its space? i am trying to deduce
- how one calls dir_lookup() on the root translator in glibc's open().
- <kilobug> mjjc: yes, there are some default port rights, but I don't
- remember the details :/
- <mjjc> kilobug: do you know where i should search for details?
- <kilobug> mjjc: hum either in the Hurd's hacking guide
- https://www.gnu.org/software/hurd/hacking-guide/ or directly in the
- source code of exec server/libc I would say, or just ask again the
- question here later on to see if someone else has more information
- <mjjc> ok, thanks
- <pinotree> there's also rpctrace to, as the name says, trace all the rpc's
- executed
- <braunr> some ports are introduced in new tasks, yes
- <braunr> see
- http://www.gnu.org/software/hurd/hacking-guide/hhg.html#The-main-function
- <braunr> and
- <braunr>
- http://www.gnu.org/software/hurd/gnumach-doc/Task-Special-Ports.html#Task-Special-Ports
- <mjjc> yes, the second link was just what i was looking for, thanks
- <braunr> the second is very general
- <braunr> also, the first applies to translators only
- <braunr> if you're looking for how to do it for a non-translator
- application, the answer is probably somewhere in glibc
- <braunr> _hurd_startup i'd guess
-
-
-# IRC, freenode, #hurd, 2013-05-23
+## IRC, freenode, #hurd, 2013-05-23
<gnu_srs> Hi, is there any efficient way to control which backed
translators are called via RPC with a user space program?
@@ -409,18 +362,7 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l
same as for regular files
<braunr> gnu_srs: this *must* be obvious for you to do any tricky work on
the hurd
- <gnu_srs> fsysopts /servers/socket/2 works by /1 gives Operation not
- supported.
-
-[[!taglink open_issue_hurd]].
-
- <braunr> ah right, some servers don't implement that
- <braunr> work around this by using showtrans
- <braunr> fsysopts asks the server itself how it's running, usually giving
- its command name and options
- <braunr> showtrans asks the parent how it starts a passive translator
- attached to the node
- <gnu_srs> Yes showtrans works :), thanks.
+
<gnu_srs> Anyway, if I create a test program calling io_stat I assume
S_io_stat in pflocal is called.
<gnu_srs> How to make the program call S_io_stat in pfinet instead?
@@ -447,6 +389,127 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l
implementation
+## IRC, freenode, #hurd, 2013-06-30
+
+ <hacklu> hi, what is the replacer of netname_check_in?
+
+ <hacklu> I want to ask another question. in my opinion, the rpc is the
+ mach's way, and the translator is the hurd's way. so somebody want to
+ lookup a service, it should not need to ask the mach kernel know about
+ this query. the hurd will take the control.
+ <hacklu> am I right?
+ <braunr> no
+ <braunr> that's nonsense
+ <braunr> service lookups has never been in mach
+ <braunr> first mach based systems used a service directory, whereas the
+ hurd uses the file system for that
+ <braunr> you still need mach to communicate with either of those
+ <hacklu> how to understand the term of service directory here?
+ <braunr> a server everyone knows
+ <braunr> which gives references to other servers
+ <braunr> usually, depending on the name
+ <braunr> e.g. name_lookup("net") -> port right to network server
+ <hacklu> is that people use netname_check_in to register service in the
+ past? now used libtrivfs?
+ <braunr> i don't know about netname_check_in
+ <braunr> old mach (not gnumach) documentation might mention this service
+ directory
+ <braunr> libtrivfs doesn't have much to do with that
+ <braunr> on the hurd, the equivalent is the file system
+ <hacklu> maybe that is outdate, I just found that exist old doc, and old
+ code which can't be build.
+ <braunr> every process knows /
+ <braunr> the file system is the service directory
+ <braunr> nodes refer to services
+ <hacklu> so the file system is the nameserver, any new service should
+ register in it before other can use
+ <braunr> and the file system is distributed, so looking up a service may
+ require several queries
+ <braunr> setting a translator is exactly that, registering a program to
+ service requests on a node
+ <braunr> the file system isn't one server though
+ <braunr> programs all know about /, but then, lookups are recursive
+ <braunr> e.g. if you have / and /home, and are looking for
+ /home/hacklu/.profile, you ask / which tells you about /home, and /home
+ will give you a right to /home/hacklu/.profile
+ <hacklu> even in the past, the mach don't provide name register service,
+ there must be an other server to provide this service?
+ <braunr> yes
+ <braunr> what's nonsense in your sentence is comparing RPCs and translators
+ <braunr> translators are merely servers attached to the file system, using
+ RPCs to communicate with the rest of the system
+ <hacklu> I know yet, the two just one thing.
+ <braunr> no
+ <braunr> two things :p
+ <braunr> completely different and unrelated except for one using the other
+ <hacklu> ah, just one used aonther one.
+ <hacklu> is exist anyway to anounce service except settrans with file node?
+ <braunr> more or less
+ <braunr> tasks can have special ports
+ <braunr> that's how one task knows about / for example
+ <braunr> at task creation, a right to / is inserted in the new task
+ <hacklu> I think this is also a file node way.
+ <braunr> no
+ <braunr> if i'm right, auth is referenced the same way
+ <braunr> and there is no node for auth
+ <hacklu> how the user get the port of auth with node?
+ <braunr> it's given when a task is created
+ <hacklu> pre-set in the creation of one task?
+ <braunr> i'm unconfortable with "pre-set"
+ <braunr> inserted at creation time
+ <braunr> auth is started very early
+ <braunr> then tasks are given a reference to it
+
+
+# IRC, freenode, #hurd, 2012-12-10
+
+ <spiderweb> I want to work on hurd, but I think I'm going to start with
+ minix, I own the minix book 3rd ed. it seems like a good intro to
+ operating systems in general. like I don't even know what a semaphore is
+ yet.
+ <braunr> well, enjoy learning :)
+ <spiderweb> once I finish that book, what reading do you guys recommend?
+ <spiderweb> other than the wiki
+ <braunr> i wouldn't recommend starting with a book that focuses on one
+ operating system anyway
+ <braunr> you tend to think in terms of what is done in that specific
+ implementation and compare everything else to that
+ <braunr> tannenbaum is not only the main author or minix, but also the one
+ of the book http://en.wikipedia.org/wiki/Modern_Operating_Systems
+ <braunr>
+ http://en.wikipedia.org/wiki/List_of_important_publications_in_computer_science#Operating_systems
+ should be a pretty good list :)
+
+
+# IRC, freenode, #hurd, 2013-03-12
+
+ <mjjc> i have a question regarding ipc in hurd. if a task is created, does
+ it contain any default port rights in its space? i am trying to deduce
+ how one calls dir_lookup() on the root translator in glibc's open().
+ <kilobug> mjjc: yes, there are some default port rights, but I don't
+ remember the details :/
+ <mjjc> kilobug: do you know where i should search for details?
+ <kilobug> mjjc: hum either in the Hurd's hacking guide
+ https://www.gnu.org/software/hurd/hacking-guide/ or directly in the
+ source code of exec server/libc I would say, or just ask again the
+ question here later on to see if someone else has more information
+ <mjjc> ok, thanks
+ <pinotree> there's also rpctrace to, as the name says, trace all the rpc's
+ executed
+ <braunr> some ports are introduced in new tasks, yes
+ <braunr> see
+ http://www.gnu.org/software/hurd/hacking-guide/hhg.html#The-main-function
+ <braunr> and
+ <braunr>
+ http://www.gnu.org/software/hurd/gnumach-doc/Task-Special-Ports.html#Task-Special-Ports
+ <mjjc> yes, the second link was just what i was looking for, thanks
+ <braunr> the second is very general
+ <braunr> also, the first applies to translators only
+ <braunr> if you're looking for how to do it for a non-translator
+ application, the answer is probably somewhere in glibc
+ <braunr> _hurd_startup i'd guess
+
+
# IRC, freenode, #hurd, 2013-06-15
<damo22> ive been reading a little about exokernels or unikernels, and i
diff --git a/open_issues/arm_port.mdwn b/open_issues/arm_port.mdwn
index 8a2a037f..b07df939 100644
--- a/open_issues/arm_port.mdwn
+++ b/open_issues/arm_port.mdwn
@@ -273,4 +273,3 @@ architecture.
<slpz> braunr: OK, thanks. I'm interested on it, and didn't want to
duplicate efforts.
<braunr> little addition: it may have started, but we don't know about it
-
diff --git a/open_issues/code_analysis.mdwn b/open_issues/code_analysis.mdwn
index bdd2ae18..67798c6a 100644
--- a/open_issues/code_analysis.mdwn
+++ b/open_issues/code_analysis.mdwn
@@ -193,3 +193,17 @@ There is a [[!FF_project 276]][[!tag bounty]] on some of these tasks.
* [Trinity: A Linux kernel fuzz tester (and then
some)](http://www.socallinuxexpo.org/scale11x/presentations/trinity-linux-kernel-fuzz-tester-and-then-some),
Dave Jones, The Eleventh Annual Southern California Linux Expo, 2013.
+
+ * Mayhem, *an automatic bug finding system*
+
+ IRC, freenode, #hurd, 2013-06-29:
+
+ <teythoon> started reading the mayhem paper referenced here
+ http://lists.debian.org/debian-devel/2013/06/msg00720.html
+ <teythoon> that's nice work, they are doing symbolic execution of x86
+ binary code, that's effectively model checking with some specialized
+ formulas
+ <teythoon> (too bad the mayhem code isn't available, damn those
+ academic people keeping the good stuff to themselvs...)
+ <teythoon> (and I really think that's bad practice, how should anyone
+ reproduce their results? that's not how science works imho...)
diff --git a/open_issues/dde.mdwn b/open_issues/dde.mdwn
index 65d84886..76b80211 100644
--- a/open_issues/dde.mdwn
+++ b/open_issues/dde.mdwn
@@ -598,3 +598,47 @@ In context of [[libpthread]].
<braunr> hm, i haven't looked but, does someone know if virtio is included
in netdde ?
<youpi> braunr: nope, there's an underlying virtio layer needed before
+
+
+## IRC, freenode, #hurd, 2013-07-24
+
+ <teythoon> btw, I'd love to see libvirt support in hurd
+ <teythoon> I tried to hack up a dde based net translator
+ <teythoon> afaics they are very much like any other pci device, so the
+ infrastructure should be there
+ <teythoon> if anything I expect the libvirt stuff to be more easily
+ portable
+ <youpi> what do you mean by "a dde based net translator" ?
+ <youpi> ah, you mean virtio support in netdde ?
+ <teythoon> yes
+ <teythoon> virtio net is present in the kernel version we use for the dde
+ drivers
+ <teythoon> so I just copied the dde driver over, but I had no luck
+ compiling it
+ <youpi> ok, but what would be the benefice over e1000 & co?
+ <teythoon> any of the dde drivers btw
+ <teythoon> youpi: less overhead
+ <youpi> e1000 is already low overhead actually
+ <youpi> there are less and less differences in strategies for driving a
+ real board, and a virtual one
+ <youpi> we are seeing shared memory request buffer, dma, etc. in real
+ boards
+ <youpi> which ends up being almost exactly what virtio does :)
+ <youpi> ahci, for instance, really looks extremely like a virtio interface
+ <youpi> (I know, it's a disk, but that's the same idea, and I do know what
+ I'm talking about here :) )
+ <teythoon> that would actually be my next wish, a virtio disk driver, and
+ virt9p ;)
+ <braunr> on the other hand, i wouldn't spend much time on a virtio disk
+ driver for now
+ <braunr> the hurd as it is can't boot on a device that isn't managed by the
+ kernel
+ <braunr> we'd need to change the boot protocol
+ <teythoon> ok, I wasn't planning to, just wanted to see if I can easily
+ hack up the virtio-net translator
+ <braunr> well, as youpi pointed, there is little benefit to that as well
+ <braunr> but if that's what you find fun, help yourself :)
+ <teythoon> I didn't know that, I assumed there was some value to the virtio
+ stuff
+ <braunr> there is
+ <braunr> but relatively to other improvements, it's low
diff --git a/open_issues/gdb_signal_handler.mdwn b/open_issues/gdb_signal_handler.mdwn
new file mode 100644
index 00000000..3084f7e3
--- /dev/null
+++ b/open_issues/gdb_signal_handler.mdwn
@@ -0,0 +1,403 @@
+[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_gdb open_issue_glibc]]
+
+
+# IRC, freenode, #hurd, 2013-07-07
+
+ <zyg> Hi, I'm in GDB inside a handler for SIGHUP, after stepping out, gdb
+ will hang on instruction: <_hurd_sigstate_lock+88>: xchg
+ %edx,0x4(%eax)
+ <zyg> here is my signal test pasted: http://pastebin.com/U72qw3FC
+ #include <stdio.h>
+ #include <stdlib.h>
+ #include <signal.h>
+
+ void *
+ my_handler(int signal, void *info, void *context)
+ {
+ printf("got SIGHUP\n");
+ return NULL;
+ }
+
+ void
+ install_handler (int signal)
+ {
+ struct sigaction sa;
+ sa.sa_sigaction = my_handler;
+ sa.sa_flags = SA_SIGINFO;
+ sigemptyset(&sa.sa_mask);
+ sigaction(signal, &sa, NULL);
+ }
+
+ void test_sighup(void)
+ {
+ raise(SIGHUP);
+ }
+
+ int main(int argc, char **argv){
+ install_handler(SIGHUP);
+ test_sighup();
+ exit(1);
+ }
+ <braunr> zyg: thanks
+ <braunr> zyg: what is the problem exactly ?
+ <braunr> zyg: i mean, does it hand before attaching with gdb ?
+ <zyg> braunr: it doesn't hang if runned without gdb. I've pasted here when
+ I step out of the handler, and get to the hanging instruction:
+ http://pastebin.com/nUyCx6Wj
+ $ gdb --args a.out
+ GNU gdb (GDB) 7.6-debian
+ Copyright (C) 2013 Free Software Foundation, Inc.
+ License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
+ This is free software: you are free to change and redistribute it.
+ There is NO WARRANTY, to the extent permitted by law. Type "show copying"
+ and "show warranty" for details.
+ This GDB was configured as "i486-gnu".
+ For bug reporting instructions, please see:
+ <http://www.gnu.org/software/gdb/bugs/>...
+ Reading symbols from /home/shrek/a.out...(no debugging symbols found)...done.
+ (gdb)
+
+ (gdb) display/i $pc
+ (gdb) handle SIGHUP pass stop print
+ Signal Stop Print Pass to program Description
+ SIGHUP Yes Yes Yes Hangup
+ (gdb)
+
+ (gdb) run
+ Starting program: /home/shrek/a.out
+ [New Thread 3571.5]
+
+ Program received signal SIGHUP, Hangup.
+ 0x010548ec in mach_msg_trap ()
+ at /build/buildd-eglibc_2.17-6-hurd-i386-g946kE/eglibc-2.17/build-tree/hurd-i386-libc/mach/mach_msg_trap.S:2
+ 2 /build/buildd-eglibc_2.17-6-hurd-i386-g946kE/eglibc-2.17/build-tree/hurd-i386-libc/mach/mach_msg_trap.S: No such file or directory.
+ 1: x/i $pc
+ => 0x10548ec <mach_msg_trap+12>: ret
+ (gdb)
+
+ (gdb) si
+ 0x0804862d in my_handler ()
+ 1: x/i $pc
+ => 0x804862d <my_handler>: push %ebp
+ (gdb) x/20xi 0x804862d
+ => 0x804862d <my_handler>: push %ebp
+ 0x804862e <my_handler+1>: mov %esp,%ebp
+ 0x8048630 <my_handler+3>: sub $0x18,%esp
+ 0x8048633 <my_handler+6>: movl $0x8048750,(%esp)
+ 0x804863a <my_handler+13>: call 0x8048500 <puts@plt>
+ 0x804863f <my_handler+18>: mov $0x0,%eax
+ 0x8048644 <my_handler+23>: leave
+ 0x8048645 <my_handler+24>: ret
+ 0x8048646 <install_handler>: push %ebp
+ 0x8048647 <install_handler+1>: mov %esp,%ebp
+ 0x8048649 <install_handler+3>: sub $0x28,%esp
+ 0x804864c <install_handler+6>: movl $0x804862d,-0x14(%ebp)
+ 0x8048653 <install_handler+13>: movl $0x40,-0xc(%ebp)
+ 0x804865a <install_handler+20>: lea -0x14(%ebp),%eax
+ 0x804865d <install_handler+23>: add $0x4,%eax
+ 0x8048660 <install_handler+26>: mov %eax,(%esp)
+ 0x8048663 <install_handler+29>: call 0x80484d0 <sigemptyset@plt>
+ 0x8048668 <install_handler+34>: movl $0x0,0x8(%esp)
+ 0x8048670 <install_handler+42>: lea -0x14(%ebp),%eax
+ 0x8048673 <install_handler+45>: mov %eax,0x4(%esp)
+ (gdb)
+
+ (gdb) break *0x804863f
+ Breakpoint 1 at 0x804863f
+ (gdb) c
+ Continuing.
+ got SIGHUP
+
+ Breakpoint 1, 0x0804863f in my_handler ()
+ 1: x/i $pc
+ => 0x804863f <my_handler+18>: mov $0x0,%eax
+ (gdb)
+
+ (gdb) si
+ 0x08048644 in my_handler ()
+ 1: x/i $pc
+ => 0x8048644 <my_handler+23>: leave
+ (gdb)
+ 0x08048645 in my_handler ()
+ 1: x/i $pc
+ => 0x8048645 <my_handler+24>: ret
+ (gdb)
+ 0x010708b2 in trampoline () from /lib/i386-gnu/libc.so.0.3
+ 1: x/i $pc
+ => 0x10708b2 <trampoline+2>: add $0xc,%esp
+ (gdb)
+ 0x010708b5 in trampoline () from /lib/i386-gnu/libc.so.0.3
+ 1: x/i $pc
+ => 0x10708b5 <trampoline+5>: ret
+ (gdb)
+ __sigreturn (scp=0x102988c) at ../sysdeps/mach/hurd/i386/sigreturn.c:30
+ 30 ../sysdeps/mach/hurd/i386/sigreturn.c: No such file or directory.
+ 1: x/i $pc
+ => 0x1096340 <__sigreturn>: push %ebp
+ (gdb)
+ 0x01096341 30 in ../sysdeps/mach/hurd/i386/sigreturn.c
+ 1: x/i $pc
+ => 0x1096341 <__sigreturn+1>: push %edi
+ (gdb)
+ 0x01096342 30 in ../sysdeps/mach/hurd/i386/sigreturn.c
+ 1: x/i $pc
+ => 0x1096342 <__sigreturn+2>: push %esi
+ (gdb)
+ 0x01096343 30 in ../sysdeps/mach/hurd/i386/sigreturn.c
+ 1: x/i $pc
+ => 0x1096343 <__sigreturn+3>: push %ebx
+ (gdb)
+ 0x01096344 30 in ../sysdeps/mach/hurd/i386/sigreturn.c
+ 1: x/i $pc
+ => 0x1096344 <__sigreturn+4>: sub $0x2c,%esp
+ (gdb)
+ 0x01096347 30 in ../sysdeps/mach/hurd/i386/sigreturn.c
+ 1: x/i $pc
+ => 0x1096347 <__sigreturn+7>: mov 0x40(%esp),%esi
+ (gdb)
+ 0x0109634b 30 in ../sysdeps/mach/hurd/i386/sigreturn.c
+ 1: x/i $pc
+ => 0x109634b <__sigreturn+11>: call 0x11a0609 <__x86.get_pc_thunk.bx>
+ (gdb)
+ 0x011a0609 in __x86.get_pc_thunk.bx () from /lib/i386-gnu/libc.so.0.3
+ 1: x/i $pc
+ => 0x11a0609 <__x86.get_pc_thunk.bx>: mov (%esp),%ebx
+ (gdb)
+ 0x011a060c in __x86.get_pc_thunk.bx () from /lib/i386-gnu/libc.so.0.3
+ 1: x/i $pc
+ => 0x11a060c <__x86.get_pc_thunk.bx+3>: ret
+ (gdb)
+ 0x01096350 in __sigreturn (scp=0x102988c) at ../sysdeps/mach/hurd/i386/sigreturn.c:30
+ 30 in ../sysdeps/mach/hurd/i386/sigreturn.c
+ 1: x/i $pc
+ => 0x1096350 <__sigreturn+16>: add $0x15ccb0,%ebx
+ (gdb)
+ 35 in ../sysdeps/mach/hurd/i386/sigreturn.c
+ 1: x/i $pc
+ => 0x1096356 <__sigreturn+22>: test %esi,%esi
+ (gdb)
+ 0x01096358 35 in ../sysdeps/mach/hurd/i386/sigreturn.c
+ 1: x/i $pc
+ => 0x1096358 <__sigreturn+24>: je 0x10964f0 <__sigreturn+432>
+ (gdb)
+ 0x0109635e 35 in ../sysdeps/mach/hurd/i386/sigreturn.c
+ 1: x/i $pc
+ => 0x109635e <__sigreturn+30>: testl $0x10100,0x4(%esi)
+ (gdb)
+ 0x01096365 35 in ../sysdeps/mach/hurd/i386/sigreturn.c
+ 1: x/i $pc
+ => 0x1096365 <__sigreturn+37>: jne 0x10964f0 <__sigreturn+432>
+ (gdb)
+ __hurd_threadvar_location_from_sp (__sp=<optimized out>, __index=<optimized out>) at ../hurd/hurd/threadvar.h:94
+ 94 ../hurd/hurd/threadvar.h: No such file or directory.
+ 1: x/i $pc
+ => 0x109636b <__sigreturn+43>: mov -0x38(%ebx),%ebp
+ (gdb)
+ __hurd_threadvar_location (__index=_HURD_THREADVAR_SIGSTATE) at ../hurd/hurd/threadvar.h:116
+ 116 in ../hurd/hurd/threadvar.h
+ 1: x/i $pc
+ => 0x1096371 <__sigreturn+49>: mov %esp,%edx
+ (gdb)
+ __hurd_threadvar_location_from_sp (__sp=0x1029848, __index=_HURD_THREADVAR_SIGSTATE) at ../hurd/hurd/threadvar.h:94
+ 94 in ../hurd/hurd/threadvar.h
+ 1: x/i $pc
+ => 0x1096373 <__sigreturn+51>: cmp 0x0(%ebp),%esp
+ (gdb)
+ 0x01096376 94 in ../hurd/hurd/threadvar.h
+ 1: x/i $pc
+ => 0x1096376 <__sigreturn+54>: jae 0x10964d0 <__sigreturn+400>
+ (gdb)
+ 0x0109637c 94 in ../hurd/hurd/threadvar.h
+ 1: x/i $pc
+ => 0x109637c <__sigreturn+60>: mov -0x15c(%ebx),%eax
+ (gdb)
+ 0x01096382 94 in ../hurd/hurd/threadvar.h
+ 1: x/i $pc
+ => 0x1096382 <__sigreturn+66>: and (%eax),%edx
+ (gdb)
+ 0x01096384 94 in ../hurd/hurd/threadvar.h
+ 1: x/i $pc
+ => 0x1096384 <__sigreturn+68>: mov -0x90(%ebx),%eax
+ (gdb)
+ 0x0109638a 94 in ../hurd/hurd/threadvar.h
+ 1: x/i $pc
+ => 0x109638a <__sigreturn+74>: add (%eax),%edx
+ (gdb)
+ _hurd_self_sigstate () at ../hurd/hurd/signal.h:165
+ 165 ../hurd/hurd/signal.h: No such file or directory.
+ 1: x/i $pc
+ => 0x109638c <__sigreturn+76>: mov 0x8(%edx),%edi
+ (gdb)
+ 0x0109638f 165 in ../hurd/hurd/signal.h
+ 1: x/i $pc
+ => 0x109638f <__sigreturn+79>: test %edi,%edi
+ (gdb)
+ 0x01096391 165 in ../hurd/hurd/signal.h
+ 1: x/i $pc
+ => 0x1096391 <__sigreturn+81>: je 0x1096598 <__sigreturn+600>
+ (gdb)
+ __sigreturn (scp=0x102988c) at ../sysdeps/mach/hurd/i386/sigreturn.c:42
+ 42 ../sysdeps/mach/hurd/i386/sigreturn.c: No such file or directory.
+ 1: x/i $pc
+ => 0x1096397 <__sigreturn+87>: mov %edi,(%esp)
+ (gdb)
+ 0x0109639a 42 in ../sysdeps/mach/hurd/i386/sigreturn.c
+ 1: x/i $pc
+ => 0x109639a <__sigreturn+90>: call 0x1051d70 <_hurd_sigstate_lock@plt>
+ (gdb)
+ 0x01051d70 in _hurd_sigstate_lock@plt () from /lib/i386-gnu/libc.so.0.3
+ 1: x/i $pc
+ => 0x1051d70 <_hurd_sigstate_lock@plt>: jmp *0x864(%ebx)
+ (gdb)
+ _hurd_sigstate_lock (ss=ss@entry=0x1244008) at hurdsig.c:170
+ 170 hurdsig.c: No such file or directory.
+ 1: x/i $pc
+ => 0x106bb90 <_hurd_sigstate_lock>: sub $0x1c,%esp
+ (gdb)
+ 0x0106bb93 170 in hurdsig.c
+ 1: x/i $pc
+ => 0x106bb93 <_hurd_sigstate_lock+3>: mov %ebx,0x14(%esp)
+ (gdb)
+ 0x0106bb97 170 in hurdsig.c
+ 1: x/i $pc
+ => 0x106bb97 <_hurd_sigstate_lock+7>: call 0x11a0609 <__x86.get_pc_thunk.bx>
+ (gdb)
+ 0x011a0609 in __x86.get_pc_thunk.bx () from /lib/i386-gnu/libc.so.0.3
+ 1: x/i $pc
+ => 0x11a0609 <__x86.get_pc_thunk.bx>: mov (%esp),%ebx
+ (gdb)
+ 0x011a060c in __x86.get_pc_thunk.bx () from /lib/i386-gnu/libc.so.0.3
+ 1: x/i $pc
+ => 0x11a060c <__x86.get_pc_thunk.bx+3>: ret
+ (gdb)
+ 0x0106bb9c in _hurd_sigstate_lock (ss=ss@entry=0x1244008) at hurdsig.c:170
+ 170 in hurdsig.c
+ 1: x/i $pc
+ => 0x106bb9c <_hurd_sigstate_lock+12>: add $0x187464,%ebx
+ (gdb)
+ 0x0106bba2 170 in hurdsig.c
+ 1: x/i $pc
+ => 0x106bba2 <_hurd_sigstate_lock+18>: mov %esi,0x18(%esp)
+ (gdb)
+ 170 in hurdsig.c
+ 1: x/i $pc
+ => 0x106bba6 <_hurd_sigstate_lock+22>: mov 0x20(%esp),%esi
+ (gdb)
+ sigstate_is_global_rcv (ss=0x1244008) at hurdsig.c:162
+ 162 in hurdsig.c
+ 1: x/i $pc
+ => 0x106bbaa <_hurd_sigstate_lock+26>: lea 0x57c0(%ebx),%eax
+ (gdb)
+ 0x0106bbb0 162 in hurdsig.c
+ 1: x/i $pc
+ => 0x106bbb0 <_hurd_sigstate_lock+32>: mov (%eax),%eax
+ (gdb)
+ 163 in hurdsig.c
+ 1: x/i $pc
+ => 0x106bbb2 <_hurd_sigstate_lock+34>: test %eax,%eax
+ (gdb)
+ 0x0106bbb4 163 in hurdsig.c
+ 1: x/i $pc
+ => 0x106bbb4 <_hurd_sigstate_lock+36>: je 0x106bbbc <_hurd_sigstate_lock+44>
+ (gdb)
+ 0x0106bbb6 163 in hurdsig.c
+ 1: x/i $pc
+ => 0x106bbb6 <_hurd_sigstate_lock+38>: cmpl $0x1,0x18(%esi)
+ (gdb)
+ 0x0106bbba 163 in hurdsig.c
+ 1: x/i $pc
+ => 0x106bbba <_hurd_sigstate_lock+42>: je 0x106bbe0 <_hurd_sigstate_lock+80>
+ (gdb)
+ _hurd_sigstate_lock (ss=ss@entry=0x1244008) at hurdsig.c:172
+ 172 in hurdsig.c
+ 1: x/i $pc
+ => 0x106bbe0 <_hurd_sigstate_lock+80>: lea 0x4(%eax),%ecx
+ (gdb)
+ __spin_try_lock (__lock=0x124480c) at ../sysdeps/mach/i386/machine-lock.h:59
+ 59 ../sysdeps/mach/i386/machine-lock.h: No such file or directory.
+ 1: x/i $pc
+ => 0x106bbe3 <_hurd_sigstate_lock+83>: mov $0x1,%edx
+ (gdb)
+ 0x0106bbe8 59 in ../sysdeps/mach/i386/machine-lock.h
+ 1: x/i $pc
+ => 0x106bbe8 <_hurd_sigstate_lock+88>: xchg %edx,0x4(%eax)
+ (gdb)
+ <braunr> zyg: i don't get what you mean
+ <braunr> are you starting it with gdb ?
+ <zyg> braunr: yes: "gdb --args a.out"
+ <braunr> ok
+ <braunr> can't reproduce it
+ <braunr> i get "Program received signal SIGHUP, Hangup.
+ <braunr> "
+ <braunr> then continue, then the program has exited
+ <zyg> braunr: do you run it in gdb or without?
+ <zyg> braunr: Ah "Program received signal SIGHUP, Hangup." is from
+ gdb.. try issue continue, not sure why gdb stops at SIGHUP (default?).
+ <braunr> 10:34 < braunr> then continue, then the program has exited
+ <braunr> gdb stops at signals
+ <zyg> braunr: yes, try repeating that, but instead of continue, just issue
+ "si"
+ <zyg> braunr: sorry.. you would need to remove that printf/fprintf, else it
+ gets too long. That's why I put a breakpoint.
+ <braunr> a breakpoint ?
+ <braunr> on the signal handler ?
+ <zyg> braunr: yes, put a break after having entered the handler. Or edit
+ the pasted C code an remove that printf("got SIGHUP\n");
+ <braunr> i'm not sure that's correctly supported
+ <braunr> and i can see why glibc would deadlock on the sigstate lock
+ <braunr> don't do that :p
+ <zyg> braunr: why does it deadlock?
+ <braunr> because both the signal handler and messages from gdb will cause
+ common code paths to be taken
+ <zyg> braunr: oh.. when I step instruction I'm inside an SIGTRAP handler
+ probably?
+ <braunr> possible
+ <braunr> i don't know the details but that's the kind of things i expect
+ <braunr> and signals on the hurd are definitely buggy
+ <braunr> i don't know if we support nesting them
+ <braunr> i'd say we don't
+ <zyg> braunr: I'll try to put a break beyond that xchg and continue
+ <braunr> xhcg is probably the sigstate spinlock
+ <braunr> xchg*
+ <braunr> you'd need to reach the unlock instruction, which is probably
+ executed once the handler has finished running
+ <zyg> braunr: yes :) ... one instruction beyond didn't help
+ <zyg> braunr: thanks alot, putting a break in __sigreturn, after that
+ function has called _hurd_sigstate_unlock@plt works!
+ <braunr> works ?
+ <braunr> what did you want to do ?
+ <zyg> braunr: I want to trace user code inside the signal handler, also how
+ we enter and how we leave.
+ <braunr> well you can't do that inside, so no it doesn't work for you :/
+ <braunr> but that's a start i guess
+ <zyg> braunr: I seem to do most normal things inside the handler,
+ step-instruction and put breaks.
+ <braunr> ?
+ <braunr> i thought that's what made the program deadlock
+ <zyg> braunr: as you said earlier, the deadlock came when i "step
+ instruction" into the area between _hurd_sigstate_lock and
+ _hurd_sigstate_unlock. Otherwise I havn't had any issues.
+ <braunr> but isn't the sigstate locked during the handler execution ?
+ <zyg> braunr: no it locks and unlocks in __sigreturn which is done when
+ leaving the handler.
+ <braunr> than how could it deadlock on handler entry ?
+ <braunr> or perhaps the fact your handler was empty made the entry point
+ directly reach __sigreturn
+ <braunr> hm no i don't buy it
+ <braunr> the sigstate must also be locked on entry
+ <zyg> braunr: there was never any problem with entering
+ <braunr> then describe the problem with more details please
+ <braunr> ah sorry
+ <zyg> braunr: are you sure? there is minimal user-code run before the
+ signal is going into the handler.
+ <braunr> you "step out of the handler"
diff --git a/open_issues/glibc/0.4.mdwn b/open_issues/glibc/0.4.mdwn
index ceb5ea21..f864469d 100644
--- a/open_issues/glibc/0.4.mdwn
+++ b/open_issues/glibc/0.4.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -10,6 +10,8 @@ License|/fdl]]."]]"""]]
[[!tag open_issue_glibc open_issue_libpthread]]
+Things to consider doing when bumping the glibc SONAME.
+
# IRC, freenode, #hurd, 2012-12-14
diff --git a/open_issues/glibc/debian.mdwn b/open_issues/glibc/debian.mdwn
index 331632f3..9886ec98 100644
--- a/open_issues/glibc/debian.mdwn
+++ b/open_issues/glibc/debian.mdwn
@@ -62,3 +62,70 @@ apter applying patches.
If the Debian symbol versioning file is not up to date and the build of Debian
packages fails due to this, putting `DPKG_GENSYMBOLS_CHECK_LEVEL=0` in the
environment \`\`helps''; see `man dpkg-gensymbols`.
+
+
+# IRC, freenode, #hurd, 2013-07-01
+
+ <braunr> something seems to have changed with regard to patch handling in
+ eglibc 2.17
+ <braunr> pinotree: when i add a patch to series and use dpkg-buildpackage,
+ i'm told there are local modifications and the build stops :/
+ <braunr> any idea what i'm doing wrong ?
+ <pinotree> which steps do you do?
+ <braunr> i extract the sources, copy the patch to debian/patches/hurd-i386,
+ add the appropriate line to debian/patches/series, call dch -i, then
+ dpkg-buildpackage
+ <pinotree> eglibc is a "3.0 (quilt)" format source package
+ <pinotree> this means its default patches are in a quilt-style system, and
+ they are applied on extraction
+ <braunr> ok
+ <braunr> and it can't detect new patches ?
+ <pinotree> so if you add a new patch to the global serie, you have to push
+ it manually
+ <braunr> i have to revert them all ?
+ <braunr> ok
+ <braunr> how do i do that ?
+ <pinotree> quilt push -a
+ <braunr> ok
+ <braunr> thanks
+ <pinotree> remember to do that before starting the build, since the rest
+ assumes the quilt-style patches are fully applied
+ <bddebian> No push applies them, quilt pop -a reverts them
+ <pinotree> yeah, and he has to push the new over the dpkg-applied ones
+ <bddebian> Oh, aye
+ <braunr> does quilt change series ?
+ <pinotree> no
+ <braunr> ok
+ <pinotree> i mean, some commands do that
+ <braunr> so i do everything i did, with an additional push, right ?
+ <pinotree> ok, screw me, i didn't get your question above :P
+ <braunr> does that change your answer ?
+ <pinotree> <braunr> does quilt change series ?
+ <braunr> yes
+ <pinotree> if you import or create a new patch, it changes series indeed
+ <braunr> ok
+ <pinotree> push or pop of patches does not
+ <braunr> i'm doing it wron
+ <braunr> g
+ <pinotree> btw, in a quilt patch stack you can easily import a new patch
+ using the import command
+ <pinotree> so for example you could do
+ <pinotree> apt-get source eglibc # or get it somehow else
+ <pinotree> cd eglibc-*
+ <pinotree> quilt import /location/of/my/patch
+ <pinotree> quilt push # now your patch is applied
+ <braunr> ah thanks
+ <pinotree> dpkg-buildpackage as usual
+ <braunr> that's what i was looking for
+ <bddebian> quilt new adds a new entry in series
+ <pinotree> y
+ <bddebian> or import, aye
+ <pinotree> braunr: if you want to learn quilt, a very good doc is its own,
+ eg /usr/share/doc/quilt/quilt.txt.gz
+ * bddebian has never actually used import
+ <braunr> ok
+ <pinotree> it is basically a simple stack of patches
+
+ <youpi> braunr: yes, patch handling is a bit different
+ <youpi> the arch-independant patches are applied by dpkg-source -x
+ <youpi> and the arch-dependent patches are applied during build
diff --git a/open_issues/glibc/debian/experimental.mdwn b/open_issues/glibc/debian/experimental.mdwn
index 8d117e99..5168479d 100644
--- a/open_issues/glibc/debian/experimental.mdwn
+++ b/open_issues/glibc/debian/experimental.mdwn
@@ -11,6 +11,7 @@ License|/fdl]]."]]"""]]
[[!tag open_issue_glibc]]
Issues with the current 2.17 version of glibc/EGLIBC in Debian experimental.
+Now in unstable.
# IRC, OFTC, #debian-hurd, 2013-03-14
@@ -113,3 +114,62 @@ Issues with the current 2.17 version of glibc/EGLIBC in Debian experimental.
eventually? (Some experimental package(s), but which?)
<youpi> that was libc0.3 packages
<youpi> which are indeed known to break the network
+
+
+# IRC, freenode, #hurd, 2013-06-18
+
+ <braunr> root@darnassus:~# dpkg-reconfigure locales
+ <braunr> Generating locales (this might take a
+ while)... en_US.UTF-8...Segmentation fault
+ <braunr> is it known ?
+ <youpi> uh, no
+
+
+## IRC, OFTC, #debian-hurd, 2013-06-19
+
+ <pinotree> btw i saw too the segmentation fault when generating locales
+
+
+# IRC, OFTC, #debian-hurd, 2013-06-20
+
+ <youpi> damn
+ <youpi> hang at ext2fs boot
+ <youpi> static linking issue, clearly
+
+
+## IRC, freenode, #hurd, 2013-06-30
+
+ <youpi> Mmm
+ <youpi> __access ("/etc/ld.so.nohwcap", F_OK) at startup of ext2fs
+ <youpi> deemed to fail....
+ <pinotree> when does that happen?
+ <youpi> at hwcap initialization
+ <youpi> at least that's were ext2fs.static linked against libc 2.17 hangs
+ at startup
+ <youpi> and this is indeed a very good culprit :)
+ <pinotree> ah, a debian patch
+ <youpi> does anybody know a quick way to know whether one is the / ext2fs ?
+ :)
+ <pinotree> isn't the root fs given a special port?
+ <youpi> I was thinking about something like this, yes
+ <youpi> ok, boots
+ <youpi> I'll build a 8~0 that includes the fix
+ <youpi> so people can easily build the hurd package
+ <youpi> Mmm, no, the bootstrap port is also NULL for normally-started
+ processes :/
+ <youpi> I don't understand why
+ <youpi> ah, only translators get a bootstrap port :/
+ <youpi> perhaps CRDIR then
+ <youpi> (which makes a lot of sense)
+
+
+## IRC, freenode, #hurd, 2013-07-01
+
+ <braunr> youpi: what is local-no-bootstrap-fs-access.diff supposed to fix ?
+ <youpi> ext2fs.static linked againt debian glibc 2.17
+ <youpi> well, as long as you don't build & use ext2fs.static with it...
+ <braunr> that's thing, i want to :)
+ <braunr> +the
+ <youpi> I'd warmly welcome a way to detect whether being the / translator
+ process btw
+ <youpi> it seems far from trivial
diff --git a/open_issues/glibc/t/tls-threadvar.mdwn b/open_issues/glibc/t/tls-threadvar.mdwn
index 105a07c7..609d866b 100644
--- a/open_issues/glibc/t/tls-threadvar.mdwn
+++ b/open_issues/glibc/t/tls-threadvar.mdwn
@@ -64,3 +64,15 @@ dropped altogether, and `__thread` directly be used in glibc.
<youpi> I saw the mails, but didn't investigate at all
[[!message-id "878vdyqht3.fsf@kepler.schwinge.homeip.net"]].
+
+
+# IRC, freenode, #hurd, 2013-07-08
+
+ <youpi> tschwinge: apparently there were a lot of changes missing in the
+ threadvars branch I had commited long time ago
+ <youpi> I'm gathering things
+ <tschwinge> youpi: t/tls-threadvar you mean?
+ <youpi> yes
+ <youpi> tschwinge: yes, there were a lot other occurences of threadvars
+ stuff in various places
+ <youpi> I'm building libc again, and will see what issue would remain
diff --git a/open_issues/libc_variant_selection.mdwn b/open_issues/libc_variant_selection.mdwn
index afcd9ae0..71370a43 100644
--- a/open_issues/libc_variant_selection.mdwn
+++ b/open_issues/libc_variant_selection.mdwn
@@ -1,4 +1,5 @@
-[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2010, 2011, 2013 Free Software Foundation,
+Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -28,6 +29,28 @@ On Thu, Oct 07, 2010 at 11:22:46AM +0200, Samuel Thibault wrote:
> Yes, you need to copy it by hand. Same for libc0.3-i686, we just need to
> steal the cpuid code from the kfreebsd port of glibc.
+
+# IRC, freenode, #hurd, 2013-06-30
+
+ <pinotree> other than that, the hwcap system is not working for us yet,
+ right?
+ <youpi> no but we'd like to use e.g. cpuid for that
+ <youpi> like kfreebsd does
+ <pinotree> do they use cpuid for that?
+ <pinotree> i kind of lost myself in glibc's loading internals, trying to
+ find out where the hwcap bits come from
+ <youpi> on linux it comes from the kernel
+ <youpi> on kfreebsd aiui they use cpuid to figure it out from the process
+ itself
+ <pinotree> do you have any pointer to the kfreebsd way? iirc i had a look
+ in their sysdeps, but found nothing related to that
+ <youpi> it's in local-sysdeps.diff aiui
+ <youpi> +dl_platform_kfreebsd_i386_init
+ <youpi> which fills dl_hwcap
+ <youpi> called at _dl_sysdep_start
+ <pinotree> interesting
+
+
---
Having working CPUID code inside [[glibc]] is also a prerequisite for proper
diff --git a/open_issues/libnetfs_argument_parsing.mdwn b/open_issues/libnetfs_argument_parsing.mdwn
new file mode 100644
index 00000000..e1e0d794
--- /dev/null
+++ b/open_issues/libnetfs_argument_parsing.mdwn
@@ -0,0 +1,62 @@
+[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_hurd]]
+
+# IRC, freenode, #hurd, 2013-06-27
+
+ <teythoon> the arg parsing in libdiskfs and libnetfs differ :/
+ <teythoon> afaics libdiskfs gets it right, libnetfs does not
+ <pinotree> what do you mean?
+ <teythoon> wrt to *_std_{runtime,startup}_argp
+ <teythoon> see eg netfs.h
+ <teythoon> libdiskfs/opts-std-runtime.c:const struct argp
+ diskfs_std_runtime_argp =
+ <teythoon> libdiskfs/opts-std-runtime.c-{
+ <teythoon> libdiskfs/opts-std-runtime.c- std_runtime_options, parse_opt,
+ 0, 0, children
+ <teythoon> libdiskfs/opts-std-runtime.c-};
+ <teythoon> but
+ <teythoon> libnetfs/std-runtime-argp.c:const struct argp
+ netfs_std_runtime_argp = { 0 };
+ <pinotree> well there are no common startup/runtime options provided by
+ netfs
+ <pinotree> usually netfs-based translators put netfs_std_startup_argp as
+ child as their options, so if netfs starts providing options they would
+ work
+ <teythoon> ah
+ <pinotree> if you have a test showing issues, we can certainly look it :)
+ <teythoon> ok, m/b I was confused...
+ <pinotree> no worries, feel free to ask anytime
+ <teythoon> I thought about providing --update as common runtime flag, like
+ diskfs does, you think it's the right thing to do?
+ <pinotree> what would it do?
+ <teythoon> or should it be left for each translator to implement?
+ <teythoon> nothing by default I guess
+ <pinotree> options provided in libdiskfs are implemented and handled mostly
+ in libdiskfs itself
+ <pinotree> so imho a new option for libnetfs would be there because its
+ behaviour is implemented mostly within libnetfs itself
+ <teythoon> libdiskfs calls diskfs_reload_global_state
+ <teythoon> libnetfs could do the same, allowing translators to plug in
+ anything they wish
+ <teythoon> but I'll implement it in procfs for the time being
+ <pinotree> ah, its alias is remount
+ <teythoon> yes
+ <teythoon> I need that working for procfs
+ <teythoon> btw, I think I got your mount confusion thing figured out
+ <pinotree> but procfs has nothing to update/flush, all the information are
+ fetched at every rpc
+ <teythoon> yes
+ <teythoon> but we still need to ignore the flag
+ <teythoon> otherwise the set_options rpc fails
+ <teythoon> http://paste.debian.net/12938/
+ <teythoon> whee, remounting proc works :)
+ <braunr> :)
diff --git a/open_issues/libpthread.mdwn b/open_issues/libpthread.mdwn
index e2fda122..8e3fde71 100644
--- a/open_issues/libpthread.mdwn
+++ b/open_issues/libpthread.mdwn
@@ -1260,7 +1260,7 @@ Most of the issues raised on this page has been resolved, a few remain.
<braunr> i'll add traces to know which step causes the error
-### IRC, freenode, #hurd, 2012-12-11
+#### IRC, freenode, #hurd, 2012-12-11
<youpi> braunr: mktoolnix seems like a reproducer for the libports thread
priority issue
@@ -1273,7 +1273,7 @@ Most of the issues raised on this page has been resolved, a few remain.
<youpi> that's it, yes
-### IRC, freenode, #hurd, 2013-03-01
+#### IRC, freenode, #hurd, 2013-03-01
<youpi> braunr: btw, "unable to adjust libports thread priority: (ipc/send)
invalid destination port" is actually not a sign of fatality
@@ -1284,6 +1284,34 @@ Most of the issues raised on this page has been resolved, a few remain.
<braunr> weird sentence, agreed :p
+#### IRC, freenode, #hurd, 2013-06-14
+
+ <gnu_srs> Hi, when running check for gccgo the following occurs (multiple
+ times) locking up the console
+ <gnu_srs> unable to adjust libports thread priority: (ipc/send) invalid
+ destination port
+ <gnu_srs> (not locking up the console, it was just completely filled with
+ messages))
+ <braunr> gnu_srs: are you running your translator as root ?
+ <braunr> or, do you have a translator running as an unprivileged user ?
+ <braunr> hm, invalid dest port
+ <braunr> that's a bug :p
+ <braunr> but i don't know why
+ <braunr> i'll have to take some time to track it down
+ <braunr> it might be a user ref overflow or something similarly tricky
+ <braunr> gnu_srs: does it happen everytime you run gccgo checks or only
+ after the system has been alive for some time ?
+ <braunr> (some time being at least a few hours, more probably days)
+
+#### IRC, freenode, #hurd, 2013-07-05
+
+ <braunr> ok, found the bug about invalid ports when adjusting priorities
+ <braunr> thhe hurd must be plagued with wrong deallocations :(
+ <braunr> i have so many problems when trying to cleanly destroy threads
+
+[[libpthread/t/fix_have_kernel_resources]].
+
+
### IRC, freenode, #hurd, 2013-03-11
<braunr> youpi: oh btw, i noticed a problem with the priority adjustement
@@ -1296,6 +1324,171 @@ Most of the issues raised on this page has been resolved, a few remain.
<youpi> uh
<youpi> indeed
+### IRC, freenode, #hurd, 2013-07-01
+
+ <youpi> braunr: it seems as if pfinet is not prioritized enough
+ <youpi> I'm getting network connectivity issues when the system is quite
+ loaded
+ <braunr> loaded with what ?
+ <braunr> it could be ext2fs having a lot more threads than other servers
+ <youpi> building packages
+ <youpi> I'm talking about the buildds
+ <braunr> ok
+ <braunr> ironforge or others ?
+ <youpi> they're having troubles uploading packages while building stuff
+ <youpi> ironforge and others
+ <youpi> that happened already in the past sometimes
+ <youpi> but at the moment it's really pronounced
+ <braunr> i don't think it's a priority issue
+ <braunr> i think it's swapping
+ <youpi> ah, that's not impossible indeed
+ <youpi> but why would it swap?
+ <youpi> there's a lot of available memory
+ <braunr> a big file is enough
+ <braunr> it pushes anonymous memory out
+ <youpi> to fill 900MiB memory ?
+ <braunr> i see 535M of swap on if
+ <braunr> yes
+ <youpi> ironforge is just building libc
+ <braunr> and for some reason, swapping is orders of magnitude slower than
+ anything else
+ <youpi> not linking it yet
+ <braunr> i also see 1G of free memory on it
+ <youpi> that's what I meant with 900MiB
+ <braunr> so at some point, it needed a lot of memory, caused swapping
+ <braunr> and from time to time it's probably swapping back
+ <youpi> well, pfinet had all the time to swap back already
+ <youpi> I don't see why it should be still suffering from it
+ <braunr> swapping is a kernel activity
+ <youpi> ok, but once you're back, you're back
+ <youpi> unless something else pushes you out
+ <braunr> if the kernel is busy waiting for the default pager, nothing makes
+ progress
+ <braunr> (eccept the default pager hopefully)
+ <youpi> sure but pfinet should be back already, since it does work
+ <youpi> so I don't see why it should wait for something
+ <braunr> the kernel is waiting
+ <braunr> and the kernel isn't preemptibl
+ <braunr> e
+ <braunr> although i'm not sure preemption is the problem here
+ <youpi> well
+ <youpi> what I don't understand is what we have changed that could have so
+ much impact
+ <youpi> the only culprit I can see is the priorities we have changed
+ recently
+ <braunr> do you mean it happens a lot more frequently than before ?
+ <youpi> yes
+ <youpi> way
+ <braunr> ok
+ <youpi> ironforge is almost unusable while building glibc
+ <youpi> I've never seen that
+ <braunr> that's weird, i don't have these problems on darnassus
+ <braunr> but i think i reboot it more often
+ <braunr> could be a scalability issue then
+ <braunr> combined with the increased priorities
+ <braunr> if is indeed running full time on the host, whereas swapping
+ issues show the cpu being almost idle
+ <braunr> loadavg is high too so i guess there are many threads
+ <braunr> 0 971 3 -20 -20 1553 305358625 866485906 523M 63M * S<o
+ ? 13hrs /hurd/ext2fs.static -A /dev/hd0s2
+ <braunr> 0 972 3 -20 -20 1434 125237556 719443981 483M 5.85M * S<o
+ ? 13hrs /hurd/ext2fs.static -A /dev/hd0s3
+ <braunr> around 1k5 each
+ <youpi> that's quite usual
+ <braunr> could be the priorities then
+ <braunr> but i'm afraid that if we lower them, the number of threads will
+ grow out of control
+ <braunr> (good thing is i'm currently working on a way to make libpthread
+ actually remove kernel resources)
+ <youpi> but the priorities should be the same in ext2fs and pfinet,
+ shouldn't they?
+ <braunr> yes but ext2 has a lot more threads than pfinet
+ <braunr> the scheduler only sees threads, we don't have a grouping feature
+ <youpi> right
+ <braunr> we also should remove priority depressing in glibc
+ <braunr> (in sched_yield)
+ <braunr> it affects spin locks
+
+ <braunr> youpi: is it normal to see priorities of 26 ?
+ <youpi> braunr: we have changed the nice factor
+ <braunr> ah, factor
+ <youpi> Mm, I'm however realizing the gnumach kernel running these systems
+ hasn't been upgraded in a while
+ <youpi> it may not even have the needed priority levels
+ <braunr> ar euare you using top right now on if ?
+ <braunr> hm no i don't see it any more
+ <braunr> well yes, could be the priorities ..
+ <youpi> I've rebooted with an upgraded kernel
+ <youpi> no issue so far
+ <youpi> package uploads will tell me on the long run
+ <braunr> i bet it's also a scalability issue
+ <youpi> but why would it appear now only?
+ <braunr> until the cache and other data containers start to get filled,
+ processing is fast enough that we don't see it hapenning
+ <youpi> sure, but I haven't seen that in the past
+ <braunr> oh it's combined with the increased priorities
+ <youpi> even after a week building packages
+ <braunr> what i mean is, increased priorities don't affect much if threads
+ porcess things fast
+ <braunr> things get longer with more data, and then increased prioritis
+ give more time to these threads
+ <braunr> and that's when the problem appears
+ <youpi> but increased priorities give more time to the pfinet threads too,
+ don't they?
+ <braunr> yes
+ <youpi> so what is different ?
+ <braunr> but again, there are a lot more threads elsewhere
+ <braunr> with a lot more data to process
+ <youpi> sure, but that has alwasy been so
+ <braunr> hm
+ <youpi> really, 1k5 threads does not surprise me at all :)
+ <youpi> 10k would
+ <braunr> there aren't all active either
+ <youpi> yes
+ <braunr> but right, i don't know why pfinet would be given less time than
+ other threads ..
+ <braunr> compared to before
+ <youpi> particularly on xen-based buildds
+ <braunr> libpthread is slower than cthreads
+ <youpi> where it doesn't even have to wait for netdde
+ <braunr> threads need more quanta to achieve the same ting
+ <braunr> perhaps processing could usually be completed in one go before,
+ and not any more
+ <braunr> we had a discussion about this with antrik
+
+ <braunr> youpi: concerning the buildd issue, i don't think pfinet is
+ affected actually
+ <braunr> but the applications using the network may be
+ <youpi> why using the network would be a difference ?
+ <braunr> normal applications have a lower priority
+ <braunr> what i mean is, pfinet simply has nothing to do, because normal
+ applications don't have enough cpu time
+ <braunr> (what you said earlier seemed to imply pfinet had issues, i don't
+ think it has)
+ <braunr> it should be easy to test by pinging the machine while under load
+ <braunr> we should also check the priority of the special thread used to
+ handle packets, both in pfinet and netdde
+ <braunr> this one isn't spawned by libports and is likely to have a lower
+ priority as well
+
+ <braunr> youpi: you're right, something very recent slowed things down a
+ lot
+ <braunr> perhaps the new priority factor
+ <braunr> well not the factor but i suppose the priority range has been
+ increased
+
+[[nice_vs_mach_thread_priorities]].
+
+ <youpi> braunr: haven't had any upload issue so far
+ <youpi> over 20 uploads
+ <youpi> while it was usually 1 every 2 before...
+ <youpi> so it was very probably the kernel missing the priorities levels
+ <braunr> ok
+ <braunr> i think i've had the same problem on another virtual machine
+ <braunr> with a custom kernel i built a few weeks ago
+ <braunr> same kind of issue i guess
+ <braunr> it's fine now, and always was on darnassus
+
## IRC, freenode, #hurd, 2012-12-05
diff --git a/open_issues/libpthread/t/fix_have_kernel_resources.mdwn b/open_issues/libpthread/t/fix_have_kernel_resources.mdwn
index 10577c1e..4e35161f 100644
--- a/open_issues/libpthread/t/fix_have_kernel_resources.mdwn
+++ b/open_issues/libpthread/t/fix_have_kernel_resources.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -10,7 +10,9 @@ License|/fdl]]."]]"""]]
[[!tag open_issue_libpthread]]
-`t/have_kernel_resources`
+`t/fix_have_kernel_resources`
+
+Address problem mentioned in [[/libpthread]], *Threads' Death*.
# IRC, freenode, #hurd, 2012-08-30
@@ -19,3 +21,178 @@ License|/fdl]]."]]"""]]
<braunr> tschwinge: i.e. the ability to tell the kernel where the stack is,
so it's unmapped when the thread dies
<braunr> which requiring another thread to perform this deallocation
+
+
+## IRC, freenode, #hurd, 2013-05-09
+
+ <bddebian> braunr: Speaking of which, didn't you say you had another "easy"
+ task?
+ <braunr> bddebian: make a system call that both terminates a thread and
+ releases memory
+ <braunr> (the memory released being the thread stack)
+ <braunr> this way, a thread can completely terminates itself without the
+ assistance of a managing thread or deferring work
+ <bddebian> braunr: That's "easy" ? :)
+ <braunr> bddebian: since it's just a thread_terminate+vm_deallocate, it is
+ <braunr> something like thread_terminate_self
+ <bddebian> But a syscall not an RPC right?
+ <braunr> in hurd terminology, we don't make the distinction
+ <braunr> the only real syscalls are mach_msg (obviously) and some to get
+ well known port rights
+ <braunr> e.g. mach_task_self
+ <braunr> everything else should be an RPC but could be a system call for
+ performance
+ <braunr> since mach was designed to support clusters, it was necessary that
+ anything not strictly machine-local was an RPC
+ <braunr> and it also helps emulation a lot
+ <braunr> so keep doing RPCs :p
+
+
+## IRC, freenode, #hurd, 2013-05-10
+
+ <braunr> i'm not sure it should only apply to self though
+ <braunr> youpi: can we get a quick opinion on this please ?
+ <braunr> i've suggested bddebian to work on a new RPC that both terminates
+ a thread and releases its stack to help fix libpthread
+ <braunr> and initially, i thought of it as operating only on the calling
+ thread
+ <braunr> do you see any reason to make it work on any thread ?
+ <braunr> (e.g. a real thread_terminate + vm_deallocate)
+ <braunr> (or any reason not to)
+ <youpi> thread stack deallocation is always a burden indeed
+ <youpi> I'd tend to think it'd be useful, but perhaps ask the list
+
+
+## IRC, freenode, #hurd, 2013-06-26
+
+ <braunr> looks like there is a port right leak in libpthread
+ <braunr> grmbl, the port leak seems to come from mach_port_destroy being
+ buggy :/
+ <braunr> hum, apparently we're not the only ones to suffer from port leaks
+ wrt mach_port_destroy
+ <braunr> ew, libpthread is leaking
+ <pinotree> memory or ports?
+ <braunr> both
+ <pinotree> sounds great ;)
+ <braunr> as it is, libpthread doesn't destroy threads
+ <braunr> it queues them so they're recycled late
+ <braunr> r
+ <braunr> but there is confusion between the thread structure itself and its
+ internal resources
+ <braunr> i.e. there is pthread_alloc which allocates a thread structure,
+ and pthread_create which allocates everything else
+ <braunr> but on pthread_exit, nothing is destroyed
+ <braunr> when a thread structure is reused, its internal resources are
+ replaced by new instances
+ <pinotree> oh
+ <braunr> it's ok for joinable threads but most of our threads are detached
+ <braunr> pinotree: as expected, it's bigger than expected :p
+ <braunr> so i won't be able to write a quick fix
+ <braunr> the true way to fix this is make it possible for threads to free
+ their own resources
+ <braunr> let's do that :p
+ <braunr> ok, got the new thread termination function, i'll build eglibc
+ package providing it, then experiment with libpthread
+ <pinotree> braunr: iirc there's also a tschwinge patch in the debian eglibc
+ about that
+ <braunr> ah
+ <pinotree> libpthread_fix.diff
+ <braunr> i see
+ <braunr> thanks for the notice
+ <braunr> bddebian:
+ http://www.sceen.net/~rbraun/0001-thread_terminate_deallocate.patch
+ <braunr> bddebian: this is what it looks like
+ <braunr> see, short and easy
+ <bddebian> Aye but didn't youpi say not to bother with it??
+ <braunr> he did ?
+ <braunr> i don't remember
+ <bddebian> I thought that was the implication. Or maybe that was the one I
+ already did!?
+ <braunr> i'd be interested in reading that
+ <braunr> anyway, there still are problems in libpthread, and this call is
+ one building block to fix some of them
+ <braunr> some important ones
+ <braunr> (big leaks)
+
+
+## IRC, freenode, #hurd, 2013-06-29
+
+ <braunr> damn, i fix leaks in libpthread, only to find out leaks somewhere
+ else :(
+ <braunr> bddebian: ok, actually it was a bit more complicated than what i
+ showed you
+ <braunr> because in addition to the stack, the call must also release the
+ send right in the caller's ipc space
+ <braunr> (it can't be released before since there would be no mean to
+ reference the thread to destroy)
+ <braunr> or perhaps it should strictly be reserved to self termination
+ <braunr> hmm
+ <braunr> yes it would probably be simpler
+ <braunr> but it should be a decent compromise
+ <braunr> i'm close to having a libpthread that doesn't leak anything
+ <braunr> and that properly destroys threads and their resources
+
+
+## IRC, freenode, #hurd, 2013-06-30
+
+ <braunr> bddebian: ok, it was even more tricky, because the kernel would
+ save the return value on the user stack (which is released by the call
+ and then invalid) before checking for asynchronous software traps (ASTs,
+ a kind of software interrupts in mach), and terminating the calling
+ thread is done by a deferred AST ... :)
+ <braunr> hmm, making threads able to terminate themselves makes rpctrace a
+ bit useless :/
+ <braunr> well, more restricted
+
+ <braunr> ok so, tough question :
+ <braunr> i have a small test program that creates a thread, and inspect its
+ state before any thread dies
+ <braunr> i can see msg_report_wait requests when using ps
+ <braunr> (one per thread)
+ <braunr> one of these requests create a new receive right, apparently for
+ the second thread in the test program
+ <braunr> each time i use ps, i can see the sequence numbers of two receive
+ rights increase
+ <braunr> i guess these rights are related to proc and signal handling per
+ thread
+ <braunr> but i can't find what create them
+ <braunr> does anyone know ?
+ <braunr> tschwing_: ^ :)
+
+ <braunr> again, too many things wrong elsewhere to cleanly destroy threads
+ ..
+ <braunr> something is deeply wrong with controlling terminals ..
+
+
+## IRC, freenode, #hurd, 2013-07-01
+
+ <braunr> youpi: if you happen to notice what receive right is created for
+ each thread (beyond the obvious port used for blocking and waking up),
+ please let me know
+ <braunr> it's the only port leak i have with thread destruction
+ <braunr> and i think it's related to the proc server since i see the
+ sequence number increase every time i use ps
+
+ <braunr> pinotree: my change doesn't fix all the pthread leaks but it's a
+ lot better
+ <braunr> bddebian: i've spent almost the whole week end trying to find the
+ last port leak without success
+ <braunr> there is some weird bug related to the controlling tty that hits
+ me every time i try to change something
+ <braunr> it's the same bug that prevents ttys from being correctly closed
+ when using ssh or screen
+ <braunr> well maybe not the same, but it's close
+ <braunr> some stale receive right kept around for no apparent reason
+ <braunr> and i can't find its source
+
+
+## IRC, freenode, #hurd, 2013-07-02
+
+ <braunr> and btw, i don't think i can make my libpthread patch work
+ <braunr> i'll just aim at avoiding leaks, but destroying threads and their
+ related resources depends on other changes i don't clearly see
+
+
+## IRC, freenode, #hurd, 2013-07-03
+
+ <braunr> grmbl, i don't want to give up thread destruction ..
diff --git a/open_issues/libpthread_assertion_thread_prevp.mdwn b/open_issues/libpthread_assertion_thread_prevp.mdwn
index e8160528..f93f07d6 100644
--- a/open_issues/libpthread_assertion_thread_prevp.mdwn
+++ b/open_issues/libpthread_assertion_thread_prevp.mdwn
@@ -87,3 +87,23 @@ failed"]]
<braunr> removing the libports_stability patch exposed bugs in libpthread,
triggering assertions when queueing/dequeue threads from a queue (but i
don't know which one / in which function)
+
+
+## IRC, freenode, #hurd, 2013-06-25
+
+ <pinotree> braunr:
+ https://buildd.debian.org/status/fetch.php?pkg=libmemcached&ver=1.0.17-2&arch=hurd-i386&stamp=1372165732
+ <pinotree> make: ./pthread/pt-internal.h:122: __pthread_enqueue: Assertion
+ `thread->prevp == 0' failed. \o/
+ <pinotree> (it should rather be /o\, but better pretend not)
+ <braunr> pinotree: yes, we regularly see it
+ <braunr> pinotree: how long has the machine been running at this point ?
+ <pinotree> dunno, you should ask samuel about that
+ <pinotree> does it happen after N hours/days?
+ <braunr> a few days of moderate to high activity yes
+ <pinotree> ah ok
+ <braunr> and i actually see this error much more often when i disable the
+ libports stability patch in the hurd debian package
+ <braunr> so i guess something is wrong with thread recycling
+ <braunr> but i wanted to completely rewrite that part with the new kernel
+ call i asked bddebian to work on :)
diff --git a/open_issues/mondriaan_memory_protection.mdwn b/open_issues/mondriaan_memory_protection.mdwn
new file mode 100644
index 00000000..2c7b9ba1
--- /dev/null
+++ b/open_issues/mondriaan_memory_protection.mdwn
@@ -0,0 +1,85 @@
+[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+<http://scale.eecs.berkeley.edu/mondriaan/>.
+
+
+# IRC, freenode, #hurd, 2013-07-02
+
+ <xscript> in any case, what I wanted to check is if current hurd support
+ PIE
+ <xscript`> I just saw samuel posted some fixes to have PIE working in hurd
+ <xscript`> are those included in the official image?
+ <youpi> sure
+ <youpi> it's just a trivial fixup in some address calculation code
+ <xscript> youpi: nice
+ <xscript> and does anyone know how complex would it be to implement some
+ hackish support to have non-overlapping virtual addresses for
+ applications supporting PIE?
+ <braunr> not too difficult
+ <xscript> really? I didn't expect such an answer XD
+ <xscript> I'd like to have something similar to a SASOS
+ <xscript> (single address space os)
+ <braunr> ?
+ <braunr> you mean an sasos on top of mach ?
+ <xscript> yes, but only for a few apps I want to evaluate
+ <braunr> i see
+ <xscript> the optimal would be to have all of hurd's servers on that mode
+ <braunr> you'l probably need to implement a small allocator but other than
+ that it shouldn't be too hard, yes
+ <braunr> uh ??
+ <xscript> but running on 32 bits can be a problem here
+ <braunr> and not hurdish at all
+ <xscript> what's not hurdish?
+ <braunr> we do want address space separation
+ <xscript> well, you can have multiple address spaces (page tables), but
+ without overlapping addresses between them
+ <xscript> that's exactly what I'm looking for
+ <braunr> sorry i don't see what you mean
+ <braunr> if you run several servers in the same address space, they can
+ corrupt each other
+ <braunr> we don't want that
+ <braunr> it's that simple
+ <xscript> yes, sorry, I didn't explain myself
+ <xscript> I want a separate address space on each server
+ <xscript> but I want all memory allocations to be on addresses unique to
+ the whole OS
+ <braunr> that still doesn't make sense
+ <xscript> well, it will still be secure
+ <xscript> but I know it does not make sense per se
+ <xscript> I want to do some experiments with a simulator
+ <braunr> why do you want them non overlapping if they're separate ?
+ <xscript> well, in my simulator I wouldn't need to change the page tables,
+ protection is provided through other means
+ <braunr> segmentation ?
+ <xscript> that's one possibility
+ <xscript> (small address spaces)
+ <braunr> what do you have in mind ?
+ <braunr> it wouldn't be on top of mach anyway then
+ <braunr> mach implements paging
+ <xscript> what I'm simulating is something of the likes of Mondriaan
+ (http://www.cs.utexas.edu/~witchel/pubs/mmp-asplos2002.pdf)
+ <xscript> paging is ok for me
+ <braunr> 19:06 < xscript> well, in my simulator I wouldn't need to change
+ the page tables, protection is provided through other means
+ <braunr> it didn't sound so
+ <xscript> I meant switching page tables (cr3, etc)
+ <braunr> mach does that
+ <xscript> I know, I know, I can just ignore that part for the moment
+ <braunr> ok
+ <xscript> for now, I'd like to morph hurd into a SASOS using one page table
+ per process
+ <xscript> I just wanted to know how hard that would be, without starting
+ with a full dive into the code
+ <xscript> there are other options (OSes, microkernels), but none of them
+ provides as many readily-available applications as hurd
+ <xscript> I suppose MINIX would also be easy to modify, but there's less
+ apps there, and I also would like to tamper with MIG
+ <xscript> I just wonder how hard it would be to modify MIG
diff --git a/open_issues/some_todo_list.mdwn b/open_issues/some_todo_list.mdwn
index 80592abf..48c2944d 100644
--- a/open_issues/some_todo_list.mdwn
+++ b/open_issues/some_todo_list.mdwn
@@ -42,6 +42,21 @@ From Marcus, 2002:
* Translators
* Does settrans -g work? -fg?
* Does fsysopts work? Does setting options with fsysopts work?
+
+ IRC, freenode, #hurd, 2013-05-23:
+
+ <gnu_srs> fsysopts /servers/socket/2 works by /1 gives Operation not
+ supported.
+
+ [[!taglink open_issue_hurd]].
+
+ <braunr> ah right, some servers don't implement that
+ <braunr> work around this by using showtrans
+ <braunr> fsysopts asks the server itself how it's running, usually giving
+ its command name and options
+ <braunr> showtrans asks the parent how it starts a passive translator
+ attached to the node
+ <gnu_srs> Yes showtrans works :), thanks.
* Does stat() work on all translated nodes and give proper data?
* What about chown, chmod (some translators should pass this through to the underlying node, esp in /dev!)
* Does statfs give correct data?
diff --git a/open_issues/translator_stdout_stderr.mdwn b/open_issues/translator_stdout_stderr.mdwn
index 14ea1c6d..89efd4e1 100644
--- a/open_issues/translator_stdout_stderr.mdwn
+++ b/open_issues/translator_stdout_stderr.mdwn
@@ -1,5 +1,5 @@
-[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation,
-Inc."]]
+[[!meta copyright="Copyright © 2008, 2009, 2010, 2011, 2013 Free Software
+Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -9,25 +9,69 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
-[[!tag open_issue_hurd]]
+[[!toc]]
-There have been several discussions and proposals already, about adding a
-suitable logging mechanism to translators, for example.
+# "Weird Issue"
-Decide / implement / fix that (all?) running (passive?) translators' output
-should show up on the (Mach / Hurd) console / syslog.
+## IRC, freenode, #hurd, 2013-07-01
+[[!tag open_issue_hurd]]
-[[!taglink open_issue_documentation]]: [[!message-id
-"87oepj1wql.fsf@becket.becket.net"]]
+ <teythoon> oh, btw, there is this weird issue that I cannot figure out
+ <teythoon> I noticed that there is no newline printed by /hurd/init after
+ printing " proc" and " auth"
+ <teythoon> but there *is* a printf("\n"); fflush(stdout); in there
+ <teythoon> it's just not working
+ <pinotree> iirc a newline is supposed to be printed after all the essential
+ servers have been started
+ <pinotree> that one
+ <teythoon> yes
+ <teythoon> but this doesn't happen
+ <teythoon> for whatever reason printf("foo"); yields no output
+ <braunr> how are proc and auth printed ?
+ <teythoon> neither does printf("%s", "foo");
+ <teythoon> using printf
+ <teythoon> but printf("%i fooo", 4); works
+ <youpi> uh
+ <braunr> ??
+ <youpi> and does printf("loooooooooong line") worker?
+ <teythoon> no
+ <youpi> uh
+ <youpi> -er
+ <teythoon> and yes, the code is always fflushing stdout
+ <youpi> perhaps trying to put a sleep(1); to check for timing issues?
+ <teythoon> yes, I suspect something like that
+ <teythoon> b/c *sometimes* my hurd just freezes at this point
+ <braunr> ???
+ <teythoon> after printing proc auth and not printing the newline
+ <braunr> such horror stories .
+ <braunr> ..
+ <teythoon> and I *think* that putting more printfs there for testing
+ purposes makes this worse, but I'm not sure about this
+ <braunr> in case you need to debug using printf, there is the mach_print
+ system call
+ <braunr> (in -dbg kernels)
+[[microkernel/mach/gnumach/interface/syscall/mach_print]].
-[[!taglink open_issue_documentation]]: Neal once had written an email on this
-topic.
+ <teythoon> what's wrong with using printf?
+ <braunr> you need to write the little assembly call yourself, where you
+ intend to use it, because it's not exported by glibc
+ <braunr> printf is an RPC
+ <braunr> teythoon: RPCs are complicated stuff
+ <braunr> in particular in core glibc parts like signal handling
+ <youpi> and printf goes through the console translator
+ <braunr> also, if you don't yet have a console server available, it comes
+ in handy
+ <youpi> better direct output directly to the kernel
-IRC, freenode, #hurd, 2011-11-06
+# `stderr` buffered
+
+## IRC, freenode, #hurd, 2011-11-06
+
+[[!tag open_issue_hurd]]
<youpi> about CLI_DEBUG, you can use #define CLI_DEBUG(fmt, ...) {
fprintf(stderr, fmt, ## __VA_ARGS__); fflush(stderr); }
@@ -40,7 +84,24 @@ IRC, freenode, #hurd, 2011-11-06
<tschwinge> That sounds wrong.
-IRC, freenode, #hurd, 2011-11-23
+# Logging
+
+[[!tag open_issue_hurd]]
+
+There have been several discussions and proposals already, about adding a
+suitable logging mechanism to translators, for example.
+
+Decide / implement / fix that (all?) running (passive?) translators' output
+should show up on the (Mach / Hurd) console / syslog.
+
+[[!taglink open_issue_documentation]]: [[!message-id
+"87oepj1wql.fsf@becket.becket.net"]]
+
+[[!taglink open_issue_documentation]]: Neal once had written an email on this
+topic.
+
+
+## IRC, freenode, #hurd, 2011-11-23
<braunr> we'd need a special logging task for hurd servers
<pinotree> if syslog would work, that could be used optionally
diff --git a/open_issues/user-space_device_drivers.mdwn b/open_issues/user-space_device_drivers.mdwn
index 8cde8281..7331bb54 100644
--- a/open_issues/user-space_device_drivers.mdwn
+++ b/open_issues/user-space_device_drivers.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2009, 2011, 2012 Free Software Foundation,
+[[!meta copyright="Copyright © 2009, 2011, 2012, 2013 Free Software Foundation,
Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
@@ -154,12 +154,100 @@ A similar problem is described in
< braunr> s/disk/storage/
-### IRC, freenode, #hurd, 2012-04-25
+#### IRC, freenode, #hurd, 2012-04-25
<youpi> btw, remember the initrd thing?
<youpi> I just came across task.c in libstore/ :)
+#### IRC, freenode, #hurd, 2013-06-24
+
+ <youpi> we added a new initrd command to gnumach, to expose a new mach
+ device, which ext2fs can open and unzip
+ <youpi> we consider replacing that with simply putting the data in a dead
+ process
+ <youpi> s/process/task
+ <youpi> and let ext2fs read data from the task, and kill it when done
+ <teythoon> ok
+ <youpi> alternatively, tmps would work with an initial .tar.gz payload
+ <youpi> that would be best for memory usage
+ <youpi> tmpfs*
+ <teythoon> can't we replace the initrd concept with sub/neighbourhood?
+ <youpi> setting up tmpfs with an initial payload could be done with a
+ bootstrap subhurd
+ <teythoon> yes
+ <youpi> but it seems to me that having tmpfs being able to have an initial
+ payload is interesting
+ <teythoon> is there any advantage of the tmpfs translator prefilled with a
+ tarball over ext2fs with copy & bunzip?
+ <youpi> memory usage
+ <youpi> ext2fs with copy&bunzip takes memory for zeroes
+ <youpi> and we have to forecast how much data might be stored
+ <youpi> (if writable)
+ <teythoon> ah sure
+ <teythoon> but why would it have to be in the tmpfs translator? I why not
+ start the translator and have tar extract stuff there?
+ <teythoon> with the livecd I had trouble replacing the root translator, but
+ when using subhurds that shouldn't be a prwoblem at all
+ <youpi> I don't have a real opinion on this
+ <youpi> except that people don't usually like initrd :)
+ <braunr> 12:43 < teythoon> but why would it have to be in the tmpfs
+ translator? I why not start the translator and have tar extract stuff
+ there?
+ <braunr> that sounds an awful lot like an initramfs
+ <teythoon> yes, exactly, without actually having an initramfs of course
+ <braunr> yep
+ <braunr> i actually prefer that way too
+ <teythoon> a system on a r/o isofs cannot do much, but it can do this
+ <braunr> on the other hand, i wouldn't spend much time on a virtio disk
+ driver for now
+ <braunr> the hurd as it is can't boot on a device that isn't managed by the
+ kernel
+ <braunr> we'd need to change the boot protocol
+
+
+#### IRC, freenode, #hurd, 2013-06-28
+
+ <teythoon> I'm tempted to redo a livecd, simpler and without the initrd
+ hack that youpi used for d-i
+ <braunr> initrd hack ?
+ <braunr> you mean more a la initramfs then ?
+ <teythoon> no, I thought about using a r/o isofs translator, but instead of
+ fixing that one up with a r/w overlay and lot's of firmlinks like I used
+ to, it would just start an ext2fs translator with copy on an image stored
+ on the iso and start a subhurd
+ <braunr> why a subhurd ?
+ <teythoon> neighbourhurd even
+ <teythoon> b/c back in the days I had trouble replacing /
+ <braunr> yes, that's hard
+ <teythoon> subhurd would take of that for free
+ <braunr> are you sure ?
+ <teythoon> somewhat
+ <braunr> i'm not, but this requires thorough thinking
+ <braunr> and i'm not there yet
+ <teythoon> y would it not?
+ <teythoon> just start a subhurd and let that one take over the console and
+ let the user and d-i play nicely in that environment
+ <teythoon> no hacks involved
+ <braunr> because it would require sharing things between the two system
+ instances, and that's not easy
+ <teythoon> no but the bootstrap system does nothing after launching the
+ subhurd
+ <teythoon> I mean yes, technically true, but why would it be hard to share
+ with someone who does nothing?
+ <braunr> the context isn't well defined enough to clearly state anything
+ <braunr> if you don't use the resources of the first hurd, that's ok
+ <braunr> otherwise, it may be easy or not, i don't know yet
+ <teythoon> you think it's worth a shot and see what issues crop up?
+ <braunr> sure
+ <braunr> definitely
+ <teythoon> it doesn't sound complicated at all
+ <braunr> it's easy enough to the point we see something goes wrong or works
+ completely
+ <braunr> so worth testin
+ <teythoon> cool :)
+
+
### IRC, freenode, #hurd, 2012-07-17
<bddebian> OK, here is a stupid question I have always had. If you move