From 9667351422dec0ca40a784a08dec7ce128482aba Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Wed, 10 Jul 2013 23:39:29 +0200 Subject: IRC. --- open_issues/anatomy_of_a_hurd_system.mdwn | 217 +++++++++++++++++++++ open_issues/glibc.mdwn | 17 ++ open_issues/gnumach_integer_overflow.mdwn | 35 +++- .../gnumach_vm_object_resident_page_count.mdwn | 28 ++- open_issues/libmachuser_libhurduser_rpc_stubs.mdwn | 26 +++ open_issues/open_symlink.mdwn | 14 +- open_issues/profiling.mdwn | 105 ++++++++++ open_issues/sendmsg_scm_creds.mdwn | 77 +++++++- open_issues/translate_fd_or_port_to_file_name.mdwn | 51 +++++ 9 files changed, 564 insertions(+), 6 deletions(-) (limited to 'open_issues') diff --git a/open_issues/anatomy_of_a_hurd_system.mdwn b/open_issues/anatomy_of_a_hurd_system.mdwn index 677e4625..11a1f754 100644 --- a/open_issues/anatomy_of_a_hurd_system.mdwn +++ b/open_issues/anatomy_of_a_hurd_system.mdwn @@ -380,3 +380,220 @@ Actually, the Hurd has never used an M:N model. Both libthreads (cthreads) and l if you're looking for how to do it for a non-translator application, the answer is probably somewhere in glibc _hurd_startup i'd guess + + +# IRC, freenode, #hurd, 2013-05-23 + + Hi, is there any efficient way to control which backed + translators are called via RPC with a user space program? + Take for example io_stat: S_io_stat is defined in boot/boot.c, + pfinet/io-ops.c and pflocal/io.c + And the we have libdiskfs/io-stat.c:diskfs_S_io_stat, + libnetfs/io-stat.c:netfs_S_io_stat, libtreefs/s-io.c:treefs_S_io_stat, + libtrivfs/io-stat.c:trivfs_S_io_stat + How are they related? + gnu_srs: it depends on the server (translator) managing the files + (nodes) you're accessing + so use fsysopts to know the server, and see what this server uses + fsysopts /hurd/pfinet and fsysopts /hurd/pflocal gives the same + answer: ext2fs --writable --no-inherit-dir-group --store-type=typed + device:hd0s1 + of course + the binaries are regular files + see /servers/socket/1 and /servers/socket/2 instead + which are the nodes representing the *service* + again, the hurd uses the file system as a service directory + this usage of the file system is at the core of the hurd design + files are not mere files, they're service names + it happens that, for most files, the service behind them is the + same as for regular files + gnu_srs: this *must* be obvious for you to do any tricky work on + the hurd + fsysopts /servers/socket/2 works by /1 gives Operation not + supported. + +[[!taglink open_issue_hurd]]. + + ah right, some servers don't implement that + work around this by using showtrans + fsysopts asks the server itself how it's running, usually giving + its command name and options + showtrans asks the parent how it starts a passive translator + attached to the node + Yes showtrans works :), thanks. + Anyway, if I create a test program calling io_stat I assume + S_io_stat in pflocal is called. + How to make the program call S_io_stat in pfinet instead? + create a socket managed by pfinet + i.e. an inet or inet6 socket + you can't assume io_stat is serviced by pflocal + only stats on unix sockets of pipes will be + or* + thanks, what about the *_S_io_stat functions? + what about them ? + How they fit into the picture, e.g. diskfs_io_stat? + *diskfs_S_io_stat + gnu_srs: if you open a file managed by a server using libdiskfs, + e.g. ext2fs, that one will be called + Using the same user space call: io_stat, right? + it's all userspace + say rather, client-side + the client calls the posix stat() function, which is implemented + by glibc, which converts it into a call to io_stat, and sends it to the + server managing the open file + the io_stat can change depending on the server + the remote io_stat implementation, i mean + identify the server, and you will identify the actual + implementation + + +# IRC, freenode, #hurd, 2013-06-15 + + ive been reading a little about exokernels or unikernels, and i + was wondering if it might be relevant to the GNU/hurd design. I'm not + too familiar with hurd terminology so forgive me. what if every + privileged service was compiled as its own mini "kernel" that handled (a) + any hardware related to that service (b) any device nodes exposed by that + service etc... + yes but not really that way + under the current hurd model of the operating system, how would + you talk to hardware that required specific timings like sound hardware? + through mapped memory + is there such a thing as an interrupt request in hurd? + obviously + ok + is there any documentation i can read that involves a driver that + uses irqs for hurd? + you can read the netdde code + dde being another project, there may be documentation about it + somewhere else + i don't know where + thanks + i read a little about dde, apparently it reuses existing code from + linux or bsd by reimplementing parts of the old kernel like an api or + something + yes + it must translate these system calls into ipc or something + then mach handles it? + exactly + that's why i say it's not the exokernel way of doing things + ok + so does every low level hardware access go through mach?' + yes + well no + interrupts do + ports (on x86) + everything else should be doable through mapped memory + seems surprising that the code for it is so small + 1/ why surprising ? and 2/ "so small" ? + its like the core of the OS, and yet its tiny compared to say the + linux kernel + it's a microkenrel + well, rather an hybrid + the size of the equivalent code in linux is about the same + ok + with the model that privileged instructions get moved to + userspace, how does one draw the line between what is OS and what is user + code + privileged instructions remain in the kernel + that's one of the few responsibilities of the kernel + i see, so it is an illusion that the user has privilege in a sense + hum no + or, define "illusion" + well the user can suddenly do things never imaginable in linux + that would have required sudo + yes + well, they're not unimaginable on linux + it's just not how it's meant to work + :) + and why things like fuse are so slow + i still don't get "i see, so it is an illusion that the user has + privilege in a sense" + because the user doesnt actually have the elevated privilege its + the server thing (translator)? + it does + not at the hardware level, but at the system level + not being able to do it directly doesn't mean you can't do it + right + it means you need indirections + that's what the kernel provides + so the user cant do stuff like outb 0x13, 0x1 + he can + he also can on linux + oh + that's an x86 specifity though + but the user would need hardware privilege to do that + no + or some kind of privilege + there is a permission bitmap in the TSS that allows userspace to + directly access some ports + but that's really x86 specific, again + i was using it as an example + i mean you wouldnt want userspace to directly access everything + yes + the only problem with that is dma reall + y + because dma usually access physical memory directly + are you saying its good to let userspace access everything minus + dma? + otherwise you can just centralize permissions in one place (the + kernel or an I/O server for example) + no + you don't let userspace access everything + ah + yes + userspace asks for permission to access one specific part (a + memory range through mapping) + and can't access the rest (except through dma) + except through dma?? doesnt that pose a large security threat? + no + you don't give away dma access to anyone + only drivers + ahh + and drivers are normally privileged applications anyway + so a driver runs in userspace? + so the only effect is that bugs can affect other address spaces + indirectly + netdde does + interesting + and they all should but that's not the case for historical reasons + i want to port ALSA to hurd userspace :D + that's not so simple unfortunately + one of the reasons it's hard is that pci access needs arbitration + and we don't have that yet + i imagine that would be difficult + yes + also we're not sure we want alsa + alsa drivers, maybe, but probably not the interface itself + its tangled spaghetti + but the guy who wrote JACK for audio hates OSS, and believes it is + rubbish due to the fact it tries to read and write to a pcm device node + like a filesystem with no care for timing + i don't know audio well enough to tell you anything about that + was that about oss3 or oss4 ? + also, the hurd isn't a real time system + so we don't really care about timings + but with "good enough" latencies, it shouldn't be a problem + but if the audio doesnt reach the sound card in time, you will get + a crackle or a pop or a pause in the signal + yep + it happens on linux too when the system gets some load + some users find this unnacceptable + some users want real time systems + using soft real time is usually plenty enough to "solve" this kind + of problems + will hurd ever be a real time system? + no idea + if somebody works on it why not + it's the same as linux + it should certainly be simpler than on linux though + hmm + microkernels are well suited for real time because of the well + defined interfaces they provide and the small amount of code running in + kernel + that sounds promising + you usually need to add priority inheritance and take care of just + a few corner cases and that's all + but as youpi said, it still requires work + and nobody's working on it + you may want to check l4 fiasco.oc though diff --git a/open_issues/glibc.mdwn b/open_issues/glibc.mdwn index 33a1a071..b06b4f9f 100644 --- a/open_issues/glibc.mdwn +++ b/open_issues/glibc.mdwn @@ -1305,6 +1305,23 @@ Failures, mostly in order of appearance: Due to `ext2fs --no-atime`. + * IRC, OFTC, #debian-hurd, 2013-05-08 + + bah, tst-atime failure :) + do you have its output? + well it's very simple + I have the noatime option on / :) + oh + fortunately fsysopts works :) + the test checks whether ST_NOATIME is in the mount + options, maybe it would be a good idea to provide it + yes + unfortunately it isn't in posix, so i'm not sure whether + adding it to the general bits/statvfs.h would be welcome + or whether we should fork it, like it is done for linux + oh no, we fork it already + \o/ + `dirent/tst-fdopendir.out`: directory atime changed diff --git a/open_issues/gnumach_integer_overflow.mdwn b/open_issues/gnumach_integer_overflow.mdwn index 2166e591..08a29268 100644 --- a/open_issues/gnumach_integer_overflow.mdwn +++ b/open_issues/gnumach_integer_overflow.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -15,3 +15,36 @@ License|/fdl]]."]]"""]] yes, we have integer overflows on resident_page_count, but luckily, the member is rarely used + +See also [[gnumach_vm_object_resident_page_count]]. + + +## IRC, freenode, #hurd, 2013-06-04 + + this is declared as int on vm_object.h + and as it as counter it's always positive + yes it should be unsigned + ok + but leave it as it is for consistency with the rest + i send patch :) + please no + unless you've fully determined the side effects + i've grepped the vars and saw only comparisons > and = 0 + never less than 0 + > 0 is the same + well + > not, but >= would be a problem + http://paste.debian.net/plain/8527 + asctually no >=0 + still, i don't want to change that unless it's strictly necessary + hum, you're grepping ref_count, not resident_page_count + i did both + on resident_page_count theres resident_page_count >= 0 + = 0, == 0 + this isn't the only possible issue + anyway + for now there is no reason to change anything unless you do a full + review + only place i see resdent_page_count and page_count being decreased + it's on vm/vm_resident.c + vm_page_remove() and vm_page_replace() diff --git a/open_issues/gnumach_vm_object_resident_page_count.mdwn b/open_issues/gnumach_vm_object_resident_page_count.mdwn index cc1b8897..5c1247b2 100644 --- a/open_issues/gnumach_vm_object_resident_page_count.mdwn +++ b/open_issues/gnumach_vm_object_resident_page_count.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -20,3 +20,29 @@ License|/fdl]]."]]"""]] luckily, this should be easy to solve `vm/vm_object.h:vm_object:resident_page_count`. + + +## IRC, freenode, #hurd, 2013-06-03 + + regarding + https://www.gnu.org/software/hurd/open_issues/gnumach_vm_object_resident_page_count.html, + this is fixed. it's an int. what should happen do this page? /dev/null + ? + I guess so + + +## IRC, freenode, #hurd, 2013-06-04 + + + http://darnassus.sceen.net/~hurd-web/open_issues/gnumach_vm_object_resident_page_count/ + this is a int + how to deal with the page? delete it? archive it? + ? + the issue originallu reported was fixed, right? + i think so, yes + for now at least + so this stays on the open_issues on the wiki anyway? + no, it should go away + i dont know how to suggest deletion on the wiki + don't + i'll do it later diff --git a/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn b/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn index 670c82cb..11bebd6e 100644 --- a/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn +++ b/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn @@ -133,3 +133,29 @@ License|/fdl]]."]]"""]] (imo, the mach_debug interface should be adjusted to be used with privileged ports only) (well, maybe not all mach_debug RPCs) + + +# `gnumach.defs` + +[[!message-id +"CAEvUa7nd2LSUsMG9axCx5FeaD1aBvNxE4JMBe95b9hbpdqiLdw@mail.gmail.com"]]. + + +## IRC, freenode, #hurd, 2013-05-13 + + youpi: what's the point of the last commit in the upstream hurd + repository (utils/vmstat: Use gnumach.defs from gnumach) ? + or rather, i think i see the point, but then why do it only for + gnumach and not fot the rest ? + for* + most probably because nobody did it, probably + aiui, it makes the hurd build process not rely on system headers + (and nobody had any issue with it) + well yes, that's why i'm wondering :) + it looks perfectly fine to me to use system headers instead of + generating them + ah right + I thought there was actually a reason + I'll revert + could you answer David about it? + sure diff --git a/open_issues/open_symlink.mdwn b/open_issues/open_symlink.mdwn index 20e4a4fe..f71109a9 100644 --- a/open_issues/open_symlink.mdwn +++ b/open_issues/open_symlink.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2012 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this @@ -10,9 +10,21 @@ License|/fdl]]."]]"""]] [[!tag open_issue_glibc]] + # IRC, freenode, #hurd, 2012-01-02 hm, is it a known issue that open("somesymlink", O_RDONLY | O_NOFOLLOW) does not fail with ELOOP? pinotree: iirc there is code for it, maybe not the same behavior as on linux + + +## IRC, OFTC, #debian-hurd, 2013-05-08 + + the hurd issue is that Q_NOFOLLOW seems broken on symlinks, and + thus open(symlink, O_NOFOLLOW) doesn't fail with ELOOP + I don't really see why it should fail + since NOFOLLOW says not to follow the symlink + yeah, but you cannot open a symlink + ah right ok + interesting :) diff --git a/open_issues/profiling.mdwn b/open_issues/profiling.mdwn index 26e6c97c..545edcf6 100644 --- a/open_issues/profiling.mdwn +++ b/open_issues/profiling.mdwn @@ -9,10 +9,14 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] +[[!meta title="Profiling, Tracing"]] + *Profiling* ([[!wikipedia Profiling_(computer_programming) desc="Wikipedia article"]]) is a tool for tracing where CPU time is spent. This is usually done for [[performance analysis|performance]] reasons. + * [[hurd/debugging/rpctrace]] + * [[gprof]] Should be working, but some issues have been reported, regarding GCC spec @@ -33,3 +37,104 @@ done for [[performance analysis|performance]] reasons. * [[SystemTap]] * ... or some other Linux thing. + + +# IRC, freenode, #hurd, 2013-06-17 + + is that possible we develop rpc msg analyse tool? make it clear + view system at different level? + hurd was dynamic system, how can we just read log line by line + congzhang: well, you can use rpctrace and then analyze the logs, + but rpctrace is quite intrusive and will slow down things (like strace or + similar) + congzhang: I don't know if a low-overhead solution could be made + or not + that's the problem + when real system run, the msg cross different server, and then + the debug action should not intrusive the process itself + we observe the system and analyse os + when rms choose microkernel, it's expect to accelerate the + progress, but not + microkernel make debug a litter hard + well, it's not limited to microkernels, debugging/tracing is + intrusive and slow things down, it's an universal law of compsci + no, it makes debugging easier + I don't think so + you can gdb the various services (like ext2fs or pfinet) more + easily + and rpctrace isn't any worse than strace + how easy when debug lpc + lpc ? + because cross context + classic function call + when find the bug source, I don't care performance, I wan't to + know it's right or wrong by design, If it work as I expect + I optimize it latter + I have an idea, but don't know weather it's usefull or not + rpctrace is a lot less instrusive than ptrace based tools + congzhang: debugging is not made hard by the design choice, but by + implementation details + as a simple counter example, someone often cited usb development + on l3 being made a lot easier than on a monolithic kernel + Collect the trace information first, and then layout the msg by + graph, when something wrong, I focus the trouble rpc, and found what + happen around + "by graph" ? + yes + braunr: directed graph or something similar + and not caring about performance when debugging is actually stupid + i've seen it on many occasions, people not being able to use + debugging tools because they were far too inefficient and slow + why a graph ? + what you want is the complete trace, taking into account cross + address space boundaries + yes + well it's linear + switching server + by independent process view it's linear + it's linear on cpu's view too + yes, I need complete trace, and dynamic control at microkernel + level + os, if server crash, and then I know what's other doing, from + the graph + graph needn't to be one, if the are not connect together, time + sort them + when hurd was complete ok, some tools may be help too + i don't get what you want on that graph + sorry, I need a context + like uml sequence diagram, I need what happen one by one + from server's view and from the function's view + that's still linear + so please stop using the word graph + you want a trace + a simple call trace + yes, and a tool + with some work gdb could do it + you mean under some microkernel infrastructure help + ? + if needed + braunr: will that be easy? + not too hard + i've had this idea for a long time actually + another reason i insist on migrating threads (or rather, binding + server and client threads) + braunr: that's great + the current problem we have when using gdb is that we don't know + which server thread is handling the request of which client + we can guess it + but it's not always obvious + I read the talk, know some of your idea + make things happen like classic kernel, just from function + ,sure:) + that's it + I think you and other do a lot of work to improve the mach and + hurd, buT we lack the design document and the diagram, one diagram was + great than one thousand words + diagrams are made after the prototypes that prove they're doable + i'm not a researcher + and we have little time + the prototype is the true spec + that's why i wan't cllector the trace info and show, you can + know what happen and how happen, maybe just suitable for newbie, hope + more young hack like it + once it's done, everything else is just sugar candy around it diff --git a/open_issues/sendmsg_scm_creds.mdwn b/open_issues/sendmsg_scm_creds.mdwn index cf0103df..d4a6126e 100644 --- a/open_issues/sendmsg_scm_creds.mdwn +++ b/open_issues/sendmsg_scm_creds.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2010, 2011, 2012 Free Software Foundation, +[[!meta copyright="Copyright © 2010, 2011, 2012, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -11,7 +11,8 @@ License|/fdl]]."]]"""]] [[!tag open_issue_glibc]] -IRC, unknown channel, unknown date. + +# IRC, unknown channel, unknown date Credentials: s_uid 1000, c_uid 1000, c_gid 100, c_pid 2722 2722: Credentials: s_uid 1000, c_uid 1000, c_gid 100, c_pid 2724 @@ -91,10 +92,80 @@ IRC, unknown channel, unknown date. yep ok, good :) -/!\ IRC, freenode, #hurd, 2011-08-11 + +## IRC, freenode, #hurd, 2011-08-11 < pinotree> (but that patch is lame) + +## IRC, freenode, #hurd, 2013-05-09 + + youpi: Since you are online tonight, which authentication + callbacks to be used for SCM_CREDS calls. + I have working code and need to add this to make things + complete. The auth server, lib* or where? + I don't understand the question + authentication callbacks like for SCM_RIGHTS, see + + http://www.gnu.org/software/hurd/open_issues/sendmsg_scm_creds.html + I still don't understand: what are you trying to do actually? + solving the SCM_CREDS propbems with e.g. dbus. + so what is the relation with pinotree's patch on the page above? + (I have no idea of the current status of all that) + his patch was not merged, right? have to shut down, sorry, bbl, + gn8 + that patch was not merged since it is not in the correct place + as I said, I have no idea about the status + youpi: basically, it boils down to knowing, when executing the + code implementing an rpc, who requested that rpc (pid, uid, gid) + i.e. getting information about the reply port for instance? + well that might be somehow faked + (by perhaps giving another task's port as reply port) + for example (which would be the code path for SCM_CREDS), when + you call call the socket sendmsg(), pflocal would know who did that rpc + and fill the auxilliary data) + s,)$,, + youpi: yes, i know about this faking issue, iirc also antrik + mentioned quite some time ago + ok + that's one of the (imho) two issues of this + my hurd-foo is not enough to know whether there are solutions to + the problem above + + +### IRC, freenode, #hurd, 2013-05-14 + + Hi, regarding SCM_CREDS, I have some working code in + sendmsg.c. Now I need to make a callback to authenticate the pid, uid, + etc + Where to hook call that into pflocal? + the auth server? + maybe _io_restrict_auth is the correct call to use (same as for + SCM_RIGHTS)? + + +### IRC, freenode, #hurd, 2013-05-17 + + I'm working on the scm credentials right now to enable (via dbus) + more X window managers to work properly. + seems to be rather tricky:-( + gnu_srs: I guess you also need SCM_CREDS, right? + hi pochu, that's what I'm working on, extending your SCM_RIGHTS + work to SCM_CREDS + that's what i did as proof, years ago? + it would be good to know which server calls to make, I'll be back + with proposals of functions to use. + there was a talk, years ago when i started with this, and few + days ago too + every methods has its own drawbacks, and basically so far it + seems that in every method the sender identity can be faked somehow + pinotree: Yes of course your patch was perfect, but it seemed + like people wanted a server acknowledgement too. + no, my patch was not perfect at all + if it was, it would have been cleaned up and sent few years ago + already + + --- See also [[dbus]], [[pflocal_socket_credentials_for_local_sockets]] and diff --git a/open_issues/translate_fd_or_port_to_file_name.mdwn b/open_issues/translate_fd_or_port_to_file_name.mdwn index 0d786d2a..0d6a460c 100644 --- a/open_issues/translate_fd_or_port_to_file_name.mdwn +++ b/open_issues/translate_fd_or_port_to_file_name.mdwn @@ -105,6 +105,57 @@ License|/fdl]]."]]"""]] Ah, for /proc/*/maps, right. I've been thinking more globally. +## task_get_name, task_set_name RPCs + +[[!message-id "518AA5B0.6030409@verizon.net"]] + + +## IRC, freenode, #hurd, 2013-05-10 + + tschwinge's suggestion to put names on ports instead of tasks would + be useful too + do you get task ports as easily as you get tasks in kdb ? + there is task->itk_self & such + or itk_space + I don't remember which one is used by userspace + i mean + when you use the debugger, can you easily find its ports ? + the task ports i mean + or thread ports or whatever + once you have a task, it's a matter of getting the itk_self port + s/port/field member/ + so the debugger provides you with the addresses of the structs + right ? + yes, that's what we have already + then ok + bddebian: do that :p + hehe + see show all thread + (haven't used kdb in a long time) + So, adding a name to ports like I did with tasks? + remove what you did for tasks + move it to ports + it's very similar + but hm + i'm not sure where the RPC would be + this RPC would exist for *all* ports + or only for kernel objects if added to gnumach.defs + it's just about moving the char array field to another structure + and plugging that + But mach_task_self is a syscal, it looks like itk_self is just a + pointer to an ipc_port ? + so ? + you take that pointer and you get the port + just like vm_map gets a struct vm_map from a task + So I am just adding ipc_port_name to the ipc_port struct in this + case? + yes + actually + don't do anything just yet + we need to sort a few details out first + see bug-hurd + + # IRC, freenode, #hurd, 2011-07-13 A related issue: -- cgit v1.2.3