From eccdd13dd3c812b8f0b3d046ef9d8738df00562a Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Wed, 25 Sep 2013 21:45:38 +0200 Subject: IRC. --- .../gsoc/project_ideas/download_backends.mdwn | 11 +- community/gsoc/project_ideas/mtab/discussion.mdwn | 1167 +++++++++++++++++++- community/gsoc/project_ideas/object_lookups.mdwn | 29 + community/gsoc/project_ideas/sound/discussion.mdwn | 47 + 4 files changed, 1248 insertions(+), 6 deletions(-) create mode 100644 community/gsoc/project_ideas/sound/discussion.mdwn (limited to 'community/gsoc/project_ideas') diff --git a/community/gsoc/project_ideas/download_backends.mdwn b/community/gsoc/project_ideas/download_backends.mdwn index f794e814..c0bdc5b2 100644 --- a/community/gsoc/project_ideas/download_backends.mdwn +++ b/community/gsoc/project_ideas/download_backends.mdwn @@ -1,12 +1,12 @@ -[[!meta copyright="Copyright © 2009 Free Software Foundation, Inc."]] +[[!meta copyright="Copyright © 2009, 2013 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="Use Internet Protocol Translators (ftpfs etc.) as Backends for Other Programs"]] @@ -19,8 +19,9 @@ Download protocols like FTP, HTTP, BitTorrent etc. are very good candidates for this kind of modularization: a program could simply use the download functionality by accessing FTP, HTTP etc. translators. -There is already an ftpfs translator in the Hurd tree, as well as an [httpfs -translator on hurdextras](http://www.nongnu.org/hurdextras/#httpfs); however, +There is already an [[hurd/translator/ftpfs]] translator in the Hurd tree, as +well as an [[hurd/translator/httpfs]] on +[hurdextras](http://www.nongnu.org/hurdextras/); however, these are only suitable for very simple use cases: they just provide the actual file contents downloaded from the URL, but no additional status information that are necessary for interactive use. (Progress indication, error codes, HTTP diff --git a/community/gsoc/project_ideas/mtab/discussion.mdwn b/community/gsoc/project_ideas/mtab/discussion.mdwn index 0e322c11..716fb492 100644 --- a/community/gsoc/project_ideas/mtab/discussion.mdwn +++ b/community/gsoc/project_ideas/mtab/discussion.mdwn @@ -106,7 +106,7 @@ License|/fdl]]."]]"""]] # IRC, freenode, #hurd, 2013-06-25 -In context of [[microkernel/mach/mig/documentation/structured_data]]. +In context of [[open_issues/mig_portable_rpc_declarations]]. should I go for an iterator like interface instead? btw, what's the expected roundtrip time? @@ -905,3 +905,1168 @@ In context of [[microkernel/mach/mig/documentation/structured_data]]. ah, i think so then you don't need to do it again right, I overlooked that + + +## IRC, freenode, #hurd, 2013-07-12 + + recursively traversing all translators from / turns out to be + more dangerous than I expected + ... if done by a translator bound somewhere below /... + my interpretation is that the mtab translator tries to talk to + itself and deadlocks + (and as a side effect the whole system kinda just stops...) + + +## IRC, freenode, #hurd, 2013-07-15 + + teythoon: did you discuss with braunr about returning port vs path + in fsys_get_children? + youpi: we did + as I wrote I looked at the getcwd source you pointed me at + and I started to code up something similar + but as far as I can see there's no way to tell from a port + referencing a file the directory this file is located in + ah, right, there was a [0] mail + teythoon: because it doesn't have a "..", right + about Neals concerns, he's right about not covering passive + translators very well + but the solution he proposed was similar to what I tried to do + first + I don't like half-covering passive translators at all, to be honest + :) + either covering them completely, or not at all, would be fine + and then braunr convinced me that the "recursive" approach is + more elegant and hurdish, and I came to agree with him + youpi: one could scan the filesystem at translator startup and + populate the list + by "Neal's solution", you mean an mtab registry? + yes + so, let's see what linux does when renaming parent directories + mount points you mean? + teythoon: browsing the whole filesystem just to find passive + translators is costly + teythoon, braunr: and that won't prevent the user from unexpectedly + starting other translators at will + scary + youpi: but that requires the privilege to open the device + the fact that a passive translator is set is nothing more than a + user having the intent of starting a translator + linux retains the original path in the mount table + heh + youpi: any unprivileged user can trigger a translator startup + sure, but root can do that too + and expect the system to behave nicely + but if I'm root and want to fsck something, I won't start + translators accessing the device just before that + but if there's a passive translator targetting the device, + someone else might do that + root does not always completely control what he's doing + linux for instance does prevent from mounting a filesystem being + checked + but still, including passive translators in the list would at + least prevent anyone starting an translator by accident, isn't that worth + doing then? + if there's a way to prevent root too, that's better than having a + half-support for something which we don't necessarily really want + (i.e. an exclusive lock on the underlying device) + right, that would also do the trick + btw, some programs or scripts seem to hardcode /proc/mounts and + procfs and I cannot bind a translator to /proc/mounts since it is + read-only and the node does not exist + IMHO automatically starting translators is a generic feature, and + passive translator is just a specific instance of it; but we could very + well have, like an "autofs" that automatically start translators in tar + archives and iso images, allowing to cd into any tar/iso on the system; + implementing such things is part of the Hurd flexibility, the "core + system" shouldn't be too aware on how translators are started + so in the end, storing where the active translator was started + first seems okayish according to what linux has been exposing for decades + kilobug: indeed + it could serve a mounts with a passive translator by default, or + a link to /run/mtab, or an simple file so we could bind a translator to + that node + I'd tend to think that /proc/mounts should be a passive translator + and /run/mtab / /etc/mtab a symlink to it + not being to choose the translator is a concern however + ok, I'll look into that + it could be an empty file, and people be able to set a translator + on it + if it had a passive translator, people still could bind their + own translator to it later on, right? + afaics the issue currently is mostly, that there is no mounts + node and it is not possible to create one + right + cool + so with the actual path, you can even check for caller's permission + to read the path + i.e. not provide any more information than the user would be able + to get from browsing by hand + sure, that concern of Neil's is easy to address + I'm not so much concerned by stale paths being shown in mtab + the worst that can happen is a user not being able to umount the + path + but he can settrans -g it + (which he can't on linux ;) ) + yes, and the device information is still valid + yes + despite the parent dir being renamed, linux is still able to + umount the new path + and so is our current umount + good + (if one uses the mount point as argument) + what's the current plan concerning /proc/mounts ? + serving a node with a passive translator record + ? + so that /hurd/mtab / is started on access + i mean, still planning on using the recursive approach instead of + a registry ? + ah + I do not feel confident enough to decide this, but I agree with + you, it feels elegant + and it works :) + modulo the translator deadlocking if it talks to itself, any + thoughts on that? + it is a non-threaded translator I guess? + currently yes + making it threaded should fix the issue + I tried to make the mtab translator multithreaded but that + didn't help + that's odd + maybe I did it wrong + i don't find it surprising + well, not that surprising :p + on what lock does it block ? + as far as i can see the only difference of hello and hellot-mt + is that it uses a different dispatcher and has lot's of locking, right? + braunr: I'm not sure, partly because that wrecked havoc on the + whole system + it just freezes + but it wasn't permanent. once i let it running and it recovered + consider using a subhurd + ah right, I ment to set up one anyway, but my first attempts + were not successful, not sure why + anyway, is there a way to prevent this in the first place? + if one could compare ports that'd be helpful + Mmm, did you try to simply compare the number? + with the bootstrap port I presume? + Mmm, no, the send port and the receive port would be different + no, with the receive port + ah + comparing the numbers should work + youpi: no they should be the same + braunr: ah, then it should work yes + that's why there are user ref counts + ok + only send-once rights have their own names + btw, I'll push my work to darnassus from now on, + e.g. http://darnassus.sceen.net/gitweb/?p=teythoon/hurd.git;a=shortlog;h=refs/heads/feature-mtab-translator-v3-wip + + +## [[open_issues/libnetfs_passive_translators]] + + +## IRC, freenode, #hurd, 2013-07-16 + + which port is the receive port of a translator? I mean, how is + it called in the source, there is no port in sight named receive anywhere + I looked. + teythoon: what is the "receive port of a translator" ? + braunr: we talked yesterday about preventing the mtab deadlock + by comparing ports + I asked which one to use for the comparison, youpi said the + receive port + i'm not sure what he meant + it could be the receive port used for the RPC + but i don't think it's exported past mig stub code + weird, I just reread it. I asked if i should use the bootstrap + port, and he said receive port, but it might have been addressed to you? + you were talking about send and receive ports being singletons + or not + umm + no i answered him + he was wondering if the receive port could actually be used for + comparison + i said it can + but still, i'm not sure what port + if it's urgent, send him a mail + no, my pipeline is full of stuff I can do instead ;) + :) + + +## IRC, freenode, #hurd, 2013-07-17 + + braunr: btw, comparing ports solved the deadlock in the mtab + translator rather easily + :) + which port then ? + currently I'm stuck though, I'm not sure how to address Neals + concern wrt to access permission checks + I believe it's called control port + ok + the one one gets from doing the handshake with the parent + i thought it was the bootstrap port + but i don't know the details so i may be wrong + anyway + yes + what is the permission problem again ? + 871u73j4zp.wl%neal@walfield.org + well, you could perform a lookup on the stored path + as if opening the node + if I look at any server implementation of a procedure from + fs.defs (say libtrivfs/file-chmod.c [bad example though, that looks wrong + to me]), there is permission checking being done + any server implementation of a procedure from fsys.defs lacks + permission checks, so I guess it's being done somewhere else + i must say i'm a bit lost in this discussion + i don't know :/ + can *you* sum up the permission problem please ? + i mean here, now, in just a few words ? + ok, so I'm extending the fsys api with the get_children + procedure + that one should not return any children x/y if the user doing + the request has no read permissions on x + really ? + why so ? + the same way ls x would not reveal the existence of y + i could also say unlike cat /proc/mounts + i can see why we would want that + i also can see why we could let this behaviour in place + let's admit we do want it + true, but I thought this could easily be addressed + what you could do is + now I'm not sure b/c I cannot even find the permission checking + code for any fsys_* function + for each element in the list of child translators + perform a lookup on the stored path on behalf of the user + and add to the returned list if permission checks pass + teythoon: note that i said lookup on the path, which is an fs + interface + i assume there is no permission checking for the fsys interface + because it's done at the file (fs) level + i think so too, yes + sure, if I only knew who made the request in the first place + the file-* options have a convenient credential handle passed in + as first parameter + s/options/procedures/ + surely the fsys-* procedures also have a means of retrieving + that information, I just don't know how + mig magic + teythoon: see file_t in hurd_types.defs + there is the macro FILE_INTRAN which is defined in subdirectories + (or not) + ah, retrieving the control port requires permissions, and the + fsys-* operations then operate on the control port? + see libdiskfs/fsmutations.h for example + uh yes but that's for < braunr> i assume there is no permission + checking for the fsys interface because it's done at the file (fs) level + i'm answering < teythoon> sure, if I only knew who made the + request in the first place + teythoon: do we understand each other or is there still something + fuzzy ? + braunr: thanks for the pointers, I'll read up on that a bit + later + teythoon: ok + + +## IRC, freenode, #hurd, 2013-07-18 + + braunr: back to the permission checking problem for the + fsys_get_children interface + I can see how this could be easily implemented in the mtab + translator, it asks the translator for the list of children and then + checks if the user has permission to read the parent dir + but that is pointless, it has to be implemented in the + fsys_get_children server function + yes + why is it pointless ? + because one could circumvent the restriction by doing the + fsys_get_children call w/o the mtab translator + uh no + you got it wrong + what i suggested is that fsys_get_children does it before + returning a list + the problem is that the mtab translator has a different identity + from the users accessing it + yes, but I cannot see how to do this, b/c at this point I do not + have the user credentials + get them + how? + 16:14 < braunr> mig magic + 16:15 < braunr> teythoon: see file_t in hurd_types.defs + 16:16 < braunr> there is the macro FILE_INTRAN which is defined in + subdirectories (or not) + 16:16 < braunr> see libdiskfs/fsmutations.h for example + i saw that + is there a problem i don't see then ? + i suppose you should define FSYS_INTRAN rather + but the idea is the same + won't that change all the function signatures of the fsys-* + family? + that's probably the only reason not to implement this feature + right now + then again, that change is probably easy and mechanic in nature, + might be an excuse to play around with coccinelle + why not + if you have the time + right, if this can be done, the mtab translator (if run as root) + could get credentials matching the users credentials to make that + request, right? + i suppose + i'm not sure it's easy to make servers do requests on behalf of + users on the hurd + which makes me wonder if the mtab functionality shouldn't be + implemented in glibc eheheh .... + but probably not + well, I'll try out the mig magic thing and see how painful it is + to fix everything ;) + good luck + honestly, i'm starting to think it's deviating too much from your + initial goal + i'd be fine with a linux-like /proc/mounts + with a TODO concerning permissions + ok, fine with me :) + confirm it with the other mentors please + we have to agree quickly on this + y? + + braunr: I actually believe that the permission issue can be + addressed cleanly and unobstrusively + braunr: would you still be opposed to the get_children approach + if that is solved? + the filesystem is a tree and the translators "creating" that + tree are a more coarse version of that tree + having a method to traverse that tree seems natural to me + teythoon: it is natural + i'm just worried it's a bit too complicated, unnecessary, and + out-of-scope for the problem at hand + (which is /proc/mounts, not to forget it) + + +## IRC, freenode, #hurd, 2013-07-19 + + braunr: I think you could be a bit more optimistic and + supportive of the decentralized approach + I know the dark side has cookies and strong language and it's + mighty tempting + but both are bad for you :p + + +## IRC, freenode, #hurd, 2013-07-22 + + teythoon: AIUI, you should be able to run the mtab translator as + no-user (i.e. no uid) + youpi: yes, that works fine + + teythoon: so there is actually no need to define FSYS_INTRAN, doing + it by hand as you did is fine, right? + (/me backlogs mails...) + youpi: yes, the main challenge was to figure out what mig does + and how the cpp is involved + heh :) + my patch does exactly the same, but only for this one server + function + youpi: I'm confused by your mail, why are read permissions on + all path components necessary? + teythoon: only execution normally + teythoon: to avoid letting a user discover a translator running on + a hidden directory + braunr: exactly, and that is tested + e.g. ~/home/foo is o+x, but o-r + and I have a translator running on ~/home/foo/aZeRtYuyU + I don't want that to show up on /proc/mounts + youpi: i don't understand either: why isn't execution permission + enough ? + youpi: but that requires testing for read on the *last* + component of the *dirname* of your translator, and that is tested + let me take another example :) + e.g. ~/home/foo/aZeRtYuyU is o+x, but o-r + and I have a translator running on ~/home/foo/aZeRtYuyU/foo + ergl sorry, I meant this actually: + yes, that won't show up then in the mtab for users that are not + you and not root + e.g. ~/home/foo is o+x, but o-r + and I have a translator running on ~/home/foo/aZeRtYuyU/foo + ah + hmm, good point + ? + * braunr still confused + well, qwfpgjlu is the secret + and that is revealed by the fsys_get_children procedure + then i didn't understand the description of the call right + > + /* check_access performs the same permission check as is + normally + > + done, i.e. it checks that all but the last path components + are + > + executable by the requesting user and that the last + component is + > + readable. */ + braunr: youpi argues that this is not enough in this case + from that, it looks ok to me + the function and the documentation agree, yes + but that's not what we want + and that's where i fail to understand + again, see my example + i am + 10:43 < youpi> e.g. ~/home/foo is o+x, but o-r + ok + so the user is not supposed to find out the secret + then your example isn't enough to describe what's wron + g + checking read permission only on ~/home/foo/aZeRtYuyU will not + garantee that + ah + i thought foo was the last component + no, that's why I changed my example + hum + 10:43 < youpi> e.g. ~/home/foo is o+x, but o-r + 10:43 < youpi> and I have a translator running on + ~/home/foo/aZeRtYuyU/foo + i meant, the last foo + still, this is easily fixed + sure + just has to be :) + youpi, braunr: so do you think that this approach will work? + I believe so + i still don't see the problem, so don't ask me :) + i've been sick all week end and hardly slept, which might explain + in the example, "all but the last path components" is + "~/home/foo/aZeRtYuyU" + right ? + braunr: well, I haven't looked at the details + but be it the last, or but-last doesn't change the issue + if my ~/hidden is o-r,o+x + and I have a translator on ~/hidden/a/b/c/d/e + checking only +x on hidden is not ok + but won't the call also check a b c d ? + yes, but that's not what matters + what matters is that hidden is o-r + hm + so the mtab translator is not supposed to reveal that there is an + "a" in there + ok i'm starting to understand + so r must be checked on all components too + yes + right + to simulate the user doing ls, cd, ls, cd, etc. + well, not cd + ah + for being able to do ls, you have to be able to do cd + as an ordered list of commands + ok + agreed. can you think of any more issues? + so both x and r must be checked + so in the end this RPC is really a shortcut for a find + fsysopts + script + teythoon: I don't see any + teythoon: i couldn't take a clear look at the patch but + do you perform a lookup on all nodes ? + yes, all nodes on the path from the root to the one specified by + the mount point entry in the active translator list + let me rephrase + do you at some point do a lookup, similar to a find, on all nodes + of a translator ? + no + good + yes + iirc, neal raised that concern once + and I'll also fix settrans --recursive not to iterate over *all* + nodes either + great + :) + fsys_set_options with do_children=1 currently does that (I've + only looked at the diskfs version) + + +## IRC, freenode, #hurd, 2013-07-27 + + youpi: ah, I just found msg_get_init_port, that should make the + translator detection feasible + + +## IRC, freenode, #hurd, 2013-07-31 + + braunr: can I discover the sender of an rpc message? + teythoon: no + teythoon: what do you mean by "sender" ? + braunr: well, I'm trying to do permission checks in the + S_proc_mark_essential server function + ok so, the sending user + that should be doable + I've got a struct proc *p courtesy of a mig intran mutation and + a port lookup + but that is not necessarily the sender, right? + proc is really the server i know the least :/ + there is permission checking for signals + it does work + you should look there + yes, there are permission checks there + but the only argument the rpc has is a mach_port_t refering to + an object in the proc server + yes + anyone can obtain such a handle for any process, no? + can you tell where it is exactly please ? + i don't think so, no + what? + 14:42 < teythoon> but the only argument the rpc has is a + mach_port_t refering to an object in the proc server + ah + the code you're referring to + a common way to give privileges to public objects is to provide + different types of rights + a public (usually read-only) right + and a privileged one, like host_priv which you may have seen + acting on (modifying) a remote object normally requires the latter + http://paste.debian.net/20795/ + i thought you were referring to existing code + well, there is existing code doing permission checks the same + way I'm doing it there + where is it please ? + mgt.c ? + proc/mgt.c (S_proc_setowner) for example + yes + that's different + but anyone can obtain such a reference by doing proc_pid2proc + the sender is explicitely giving the new uid + yes but not anyone is already an owner of the target process + (although it may look like anyone has the right to clear the owner + oO) + see, that's what made me worry, it is not checked who's the + sender of the message + unless i'm missing something here + ah + I am + pid2proc returns EPERM if one is not the owner of the process in + question + all is well + ok + it still requires the caller process though + what? + see check_owner + the only occurrence i find in the hurd is in libps/procstat.c + MGET(PSTAT_PROCESS, PSTAT_PID, proc_pid2proc (server, ps->pid, + &ps->process)); + server being the proc server AIUI + yes, most likely + but pid2proc describes this first argument to be the caller + process + ah but it is + ? + mig magic :p + MIGSFLAGS="-DPROCESS_INTRAN=pstruct_t reqport_find (process_t)" + \ + MIGSFLAGS="-DPROCESS_INTRAN=pstruct_t reqport_find (process_t)" + \ + ah nice + hum no + this just looks up the proc object from a port name, which is + obvious + what i mean is + 14:53 < braunr> MGET(PSTAT_PROCESS, PSTAT_PID, proc_pid2proc + (server, ps->pid, &ps->process)); + this is done in libps + which can be used by any process + server is the proc server for this process (it defines the process + namespace) + yes, but isn't the port to the proc server different for each + process? + no, the port is the same (the name changes only) + ports are global non-first class objects + and the proc server can thus tell with the lookup which process + it is talking to? + that's the thing + from pid2proc : + S_proc_pid2proc (struct proc *callerp + [...] + if (! check_owner (callerp, p)) + check_owner (struct proc *proc1, struct proc *proc2) + "Returns true if PROC1 has `owner' privileges over PROC2 (and can + thus get its task port &c)." + callerp looks like it should be the caller process + but in libps, it seems to be the proc server + this looks strange to me + yep, to me too, hence my confusion + could be a bug that allows anyone to perform pid2proc + braunr: well, proc_pid2proc (getproc (), 1, ...) fails with + EPERM as expected for me + ofc it does with getproc() + but what forces a process to pass itself as the first argument ? + braunr: nothing, but what else would it pass there? + 14:53 < braunr> MGET(PSTAT_PROCESS, PSTAT_PID, proc_pid2proc + (server, ps->pid, &ps->process)); + everyone knows the proc server + ok now, that's weird + teythoon: does getproc() return the proc server ? + I think so, yes + damn those distributed systems, all of their sources are so + distributed too + i suspect there is another layer of dark glue in the way + I cannot even find getproc :/ + hurdports.c:GETSET (process_t, proc, PROC) + that's the dark glue :p + ah, so it must be true that the ports to the proc server are + indeed process specific, right? + ? + well, it is not one port to the proc server that everyone knows + it is + what makes you think it's not ? + proc_pid2proc (getproc (), 1, ...) fails with EPERM for anyone + not being root, but succeeds for root + hm right + if getproc () were to return the same port, the proc server + couldn't distinguish these + indeed + in which case getproc() actually returns the caller's process + object at its proc server + yes, that is better worded + teythoon: i'm not sure it's true actually :/ + braunr: well, exploit or it didn't happen + teythoon: getproc() apparently returns a bootstrap port + we must find the code that sets this port + i have a hard time doing that :/ + isn't part of the stuff which is passed to a new process by + exec? + braunr: I know that feeling + pinotree: probably + still hard to find .. + search in glibc + braunr: exec/exec.c:1654 asks the proc server for the proc + object to use for the new process + so how much of hurd do I have to rebuild once i changed struct + procinfo in hurd_types.h? + oh noez, glibc uses it too :/ + + +## IRC, freenode, #hurd, 2013-08-01 + + I need some pointers on building the libc, specifically how to + point libcs build system to my modified hurd headers + nlightnfotis: hi + nlightnfotis: you rebuild the libc right? do you have any hurd + specific pointers for doing so? + teythoon, I have not yet rebuild the libc (I was planning + to, but I followed other courses of action) Thomas had pointed me to some + resources on the Hurd website. I can look them up for you + teythoon, here are the instructions + http://darnassus.sceen.net/~hurd-web/open_issues/glibc/debian/ + and the eglibc snapshot is here + http://snapshot.debian.org/package/eglibc/ + nlightnfotis: yeah, I found those. the thing is I changed a + struct in the hurd_types.h header, so now I want to rebuild the libc with + that header + and I cannot figure out how to point libcs build system to my + hurd headers + :/ + can you patch eglibc and build that one instead? + teythoon: put your header in the appropriate /usr/include/ dir + pochu: is there no other way? + iirc nope + teythoon: you may be able to pass some flag to configure, but I + don't know if that will work in this specific case + ouch >,< that explains why I haven't found one + check ./configure --help, it's usually FOO_CFLAGS (so something + like HURD_CFLAGS maybe) + but then you may need _LIBS as well depending on how you changed + the header... so in the end it's just easier to put the header in + /usr/include/ + teythoon: did you find the info for your libc build ? + braunr: well, i firmlinked my hurd_types.h into /usr/include/... + ew + i recommend building debian packages + but the build was not successful, looks unrelated to my changes + though + I tried that last week and the process took more than eight + hours and did not finish + use darnassus + it takes about 6 hours on it + I shall try again and skip the unused variants + i also suggest you use ./debian/rules build + and then interrupt the build process one you see it's building + object files + go to the hurd-libc-i386 build dir, and use make lib others + make lib builds libc, others is for companion libraries lik + libpthread + actually building libc takes less than an hour + so once you validate your build this way, you know building the + whole debian package will succedd + succeed* + so how do I get the build system to pick up my hurd_types.h? + sorry if this is obvious to you, you might be more familiar with + debian than i am :) + patch the hurd package + append your own version string like +teythoon.hurd.1 + install it + then build libc + i'll reboot darnassus so you have a fresh and fast build env + almost a month of uptime without any major issue :) + err, but I cannot install my hurd package on darnassus, can I? I + don't think that'd be wise even if it were possible + teythoon: rebooted, enjoy + why not ? + i often do it for my own developments + teythoon: screen is normally available + teythoon: be aware that fakeroot-tcp is known to hang when pfinet + is out of ports (that's a bug) + it takes more time to reach that bug since a patch that got in + less than a year ago, but it still happens + the hurd packages are quick to build, and they should only provide + the new header, right ? + you can include the functionality too in the packages if you're + confident enough + but my latest work on the killing of essential processes issues + involves patching hurd_types.h and that in a way that breaks the ABI, + hence the need to rebuild the libc (afaiui) + teythoon: yes, this isn't uncommon + braunr: this is much more intrusive than anything I've done so + far, so I'm not so confident in my changes for now + teythoon: show me the patch please + braunr: it's not split up yet, so kind of messy: + http://paste.debian.net/21403/ + teythoon: did you make sure to add RPCs at the end of defs files ? + yes, I got burned by this one on my very first attempt, you + pointed out that mistake + :) + ok + you're changing struct procinfo + this really breaks the abi + yes + i.e. you can't do that + I cannot put it at the end b/c of that variable length array + you probably should add another interface + that'd be easier, sure, but this will slow down procfs even + more, no? + that's secondary + it won't be easier, breaking the abi may break updates + in which case it's impossible + another way would be to ues a new procinfo struct + like struct procinfo2 + but then you need a transition step so that all users switch to + that new version + which is the best way to deal with these issues imo, but this time + not the easiest :) + ok, so I'll introduce another rpc and make sure that one is + extensible + hum no + this usually involves using a version anyway + no? but it is likely that we need to save more addresses of this + kind in the future + in which case it will be hanlded as an independant problem with a + true solution such as the one i mentioned + it could return an array of vm_address_ts with a length + indicating how many items were returned + it's ugly + the code is already confusing enough + keep names around for clarity + ok, point taken + really, don't mind additional RPCs when first adding new features + once the interface is stable, a new and improved version becomes a + new development of its own + you're invited to work on that after gsoc :) + but during gsoc, it just seems like an unnecessary burden + ok cool, I really like that way of extending Hurd, it's really + easy + and feels so natural + i share your concern about performances, and had a similar problem + when adding page cache information to gnumach + in the end, i'll have to rework that again + because i tried to extend it beyond what i needed + true, I see how that could happen easily + the real problem is mig + mig limits subsystems to 100 calls + it's clearly not enough + in x15, i intend to use 16 bits for subsystems and 16 bits for + RPCs, which should be plenty + that limit seems rather artificial, it's not a power of two + yes it is + so let's fix it + mach had many artificial static limits + eh :D + not easy + replies are encoded by taking the request ID and adding 100 + uh + "uh" indeed + so we need an intermediate version of mig that accepts both + id+100 and dunno id+2^x as replies for id + or -id - 1 + that would completely break the abi + braunr: how so? the change would be in the *_server functions + and be compatible with the old id scheme + how do you make sure id+2^x doesn't conflict with another id ? + oh, the id is added to the subsystem id? + to obtain a global message id? + yes + ah, I see + ah, but the hurd subsystems are 1000 ids apart + so id+100 or id +500 would work + we need to make sure it's true + always true + so how many bits do we have for the message id in mach? + (mig?) + mach shouldn't care, it's entirely a mig thing + well yes and no + mach defines the message header, which includes the message id + see mach/message.h + mach_msg_id_t msgh_id; + typedef integer_t mach_msg_id_t; + well, if that is like a 32 bit integer, then allow -id-1 as + reply and forbid ids > 2^x / 2 + yes + seems reasonable + that'd give us an smooth upgrade path, no? + i think so + + +## IRC, freenode, #hurd, 2013-08-28 + + teythoon: Mmm, your patch series does not make e.g. ext2fs provide + a diskfs_get_source, does it? + + +## IRC, freenode, #hurd, 2013-08-29 + + youpi: that is correct + teythoon: Mmm, I must be missing something then: as such the patch + series introduces an RPC, but only EOPNOTSUPP is ever returned in all + cases for now? + ah + /* Guess based on the last argument. */ + since ext2fs & such report their options with store last, it seems + ok indeed + it still seems a bit lame not to return that information in + get_source + yes + well, if it had been just for me, I would not have created that + rpc, but only guessing was frowned uppon iirc + then again, maybe this should be used and then the mtab + translator could skip any translators that do not provide this + information to filter out non-"filesystem" translators + guessing is usually trap-prone, yes + if it is to be used by mtab, then maybe it should be documented as + being used by mtab + otherwise symlink would set a source, for instance + while we don't really want it here + why would the symlink translator answer to such requests? it is + not a filesystem-like translator + no, but the name & documentation of the RPC doesn't tell it's only + for filesystem-like translators + well, the documentation does say "filesystem" + but it does not clearly specify that one shouldn't implement + get_source if one is not a filesystme + "If the concept of a source is applicable" works for a symlink + that could be the same for eth-filter, etc. + right + Mmm, that said it's fsys.defs + not io.defs + teythoon: it is the fact that we get EOPNOTSUPP (i.e. fsys + interface supported, just not that call), and not MIG_BAD_ID (i.e. fsys + interface not supported), that filters out symlink & such, right? + that's what I was thinking, but that's based on my + interpretation of EOPNOPSUPP of course ;) + teythoon: I believe that for whatever is a bit questionable, even + if you put yourself on the side that people will probably agree on, the + discussion will still take place so we make sure it's the right side :) + (re: start/end_code) + I'm not sure I follow + youpi: /proc/pid/stat seems to be used a lot: + http://codesearch.debian.net/search?q=%22%2Fproc%2F.*%2Fstat%22 + that does not mean that start/endcode is used, but still it + seems like a good thing to mimic Linux closely + stat is used a lot for cpu usage for instance, yes + start/endcode, I really wonder who is using it + using it for kernel thread detection looks weird to me :) + (questionable): I mean that even if you take the time to put + yourself on the side that people will probably agree on, the discussion + will happen + it has to happen so people know they agree on it + I've seen that a lot in various projects (not only CS-related) + ok, I think I got it + it's to document the reasons for (not) doing something? + something like this, yes + even if you look right, people will try to poke holes + just to make sure :) + btw, I think it's rather unusual that our storeio experiments + would produce such different results + you're right about the block device, no idea why I got a + character file there + I used settrans -ca /tmp/hello.unzipped /hurd/storeio -T + gunzip:file /tmp/hello + also I tried stacking the translator on /tmp/hello directly, + from what I've gathered that should be possible, but I failed + ftr I use the exec server with all my patches, so the unzipping + code has been removed from it + ah, I probably still have it + it shouldn't matter here, though + I agree + how would you stack it? + I've never had a look at that + I'm not sure attaching the translator to the node is done before or + after the translator has a change to open its target + right + but it could be done, if storeio used the reference to the + underlying node, no? + yes + btw, you had said at some point that you had issues with running + remap. Was the issue what you fixed with your patches? + * youpi realizes that he should have shown the remap.c source code during + his presentation + well, I tried to remap /servers/exec (iirc) and that failed + then again, I recently played with remap and all seemed fine + but I'm sure it has nothing to do with my patches + ok + those I came up with investigating fakeroot-hurd + and I saw that this also aplies to remap.sh + *while + yep, they're basically the same + btw, I somehow feel settrans is being abused for chroot and + friends, there is no translator setting involved + chroot, the command? or the settrans option? + I don't understand what you are pointing at + the settrans option being used by fakeroot, remap and (most + likely) our chroot + our chroot is just a file_reparent call + fakeroot and remap do start a translator + yes, but it is not being bound to a node, which is (how I + understand it) what settrans does + the point being that if settrans is being invoked with --chroot, + it does something completely different (see the big if (chroot) {...} + blocks) + to a point that it might be better of in a separate command + Mmm, indeed, a lot of the options don't make sense for chroot + + +## IRC, freenode, #hurd, 2013-09-06 + + teythoon: do you personally prefer /proc being able to implement + /proc/self on its own, or using the magic server to tell clients to + resolve those specific cases themselves ? + imho solving the "who's the sender of an rpc" could solve both + the SCM_CREDS implementation and the self case in procfs + +[[open_issues/SENDMSG_SCM_CREDS]], +[[hurd/translator/procfs/jkoenig/discussion]], *`/proc/self`*. + + pinotree: yes + but that would require servers impersonating users to some extent + and this seems against the hurd philosophy + and there was also the fact that you could create a + fake/different port when sending an rpc + to fake what ? + the sender identiy + *identity + what ? + you mean intermediate servers can do that + braunr: I don't know if I understand all the implications of + your question, but the magic server is the only hurd server that actually + implements fsys_forward (afaics), so why not use that? + teythoon: my question was rather about the principle + do people find it acceptable to entrust a server with their + authority or not + on the hurd, it's clearly wrong + but then it means you need special cases everywhere, usually + handled by glibc + and that's something i find wrong too + it restricts extensibility + the user can always change its libc at runtime, but in practice, + it's harder to perform than simply doing it in the server + braunr: then I think I didn't get the question at all + teythoon: it's kind of the same issue that you had with the mtab + translator + about showing or not some entries the user normally doesn't have + access to + this problem occurs when there is more than one server on the + execution path and the servers beyond the first one need credentials to + reply something meaningful + the /proc/self case is a perfect one + (conceptually, it's client -> procfs -> symlink) + 1/ procfs tells the client it needs to handle this specially, + which is what the hurd does with magic + 2/ procfs assumes the identity of the client and the symlink + translator can act as expected because of that + teythoon: what way do you find better ? + braunr: by "procfs assumes the identity" you mean procfs + impersonating the user? + yes + braunr: tbh I still do not see how this can be implemented at + all b/c the /proc/self symlink is not about identity (which can be + derived from the peropen struct initially created by fsys_getroot) but + the pid of the callee (which afaics is nowhere to be found) + s/callee/caller/ + the one doing the rpc + impersonating the user isn't only about identity + actually, it's impersonating the client + yes, client is the term >,< + so basically, asking proc about the properties of the process + being impersonated + proc o_O + it's not hard, it's just a big turn in the way the system would + function + teythoon: ? + you lost me somewhere + the client is the process + not the user + in order to implement /proc/self properly, one has to get the + process id of the process doing the /proc/self lookup, right? + yes + actually, we would even slice it more and have the client be a + thread + so how do you get to that piece of information at all? + the server inherits a special port designating the client, which + allows it to query proc about its properties, and assume it's identity in + servers such as auth + its* + ah, but that kind of functionality isn't there at the moment, is + it? + it's not, by design + right, hence my confusion + instead, servers use the magic translator to send a "retry with + special handling" message to clients + right, so the procfs could bounce that back to the libc handler + that of course knows its pid + yes + right, so now at last I got the whole question :) + :) + ugh, I just found the FS_RETRY_MAGICAL handler in the libc :-/ + ? + why "ugh" ? + well, I'm inclined to think this is the bad kind of magic ;) + do i need to look at the code to understand ? + ok, so I think option 1/ is easily implemented, option 2/ has + consequences that I cannot fully comprehend + same for me + no, but you yourself said that you do not like that kind of + logic being implemented in the libc + well + easily + i'm not so sure + it's easy to code, but i assume checking for magic replies has its + cost + why not? the code is doing a big switch over the retryname + supplied by the server + we could stuff getpid() logic in there + 14:50 < braunr> it's easy to code, but i assume checking for magic + replies has its cost + what kind of cost? computational cost? + yes + the big switch you mentioned + run every time a client gets a reply + (unless i'm mistaken) + a only for RETRY_MAGICAL replies + but you need to test for it + switch (retryname[0]) + { + case '/': + ... + that should compile to a jump table, so the cost of adding + another case should be minimal, no? + yes + but + it's even less than that + the real cost is checking for RETRY_MAGICAL + 14:55 < teythoon> a only for RETRY_MAGICAL replies + so it's basically a if + one if, right ? + no, it's switch'ing over doretry + you should pull up the code and see for yourself. it's in + hurd/lookup-retry.c + ok + well no, that's not what i'm looking for + it's not o_O + i'm looking for what triggers the call to lookup_retry + teythoon: hm ok, it's for lookups only, that's decent + teythoon: 1/ has the least security implications + yes + it could slightly be improved with e.g. a well defined interface + so a user could preload a library to extend it + extend the whole magic lookup thing? + yes + but that is no immediate concern, you are trying to fix + /proc/self, right? + no, i'm thinking about the big picture for x15/propel, keeping the + current design or doing something else + oh, okay + solving /proc/self looks actually very easy + well, I'd say this depends a lot on your trust model then + do you consider servers trusted? + (btw, will there be mutual authentication of clients/servers in + propel?) + there were very interesting discussions about that during the + l4hurd project + iirc, shapiro insisted that using a server without trusting it + (and there were specific terminology about trusting/relying/etc..) is + nonsense + teythoon: i haven't thought too much about that yet, for now it's + supposed to be similar to what the hurd does + hm, then again trust is not an on/off thing imho + ? + trusting someone to impersonate yourself is a very high level of + trust + s/is/requires/ + the mobile code paper suggests that mutual authentication might + be a good thing, and I tend to agree + i'll have to read that again + teythoon: for now (well, when i have time to work on it again + .. :)) + i'm focusing on the low level stuff, in a way that won't disturb + such high level features + teythoon: have you found something related to a thread-specific + port in the proc server ? + hurd/process.defs:297: /* You are not expected to understand + this. */ + \o/ + braunr: no, why would I (the thread related question) + braunr: yes, that comment also cought my eye :/ + teythoon: because you read a lot of the proc code lately + so maybe your view of it is better detailed than mine + + +## IRC, freenode, #hurd, 2013-09-13 + + * youpi crosses fingers + yay, still boots + teythoon: I'm getting a few spurious entries in /proc/mounts + none /servers/socket/26 /hurd/pfinet interface=/dev/eth0, etc. + /dev/ttyp0 /dev/ttyp0 /hurd/term name,/dev/ptyp0,type,pty-master 0 + 0 + /dev/sd1 /dev/cons ext2fs + writable,no-atime,no-inherit-dir-group,store-type=typed 0 0 + fortunately mount drops most of them + but not /dev/cons + spurious entries in df are getting more and more common on linux + too anyway... + ah, after a console restart, I don't have it any more + I'm getting df: `/dev/cons': Operation not supported instead + + +## IRC, freenode, #hurd, 2013-09-16 + + teythoon: e2fsck does not seem to be seeing that a given filesystem + is mounted + /dev/sd0s1 on /boot type ext2 (rw,no-inherit-dir-group) + and still # e2fsck -C 0 /dev/sd0s1 + e2fsck 1.42.8 (20-Jun-2013) + /dev/sd0s1 was not cleanly unmounted, check forced. + (yes, both /etc/mtab and /run/mtab point to /proc/mounts) + Yes, that is a "known" problem. + tschwinge: no, it's supposed to be fixed by the mtab translator :) + youpi: glibc's paths.h points to /var/run/mtab (for us) + youpi: Oh. But this is by means of mtab presence, and not by + proper locking? (Which is at least something, of course!) + /var/run points to /run + tschwinge: yes + anyway, got to run + + +## IRC, freenode, #hurd, 2013-09-20 + + teythoon: how come i see three mtab translators running ? + 6 now oO + looks like df -h spawns a few every time + yes, weird... + accessing /proc/mounts does actually + teythoon: more bug fixing for you :) + + +## IRC, freenode, #hurd, 2013-09-23 + + so it might be a problem with either libnetfs (which afaics has + never supported passive translator records before) or procfs, but tbh I + haven't investigated this yet diff --git a/community/gsoc/project_ideas/object_lookups.mdwn b/community/gsoc/project_ideas/object_lookups.mdwn index 5075f783..88ffc633 100644 --- a/community/gsoc/project_ideas/object_lookups.mdwn +++ b/community/gsoc/project_ideas/object_lookups.mdwn @@ -40,3 +40,32 @@ accurate measurements in a system that lacks modern profiling tools would also be helpful. Possible mentors: Richard Braun + + +# IRC, freenode, #hurd, 2013-09-18 + +In context of [[!message-id "20130918081345.GA13789@dalaran.sceen.net"]]. + + braunr: (wrt the gnumach HACK) funny, I was thinking about doind + the same for userspace servers, renaming ports to the address of the + associated object, saving the need for the hash table... + teythoon: see + http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/object_lookups/ + teythoon: my idea is to allow servers to set a label per port, + obtained at mesage recv time + because, yes, looking up an object twice is ridiculous + you normally still want port names to be close to 0 because it + allows some data structure optimizations + braunr: yes, I feared that ports should normally be smallish + integers and contigious at best + braunr: interesting that you say there that libihash suffers + from high collision rates + I've a theory to why that is, libihash doesn't do any hashing at + all + there are notes about that in the open_issues section of the + wiki + but I figured that this is probably ok for port names, as they + are small and contigious + braunr: That's called protected payload. + braunr: The idea is that the kernel appends data to the message in + flight. diff --git a/community/gsoc/project_ideas/sound/discussion.mdwn b/community/gsoc/project_ideas/sound/discussion.mdwn new file mode 100644 index 00000000..4a95eb62 --- /dev/null +++ b/community/gsoc/project_ideas/sound/discussion.mdwn @@ -0,0 +1,47 @@ +[[!meta copyright="Copyright © 2013 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!taglink open_issue_documentation]]: update [[sound]] page. + + +# IRC, freenode, #hurd, 2013-09-01 + + I'm new to the hurd but I'd love to learn enough to work on sound + support. + + http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/sound/ + says drivers should be ported to GNU Mach as a first step. + Is this information still current or should the existing Linux + driver be wrapped with DDE instead? + if i recall correctly dde is currently only being used for + network drivers. i'm not sure how much work would be involved for sound + or usb + + +## IRC, freenode, #hurd, 2013-09-02 + + The sound support proposal + (http://darnassus.sceen.net/~hurd-web/community/gsoc/project_ideas/sound/) + recommends porting some other kernel's sound driver to GNU Mach. Is this + still current or should DDE be used instead? + rekado: dde or anything userspace-based is generally preferred + rekado: both are about porting some other kernel's sound driver + dde is preferred yes + This email says that sound drivers are already partly working with + DDE: http://os.inf.tu-dresden.de/pipermail/l4-hackers/2009/004291.html + So, should I just try to get some ALSA kernel parts to compile + with DDE? + well, what is missing is also the dde←→hurd glue + rekado: there is also a problem with pci arbitration + pinotree: I assumed DDEKit works with the hurd and we could use + any DDE/ glue code with it + * rekado looks up pci arbitration + only for networking atm + ah, I see. -- cgit v1.2.3