From c4ad3f73033c7e0511c3e7df961e1232cc503478 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Wed, 26 Feb 2014 12:32:06 +0100 Subject: IRC. --- hurd/translator/mtab/discussion.mdwn | 552 ++++++++++++++++++++++++++++++++++- 1 file changed, 550 insertions(+), 2 deletions(-) (limited to 'hurd/translator/mtab') diff --git a/hurd/translator/mtab/discussion.mdwn b/hurd/translator/mtab/discussion.mdwn index 973fb938..ef5e8cbd 100644 --- a/hurd/translator/mtab/discussion.mdwn +++ b/hurd/translator/mtab/discussion.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2008, 2009, 2013 Free Software Foundation, +[[!meta copyright="Copyright © 2008, 2009, 2013, 2014 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -18,7 +18,7 @@ mounted file systems. This also means that commands like `df` can only work on explicitly specified mountpoints, instead of displaying the usual listing. One possible solution to this would be for the translator startup mechanism to -update the `mtab` on any `mount`/`unmount`, like in traditional systems. +update the `mtab` on any `mount`/`umount`, like in traditional systems. However, there are some problems with this approach. Most notably: what to do with passive translators, i.e., translators that are not presently running, but set up to be started automatically whenever the node is accessed? Probably @@ -1110,6 +1110,12 @@ In context of [[open_issues/mig_portable_rpc_declarations]]. e.g. http://darnassus.sceen.net/gitweb/?p=teythoon/hurd.git;a=shortlog;h=refs/heads/feature-mtab-translator-v3-wip +### IRC, freenode, #hurd, 2013-11-20 + + teythoon: ah thanks for making mtab multithreaded :) + i forgot about that + + ## [[open_issues/libnetfs_passive_translators]] @@ -2587,6 +2593,548 @@ In context of [[open_issues/mig_portable_rpc_declarations]]. yes +### IRC, freenode, #hurd, 2013-11-06 + + by the way, did you fix mtab for passive translators ? + or make any progress ? + I just cleaned up the patch series + ah good + I'm still trying to decide whether I leak any ports + + +### IRC, freenode, #hurd, 2013-11-15 + + btw, I haven't forgotten about the passive translator not + showing up in /proc/mounts + I have a patch series, the first patch seems fine, but if I + build hurd packages with the other two (those that actually hook into + dir-lookup), strange things happen + on the first boot, everything is fine, passive translators + showing up in /proc/mounts, nice + but when I reboot, the system kind-of comes back, but something + is very wrong with many (passive?) translators + the system never recovers, I have no idea whats going on there + ok + push that work in a branch somewhere for review please + right, thanks + braunr: + http://darnassus.sceen.net/gitweb/teythoon/hurd.git/shortlog/refs/heads/feature-translatorslist-detect-passive-translators + + +### IRC, freenode, #hurd, 2014-01-04 + + teythoon: did you eventually get any idea about why /proc/mounts is + missing mounts? + e.g. I have /boot as a separate partition, it doesn't show up + + +### IRC, freenode, #hurd, 2014-01-05 + + youpi: yes, passive translators are not currently handled + youpi: i have patches for that, but they produce weird results, + braunr promised to take a look + teythoon: hum + ^^ + i thought they were pending review + ! + where are they again ? + + http://darnassus.sceen.net/gitweb/teythoon/hurd.git/shortlog/refs/heads/feature-translatorslist-detect-passive-translators + ok + they are reasonably straight-forward + but cause this funny issue, after the first reboot with the + patched hurd everything is fine + after the second reboot, everything is weird and broken badly + right + interesting :) + + +### IRC, freenode, #hurd, 2014-01-06 + + teythoon: is it normal that, if ext2fs is started as an active + translator for /home, and then for another directory inside my home, mtab + only reports / and /home and not this third file system ? + (with the hurd master version) + (the translator for /home is run by root, but the one inside my + home is started with my uid) + (and every component on the path is readable/crossable) + + +### IRC, freenode, #hurd, 2014-01-07 + + braunr: well yes, the mtab tool/translator does not follow + translators bound to nodes not owned by root + braunr: try /hurd/mtab --insecure /home + ok + good thinking + + braunr: did you encounter any problems after the second reboot ? + i'm not yet there + work still making me busy + and i'm trying to isolate the problem first + that is, restrict tests to a single leaf translator + you reproduced the weirdness ? + teythoon: attempting + teythoon: first i'm trying to check a few things + teythoon: i'm running my leaf translator as root now + active + and i still don't see it :/ + one of the components in the path is not own by root + is that a problem ? + no, but maybe the mtab translator should check for that... + does it check for the node ownership or the process rights ? + credentials, rather + node ownership + ah, ok + ok i see it now + oh, also, it looks like settrans -g on an active translator + doesn't work, i get ebusy all the time :/ + oh ? + that could explain the fs corruption i saw + did i break that ? + i don't know + maybe + i think it was possible in the past + + teythoon: when isolated, your code works fine + i'll try applying it globally + btw, how did you do that ? + through debian packages ? + i'm trying to get to the point, but i'm still not there + with our uber-buildbot setup i picture myself pushing to a git + repo, wait a little and get the packages from my deb repo... + for now, i have my hurdtest tool that i used during gsoc + with that i can just drop files into a directory and have that + overlayed ontop of my test image + hum + so you replaced the lib*fs libraries directly ? + and the static rootfs ? + yes + * teythoon checks + hm, maybe i just missed a file, not sure anymore + + teythoon: with the new libraries, df -h doesn't see passive + translators :/ + braunr: see /proc/mounts please + same + or /hurd/mtab / + weird + no :/ + b/c when i developed it, i used a test suite to check that every + combination of tmpfs/ext2fs active/passive and every way to get rid of + any translator produced the desired results + i'll look into this + teythoon: when i remove the active translator i set on /home, i + get + /dev/hd0s1 8.1G 2.0G 5.8G 26% /home + hd0s1 being used for / :/ + this does need reviewing + this is how i expect the system to react currently + oh + w/o these patches + hm ok + well + i'm currently using them + including the root fs ? + yes + hmpf + i have to mention that i merged master into it + i did the same + currently compiling... + i only changed libdiskfs, libnetfs and libfshelp + is there something else that should be changed ? + (i.e. because of inlining ?) + i guess i should rebuild a hurd package just to be sure + braunr: isn't the translator for / statically linked ? if so it + needs to be rebuilt (sorry if I'm saying silly things by popping out of + nowhere) + yes i took care of that + + teythoon: i've built a hurd package with the three patches and i + don't see passive translators at all + :/ + well + actually i see a lot of the + them + just not /home + + braunr: please see + /home/teythoon/build/hurd-upstream/test.{bash,log} + the latter is a log i just created on darnassus + it shows no failures + do you have another translator that is started from a passive + translator record ? + besides /home, if so, does that show up ? + well, as i just said, i can actually see many of them + weird + http://darnassus.sceen.net/~rbraun/mounts + hm + let's try without --sync= + ah, now i can see it + can you give me the command line you used before + for the /home translator + /hurd/ext2fs --sync=30 /dev/hd0s6 + but hm + i can only see /home when the passive translator is started + is that intentional ? + yes + ok + and it doesn't work with --sync=30 o_O + so, actually, mtab doesn't report passive translators + no + it reports active ones only, whether they're started manually or + from passive translators + yes + ok + that's good enough + reporting passive translators wouldprobably require a complete + scan + yes, that was deemed not feasible + right + i can't reproduce any weirdness + good + it seems to just work well + except this option parsing problem + thanks for looking into this + yeah + sur + e + thanks for reminding me :) + the actual fix is implementing fsys_get_source + i forgot that "pending review" was my review here he + which should actually be file_get_source + + teythoon: why does mtab report errors for /proc/swaps and + /xconsole ? + not sure + + btw, i build hurd packages with my patches and it reliably + wreaks havoc on my test vms ... + now that's really weird + are you certain everything has been cleaned since your manual + replacement of the libraries ? + yes, i use mainly throwaway-vms for such experiments + ok + did you include the debian patches ? + yes + so did i + i based my work on my own packages + with thread destruction + i'll redo it from the sid ones + but before, i guess i should share mine with you + so you can test them in your vms + we may simply have different configurations + yes + something we might have missed just like the --sync parameter + + +### IRC, freenode, #hurd, 2014-01-08 + + teythoon: hello + see http://darnassus.sceen.net/~rbraun/teythoon/ for the custom + hurd packages + they need thread destruction so get the latest gnumach package + from unstable, and the libc packages from my repository first + + +### IRC, freenode, #hurd, 2014-01-09 + + braunr: your packages indeed seem to work + and with mine i encounter a different problem, the proc server + crashes *very* early + this is w/o a rebuilt libc + different as in not the weird one i remember having back when i + wrote those patches + hum + you're using all my glibc/hurd packages right ? + your packages work fine + i wonder if your bug is caused by + 2c9422595f41635e2f4f7ef1afb7eece9001feae + mine don't + (or rather, not having it) + look at the patches i've added + it's included, and i remember i really needed it + althouhg it was just about a leak + hm + i don't know :/ + but i strongly suspect your patches are ok + and something else is wrong + why would my packages miss that patch ? + hum + are you testing with your packages built from upstream sources ? + always + upstream hurd against debian glibc ? + yes + check the patches again + what patches ? + the debian ones that you don't include + another thing you can do is + err + get the latest debian hurd package + add your patch only + rebuild + i use hurd upstream + all the debian stuff + that's weird + but hurd upstream has received quite a lot of patches + please try from debian hurd + your patch only + + braunr: i'm afraid it might be your patch 3a3fcc81 that breaks + proc + +[[libpthread_set_stack_size]]. + + teythoon: but anyway, it does look like your patches are actually + fine + yes + i'll install my packages on darnassus and test a bit more + there is an issue however + ah + grep sd2s1 /proc/mounts + /dev/sd2s1 /dev/sd2s1 /hurd/storeio writable 0 0 + that makes fsck think that /dev/sd2s1 is mounted + hmpf + which makes debians fsck magic (when using sysvinit) drop to a + root shell at boot time + why does it report a mount point ? + or even a device + why not none /dev/sd2s1 ? + b/c of the heuristic it uses + and i know, you told me it's a bad idea + i did ? + probably + hm + we said so many things i don't remember + but that doesn't look too hard to fix + well, i'll just have to make translators provide meaningful + get_source responses + and get rid of the heuristic + ok + so if you use passive translators for the mounts, and not + /etc/fstab, you should be fine + my "traditional" hurd systems are + teythoon: i'll wait a bit before deplying it on darnassus then + + +### IRC, freenode, #hurd, 2014-01-13 + + teythoon: does your latest patch series take care of --sync ? + yeah, i finally got why the hurd would react strangely after an + reboot, it was umount --all removing vital passive translator records + i never had any issues with --sync + ah, umount -a + hm + do you recall i did :) ? + umount -a was only run with the sysvinit scripts, that's why you + didn't see that issue, only me + yes, i do + yes, i see + however, i'm also using --sync 30 on my fs + ok so you couldn't reproduce that particular issue + hum + try --sync=30 + showtrans /media/scratch + /hurd/ext2fs --sync=30 /dev/sd1s1 + works fine + now that's weird :) + then again, previously there was another bug + but then the patches i've tested are not the complete series you + sent + in the dir lookup function, if a passive translator is started + say, if i first access /proc/foo, /proc/foo would be recorded, + not /proc + i fixed that yesterday + ok + maybe it was b/c of that + but hm + why would /proc be recorded ? + now ? + i mean, /proc should be recorded at /, and /foo at /proc + right ? + yes + and that wasn't the case ? + but what happened was that proc/foo was recorded as a child + translator for / + ohh + yes + i see + well, i'm not sure it's that bug, since the translator involved + was on /home + same problem + it's unlikely that the translator was started b/c of a /home + lookup + much more likely that the first lookup is /home/someone + yes + but mtab correctly reported /home without --sync, and not when the + option was present + and that part doesn't quite make sense to me + how did you trigger the startup + ? + ssh i believe + hm + anyways, i could not reproduce this issue + do you have packages somewhere i can test ? + yes + oh and btw + http://darnassus.sceen.net/~teythoon/hurd-ci/ + something you could do to deal with umount -a + but i guess that's what you did already + is to only shut the active translator down + + http://git.sceen.net/hurd/hurd.git/commitdiff/0033d20449b3bb558f2ea470983018db39b572aa + yeah, i thought about that + but i believe that is not the right hting to do + yes i know but i'm not sure that's the right approach + hm :) + b/c if on linux you do umount /foo && ls /foo, foo will be empty + yours is probably more posix-friendly + if the passive translator lookup is left, /foo will be restarted + ok + s/lookup/record/ + that's one reason i'm not very fond of passive translators tbh + yep + i'd reserve them as a user-oriented hurd-specific feature + anything that must behave as mount/umount expects has to be active + anyway + ok let's give a quick shot at your packages :) + + teythoon: works fine :) + mtab still reports console entries though + is tha texpected ? + braunr: kind of + braunr: /bin/console is a netfs-based translator, probably for + multiplexing, dunno exactly + i see + + +### IRC, freenode, #hurd, 2014-01-14 + + teythoon: passive translators do work fine in mtab now, thanks :) + indeed :) + + +### IRC, freenode, #hurd, 2014-02-11 + + another topic: what's the rationale again behind umount removing + passive translators?... + antrik: umount is for compatibility with unixoid systems + consider umount /foo; ls /foo + if umount would leave the passive translator record on /foo, + /foo would be started again + but mount never creates passive translators, right? + so why would umount remove them? they are none of its business... + sure, you can see it this way + still, i like the way it is now, hence i implemented it this way + ;) + teythoon: but then umount -a unmounts all passive translators + include ~joe/http:/ + s/include/including/ + I tend to agree with antrik + i won't oppose a change of course + and yes, we have seen problems b/c of that. otoh those can be + fixed (and they are, i just sent a patch fixing that) + teythoon: well, it's not only about http:, joe user may want to + mount its own iso image, or whatever + thats true + no + it's not + /proc/mounts does not contain translators bound to nodes that do + not belong to root + but sure, we can change umount + i agree + active translators can be viewed as unix mounts + passive translators are an entirely hurdish feature + but then, servers such as pflocal and pfinet shouldn't probably + not be passive translators + braunr: shouldn't not? what are you trying to say? :-) + woops + i'm not trying to make this unconfusingly clearer + :p + pflocal and pfinet should probably be active translators + why? + hum wait + i had a reason weeks ago + but now it looks the opposite is better actually :p + so that they don't appear in mounts + but aiui, there is another property that is tested to make + translators appear in mounts + hm... I know this question has been discussed when first talking + about an mtab translator years ago... but don't remember whether there + was any conclusion + I think one of the ideas was that translators would opt in for + being considered as mounts... + it makes sense to only have file systems in mounts anyway + instead of going by the translator type, another option might be + ignoring anything that is backed by a passive translator?... + or have a startup option (perhaps with some "smart" defaults) to + request a translator to manifest as a mount or not + so many ideas... ;-) + + +## coreutils' `df` + +### IRC, freenode, #hurd, 2014-01-24 + + gnu_srs: "df: Warning: cannot read table of mounted file systems: + No such file or directory" + you should be able to fix that easily + + +### IRC, OFTC, #debian-hurd, 2014-02-03 + + /run/mtab also seem to be missing. df do not work. + that's a libc bug + + +### IRC, OFTC, #debian-hurd, 2014-02-05 + + i had to ln -s /proc/mounts /var/run/mtab to make df work, what's the + proper fix? + (here sysvinit+openrc) + gg0: the proper fix for df is to fix the coreutils build + + I checked the mtab problem, and /etc/hurd/runsystem.gnu via + /etc/hurd/rc fixes the symlink, while runsystem.sysv do not. I suspect + /etc/init.d/checkroot.sh should fix the symlink too. + well, atm df looks at /var/run/mtab instead of /etc/mtab only + because it hasn't been rebuilt against a recent glibc, that's all + but we can brownpaperbag fix it, yes + right. so perhaps that is the bug to fix? + yes, it is + it depends on coreutils actually building + youpi: i thought the proper way to fix the /var/run/mtab issue + is to patch the libc ? + it is + the libc defines some macro, on linux it expands to /etc/mtab, + on hurd to /var/run/mtab + but you need to rebuild coreutils to get a fixed df + ok, right + should it be /var/run/mtab or /etc/mtab on hurd? the former seem + more correct, but /run/mtab give more sense given that it should be + available also before /var/ is mounted. + to be honest, I don't really care + and I thus tend to agree on sticking with what linux does + to avoid issues + as in: keep debian working mostly the same on all kernels, to avoid + issue + s + well, linux really should move that file away from /etc/ too. :) + pere: ok, but let's move at the same time + rather than hitting bugs ourselves + perhaps df should use /proc/mount instead, and get rid of the + problem completely... + that can be a way, too + + I believe is a good fix for + the mtab problem. + + WRT /etc/mtab the main reason for keeping it is solely for + compatibility; there's no reason not to use /proc/mounts directly (or + /run/mtab if we want a kernel-agnostic location). Whether it's worth + doing something like that is debatable. + The main issue with doing stuff like this nowadays is that you get + shouted at by all the systemd people for making changes... + + ## Memory Leak in `translator_ihash_cleanup` ### IRC, freenode, #hurd, 2013-10-04 -- cgit v1.2.3