summaryrefslogtreecommitdiff
path: root/hurd/translator/ext2fs.mdwn
diff options
context:
space:
mode:
Diffstat (limited to 'hurd/translator/ext2fs.mdwn')
-rw-r--r--hurd/translator/ext2fs.mdwn364
1 files changed, 332 insertions, 32 deletions
diff --git a/hurd/translator/ext2fs.mdwn b/hurd/translator/ext2fs.mdwn
index e2f6b044..ba849cca 100644
--- a/hurd/translator/ext2fs.mdwn
+++ b/hurd/translator/ext2fs.mdwn
@@ -1,5 +1,5 @@
-[[!meta copyright="Copyright © 2007, 2008, 2010, 2011, 2012, 2013 Free Software
-Foundation, Inc."]]
+[[!meta copyright="Copyright © 2007, 2008, 2010, 2011, 2012, 2013, 2014 Free
+Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -163,6 +163,11 @@ small backend stores, like floppy devices.
<youpi> ok
+#### IRC, freenode, #hurd, 2013-10-08
+
+ <braunr> ogi: your ext2fs patches were finally merged upstream :)
+
+
## Sync Interval
[[!tag open_issue_hurd]]
@@ -209,37 +214,332 @@ That would be a nice improvement, but only after writeback throttling is impleme
<teythoon> tschwinge: well, thanks anyway ;)
-## Increased Memory Consumption
-
-### IRC, freenode, #hurd, 2013-09-18
-
- <braunr> ext2fs is using a ginormous amount of memory on darnassus since i
- last updated the hurd package :/
- <braunr> i wonder if my ext2fs large store patches rework have introduced a
- regression
- <braunr> the order of magnitude here is around 1.5G virtual space :/
- <braunr> it used to take up to 3 times less before that
- <braunr> looks like my patches didn't make it into the latest hurd package
- <braunr> teythoon: looks like there definitely is a new leak in ext2fs
- <teythoon> :/
- <braunr> memory only
- <braunr> the number of ports looks stable relative to file system usage
- <teythoon> braunr: I tested my patches on my development machine, it's up
- for 14 days (yay libvirt :) and never encountered problems like this
- <braunr> i've been building glibc to reach that state
- <teythoon> hm, that's a heavy load indeed
- <teythoon> could be the file name tracking stuff, I tried to make sure that
- everything is freed, but I might have missed something
- <braunr> teythoon: simply running htop run shows a slight, regular increase
- in physical memory usage in ext2fs
- <pinotree> old procfs stikes again? :)
- <teythoon> braunr: I see that as well... curious...
- <braunr> 16:46 < teythoon> could be the file name tracking stuff, I tried
- to make sure that everything is freed, but I might have missed something
- <braunr> how knows, maybe completely unrelated
- <teythoon> the tracking patch isn't that big, I've gone over it twice today
- and it still seems reasonable to me
+## IRC, freenode, #hurd, 2014-02-11
+
+ <gg0> task with pid 5 deallocating an invalid port 4622, most probably a
+ bug.
+ <gg0> ext2fs
+ <teythoon> yes, i've seen this
+ <teythoon> e.g. when a passive translator starts
+ <teythoon> i guess it is in libfshelp/translator-list.c
+
+
+## Inode Sizes, Fragment and Block Sizes
+
+### IRC, freenode, #hurd, 2014-02-12
+
+ <gg0> this might be interesting and could make people not to fsck hurd
+ filesystem on linux:
+ <gg0> start ext2fs: ext2fs: device:hd0s1
+ <braunr> ?
+ <gg0> : panic: get_hypermetadata: inode size 256 isn't supported
+ <gg0> (wait, also a bad typist)
+ <braunr> well, if the file system was created from the hurd, or with -o
+ hurd, as it ought to be, you wouldn't have this problem
+ <gg0> oh, good to know, especially before restoring :p
+ <braunr> i suspect your mkfs command to have created an ext4 fs
+ <gg0> nope mkfs.ext2
+ <braunr> hm ok, so it seems to create 256 size inodes by default there
+ <gg0> i guess -o hurd would set some os-specific properties
+ <braunr> it merely enforces a few restrictions
+ <gg0> some predefined defaults
+ <braunr> fragments and blocks are 4k
+ <braunr> and apparently inodes are 128 bytes
+ <gg0> because it can't support bigger values? is it worth working on remove
+ such restrictions?
+ <braunr> probably not so far
+ <braunr> certainly not the fragment/block size restriction
+ <braunr> it matches the page size
+ <braunr> larger inode sizes could be supported if they're dependencies for
+ other worthwhile features such as those someone would add in an ext4
+ translator
+
+
+## Linux' `CONFIG_EXT4_USE_FOR_EXT23`
+
+### IRC, freenode, #hurd, 2014-02-12
+
+ <gg0> why the hell i have thousands of Inode 839, i_blocks is 248, should
+ be 256. Fix<y>? yes
+ <gg0> in all cases i_blocks should be +8
+ <gg0> and /dev/sda1: (There are 245635 inodes containing multiply-claimed
+ blocks.)
+ <gnu_srs1> 10:50:08< gg0> start ext2fs: Hurd server bootstrap:
+ ext2fs[device:hd0s1] exec
+ <gnu_srs1> That's exactly where my image boot hangs!
+ <gg0> start ext2fs: Hurd server bootstrap: ext2fs[gunzip:device:rd0] exec
+ <AliciaC> gnu_srs1: you might want to check that linux isn't using the ext4
+ module to handle ext2 and ext3 filesystems
+ <AliciaC> gnu_srs1: as I understand it the idea is that the ext4 module
+ treats them as ext2/ext3 filesystems, just avoiding code duplication from
+ having three separate modules for related filesystems, so it shouldn't
+ change it from ext2 at all, but it does do something strange with it
+ <AliciaC> but I'm not sure if that's the case or if it's converting it to
+ ext4. last I heard Hurd doesn't support anything beyond ext2
+ <gnu_srs1> AliciaC: I did use ext2 when mounting from Linux: mount -t ext2
+ /dev/loop0 /mnt
+ <gnu_srs1> and when not mounted: e2fsck /dev/loop0
+ <AliciaC> gnu_srs1: I'd check the kernel config to be sure,
+ CONFIG_EXT4_USE_FOR_EXT23 must be disabled
+ <braunr> you can use the ext4 driver for ext2
+ <braunr> that's not a problem here
+ <braunr> the problem happens long before, when the file system gets
+ corrupted
+ <braunr> you must understand why
+ <AliciaC> I have done some testing on this, mounting a Hurd ext2 filesystem
+ with the ext4 module broke it for me, an easily repeated issue
+ <AliciaC> mounting Debian's ext2 image and unmounting it with ext4 broke
+ it, resulting precisely in the kind of hang ups mentioned by gnu_srs1 and
+ gg0
+ <braunr> interesting
+ <AliciaC> that's with a clean image with nothing corrupting it before hand,
+ tested to be working as well
+ <braunr> ok so the ext4 driver must ignore hurd specific stuff
+ <braunr> that's strange because i recall using it to perform small repairs
+ on darnassus and never had any issue
+ <braunr> even on the root file syste
+ <braunr> but my repairs were very quick and targetted
+ <AliciaC> different linux versions maybe
+ <AliciaC> when I was testing it I didn't even need to do anything in the
+ filesystem to trigger the issue, just mount and unmount
+ <gnu_srs1> I repaired filesystems before like this, has something happened
+ with later versions of Linux?
+ <gnu_srs1> One of my boxes is ext3 (probably worked before) another ext4
+ (the one breaking things, but worked before)
+ <gnu_srs1> ext3 and ext4 box: CONFIG_EXT4_USE_FOR_EXT23=y same kernel
+ 3.12-1.amd64
+ <gnu_srs1> what about mounting with bs=4096 (used by hurd)
+ <braunr> -t ext2 should work fine
+ <braunr> just don't use the ext4 driver if in doubt
+ <gg0> no difference between specifying -t or not, in both cases EXT4-fs
+ (sda1): mounting ext2 file system using the ext4 subsystem
+ <braunr> hmm
+ <braunr> you're screwed then ;
+ <braunr> ;p
+ <braunr> or maybe -t ext3 .. :)
+ <braunr> although i suspect ext4 would be used then too
+ <gg0> linux-image-3.2.0-4-amd64:
+ /lib/modules/3.2.0-4-amd64/kernel/fs/ext2/ext2.ko
+ <gg0> wheezy still has it. then something between 3.2.0 and 3.13(?) removed
+ it
+ <braunr> check the config file
+ <gg0> i mean ext2 module
+ <braunr> check if the config file enables it
+ <gnu_srs1> It's not: # CONFIG_EXT2_FS is not set
+ <gg0> 14:42 < gg0> wheezy still has it. then something between 3.2.0 and
+ 3.13(?) removed it
+ <braunr> how about retrying what you did without ever mounting from linux ?
+ <braunr> gg0: it wasn't clear enough that you meant removed from
+ configuration
+ <braunr> (for example, it could have been blacklisted)
+ <gg0> or present not as a module
+ <braunr> maybe yes, although it's unusual to see generic kernels embedding
+ file systems these days
+ <AliciaC> the CONFIG_EXT4_USE_FOR_EXT23 option isn't available if either
+ ext2 or ext3 are enabled though, even just as loadable modules
+ <gnu_srs1> The ext2 and ext3 modules were there in 3.10-3, not in 3.12-1
+ <gnu_srs1> (14:48:59) <srs>: It's not: # CONFIG_EXT2_FS is not set --
+ 3.12-1
+ <gg0> https://bugs.debian.org/731072
+ * gg0 rsync'ing back to new fs with 3.10 kernel
+ <gnu_srs1> seems like this bug was archived without being closed??
+ <gg0> someone should produce a testcase and file another one btw
+ <gnu_srs1> but that bug was for files systems up to 4MB, not 4GB?
+ <gg0> i pasted it just because submitter talks about config option in
+ question and when was enabled
+ <gg0> don't we want to thank AliciaC who pointed it out and who could
+ precisely file a bug? :)
+ <gg0> filed http://bugs.debian.org/738758
+ <braunr> gg0: thanks
+ <braunr> AliciaC: and thanks too
+
+
+### IRC, freenode, #hurd, 2014-02-13
+
+ <gnu_srs> gg0: Did you create and test with an ext2 Linux image too on
+ 3.10/3.12?
+ <gnu_srs> here is a diff: http://paste.debian.net/81837/
+ <gnu_srs> visible differences: Filesystem features:filetype (linux only)
+ and Free inodes:1268(hurd) / 1269(linux)
+ <AliciaC> between one created with -o Hurd and one created with -o Linux
+ (or no -o)?
+ <gnu_srs> AliciaC: -o Hurd and -b 4096 (no -o)
+ <AliciaC> I wonder if that would show any interesting difference between an
+ untouched -o Hurd ext2 image and a copy of it that has been mounted with
+ the ext4 module
+ <gnu_srs> AliciaC: here: http://paste.debian.net/81857/
+ <gnu_srs> there is a difference of one in the number of free inodes!
+ <gnu_srs> cf the number of free inodes for linux
+ <AliciaC> gnu_srs: thanks :) though I don't know what to make of that, I
+ guess just adding an inode shouldn't break anything
+ <AliciaC> wait, no, removing an inode?
+ <AliciaC> bleh, too tired, read it wrong
+ <gnu_srs> this line should read:(11:37:48) srs: visible differences:
+ Filesystem features:filetype (linux only) and Free inodes:1268(linux) /
+ 1269(hurd)
+ <gnu_srs> There are differences in ext2.h and ext4.h in the Linux source
+ code wrt hurd1, hurd2 structs.
+ <gnu_srs> one change might be interesting: http://paste.debian.net/81864/
+ <braunr> gnu_srs: probably not
+ <gnu_srs> If not, where to look?
+ <braunr> well, the first thing would be to create a (small) ext2 file
+ system, usable on the hurd, with a few files and directories
+ <braunr> save it
+ <braunr> mount it with the ext4 driver
+ <braunr> and make a binary comparison
+ <braunr> you could use a modified ext2fs translator to tell you exactly
+ what's wrong when loading the file system
+ <braunr> and then look at the corresponding code in the ext4 driver
+ <gnu_srs1> braunr: here is a binary diff of the unmounted and mounted e2fs
+ files: http://paste.debian.net/81896/
+ <braunr> gnu_srs1: i'm not going to analyze it
+ <braunr> you are
+ <braunr> :p
+ <gnu_srs1> many of them can be removed: e.g. /mnt and bug000
+ <braunr> ?
+ <gnu_srs1> many diff entries*
+ <braunr> but why ?
+ <braunr> you shouldn't have changed the content at all
+ <gnu_srs1> If I don't add a file, the fs is not corrupted
+ <gnu_srs1> this is with two vers small files created as in gg0s bug report
+ <gnu_srs1> very*
+ <braunr> ok
+ <braunr> i guess checking the source code first and the binary diffs next
+ is easier
+ <gnu_srs1> OK, I have to find out how the ext2fs files are organized.
+ <gnu_srs1> I.e. reading mke2fs source code
+ <braunr> no
+ <braunr> read the ext4 driver
+ <braunr> how a directory entry is created
+ <braunr> how a directory is saved back on the block device
+ <braunr> how any potential conversion could be triggered
+ <gnu_srs1> k, will do
+ <braunr> read about the ext2fs format if doing that first doesn't help
+ <braunr> learning a file system is complicated and long
+ <gnu_srs1> What is the inode size for Hurd/Linux?
+ <braunr> probably 128
+ <gnu_srs1> same for both?
+ <braunr> what is "Hurd/Linux" ?
+ <gnu_srs1> on Hurd / on Linux
+ <braunr> 128 on hurd, variable on linux
+ <braunr> 128-512 i'd say
+ <gnu_srs1> ext2 on linux
+ <gnu_srs1> found it from dumpe2fs: 128 for both
+ <braunr> no, it can vary on linux
+ <braunr> although once a file system is built, the inode size cannot be
+ changed
+ <gnu_srs1> k, the file created with mke2fs has 128
+
+
+## `ext2fs: ../../ext2fs/pager.c:401: file_pager_write_page: Assertion 'block' failed.`
+
+### IRC, freenode, #hurd, 2014-02-19
+
+ <pere> "ext2fs: ../../ext2fs/pager.c:401: file_pager_write_page: Assertion
+ 'block' failed." in the console.
+
+[[user/Maksym_Planeta]] also has hit that one.
+
+ <braunr> wow oO
+ <braunr> using debian hurd right ?
+ <pere> power cycling.
+ <pere> yes.
+ <braunr> with hurd 1:0.5.git20140203-1 and glibc 2.17-98~1 ?
+ <pere> braunr: not sure how to check.
+ <braunr> pere: dpkg -l | grep .. i suppose
+ <pere> gah, autofsck do not work.. :(
+ <braunr> it does :(
+ <braunr> unstable is easy to mess it seems
+ <pere> had to run fsck -y / manually...
+ <braunr> i suspect you were using a corrupted file system at mount time
+ <braunr> ah that
+ <braunr> yes it is sometimes needed
+ <braunr> but ext2 is reliable enough that only temporary files get their
+ way into lost+found
+ <braunr> temporary/recently created
+ <braunr> the crash you had, on the other hand, looks more serious
+ <braunr> it seems like you mounted a corrupted file system
+ <pere> could be.
+ <pere> hurd v1:0.5.git20140203-1 and libc0.3 v2.17-98~1, it seem.
+ <braunr> good
+ <braunr> you shouldn't have such problems then, i suspect a mess up on your
+ part
+ <braunr> but you're not the only one to have had weird file systems
+ problems lately
+ <pere> hah. I blame the hurd. :P
+ <braunr> heh :)
+ <pere> gah, another crash. :(
+ <braunr> Oo
+ <braunr> same assertion ?
+ <pere> same place, or almost the same place.
+ <pere> yes.
<braunr> hm
+ <pere> same crash. :(
+ <braunr> what kind of machine do you run the hurd on ?
+ <pere> kvm
+ <braunr> how much memory ?
+ <pere> 1G
+ <braunr> did you see if the system was swapping ?
+ <pere> no idea.
+ <braunr> i suggest always running top/htop on the hurd ;p
+ <braunr> and monitor memory usage closely
+ <gg0> unless pere lately mounted/fsck'ed fs in question with a recent linux
+ kernel, there should not be particular problems
+ <braunr> it definitely doesn't look like it was mounted by an ext4 driver,
+ no
+ <braunr> which means it's something else entirely and this is scary
+ <pere> I didn't. I fetched the prebuild image, upgraded it, switched it to
+ sysvinit and started working.
+ <braunr> sorry i can't be of more help about that
+ <braunr> ext2fs has been quite solid on my machines for a long time :(
+ <braunr> there are known assertions that trigger under some special
+ pressure, but that's not what you're having here
+ <braunr> pere: anything particular in fstab ?
+ <pere> nope, have not touched /etc/fstab.
+ <braunr> hm stupid question
+ <braunr> are you sure it's not full ?
+ <pere> nothing look full to me.
+ <pere> neither the disk nor the host file system.
+
+
+### IRC, freenode, #hurd, 2014-02-20
+
+ <pere> braunr: do you remember my ext2fs crash from yesterday? I could
+ avoid it by interrupting the triggering build and running sync once in a
+ while. and it show up again if I do not sync in between. :)
+ <braunr> ?
+ <braunr> are you sure you're not swapping ?
+ <pere> I have no idea. still. :)
+ <braunr> again, i recommend you run top/htop and monitor that
+ <braunr> pere: is your patch needed to trigger the assertion ?
+
+[[open_issues/ti-rpc_then_nfs]].
+
+ <pere> braunr: well, without it, the package do not build, so yeah. :)
+ <braunr> ok
+ <pere> tested again, and is not swapping. 850MB free memory.
+ <braunr> ok
+ <braunr> so this might be a real file system bug
+ <braunr> let me see
+ <braunr> pere: libtirpc built fine here ..
+ <braunr> pere: do you have a separate /home partition ?
+ <braunr> or any separate file system for builds
+ <pere> braunr: nope, everything on /
+ <braunr> pere: i wouldn't recommend that
+ <braunr> there very probably are bugs in the file system code and using
+ separate partitions is a way to alleviate them
+
+
+## `ext2fs: ../../libdiskfs/rdwr-internal.c:42: _diskfs_rdwr_internal: Assertion `!diskfs_readonly' failed.`
+
+### IRC, freenode, #hurd, 2014-02-22
+
+ <gg0> login: init: notifying pfinet of shutdown...init: notifying tmpfs
+ none of shutdown...init: notifying tmpfs none of shutdown...init:
+ notifyi.
+ <gg0> ext2fs: ../../libdiskfs/rdwr-internal.c:42: _diskfs_rdwr_internal:
+ Assertion `!diskfs_readonly' failed.
+ <gg0> In tight loop: hit ctl-alt-del to reboot
# Documentation