open_issues/ext2fs_file_corruption: New.

1 files changed, 199 insertions, 0 deletions

diff --git a/open_issues/ext2fs_file_corruption.mdwn b/open_issues/ext2fs_file_corruption.mdwn
new file mode 100644
index 00000000..2af35bb5
--- /dev/null
+++ b/open_issues/ext2fs_file_corruption.mdwn
@@ -0,0 +1,199 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_hurd]]
+
+[[!toc]]
+
+
+# Log
+
+Create two 5 GiB partitions.
+
+    # # Would default to 256 byte inodes otherwise.
+    # mke2fs -I 128 -o linux /dev/hd1s1
+    [...]
+    # mke2fs -o hurd /dev/hd1s2
+    [...]
+
+The only difference (apart from UUID, time stamps, hash seeds) in ext2 file
+system meta data, according to `dumpe2fs -h`:
+
+    [...]
+    -Filesystem features:      ext_attr resize_inode dir_index filetype sparse_super large_file
+    +Filesystem features:      ext_attr resize_inode dir_index sparse_super large_file
+     Filesystem flags:         signed_directory_hash
+     Default mount options:    (none)
+     Filesystem state:         clean
+     Errors behavior:          Continue
+    -Filesystem OS type:       Linux
+    +Filesystem OS type:       Hurd
+    [...]
+
+According to *mke2fs(8)*:
+
+> *filetype*  
+>                           Store file type information in directory entries.
+
+    # settrans -c /media/hd1s1 /hurd/ext2fs /dev/hd1s1
+    # settrans -c /media/hd1s2 /hurd/ext2fs /dev/hd1s2
+    # mkdir /media/hd1s{1,2}/tmp
+    # chmod 1777 /media/hd1s{1,2}/tmp
+
+    $ git-new-workdir ~/tmp/binutils/git /media/hd1s1/tmp/master master
+    error: unable to create file gas/testsuite/gas/arm/attr-mfpu-vfpv3-d16.d (Interrupted system call)
+    Checking out files: 100% (12315/12315), done.
+    Already on 'master'
+    $ cd /media/hd1s1/tmp/master
+    $ git status
+    # On branch master
+    # Changes not staged for commit:
+    #   (use "git add <file>..." to update what will be committed)
+    #   (use "git checkout -- <file>..." to discard changes in working directory)
+    #
+    #       modified:   gas/testsuite/gas/arm/attr-mfpu-vfpv3-d16.d
+    #
+    no changes added to commit (use "git add" and/or "git commit -a")
+    $ git checkout -f
+    $ git status
+    # On branch master
+    nothing to commit (working directory clean)
+
+([[Git issue|git-core-2]] is known.)
+
+    $ git-new-workdir ~/tmp/binutils/git /media/hd1s2/tmp/master master
+    error: unable to create file bfd/elf32-dlx.c (Interrupted system call)
+    error: unable to create file bfd/sunos.c (Interrupted system call)
+    error: unable to create file gas/testsuite/gas/arm/attr-mfpu-vfpv3-d16.d (Interrupted system call)
+    error: unable to create file gas/testsuite/gas/mmix/regx-op.d (Interrupted system call)
+    error: unable to create file gas/testsuite/gas/tic6x/reloc-bad-4.s (Interrupted system call)
+    error: unable to create file gold/testsuite/script_test_2.t (Interrupted system call)
+    error: unable to create file ld/testsuite/ld-mmix/loc7m.d (Interrupted system call)
+    error: unable to create file ld/testsuite/ld-powerpc/tlsexe.g (Interrupted system call)
+    Checking out files: 100% (12315/12315), done.
+    Already on 'master'
+    $ cd /media/hd1s2/tmp/master
+    $ git status
+    # On branch master
+    # Changes not staged for commit:
+    #   (use "git add <file>..." to update what will be committed)
+    #   (use "git checkout -- <file>..." to discard changes in working directory)
+    #
+    #       modified:   bfd/elf32-dlx.c
+    #       modified:   bfd/sunos.c
+    #       modified:   gas/testsuite/gas/arm/attr-mfpu-vfpv3-d16.d
+    #       modified:   gas/testsuite/gas/mmix/regx-op.d
+    #       modified:   gas/testsuite/gas/tic6x/reloc-bad-4.s
+    #       modified:   gold/testsuite/script_test_2.t
+    #       modified:   ld/testsuite/ld-mmix/loc7m.d
+    #       modified:   ld/testsuite/ld-powerpc/tlsexe.g
+    #
+    no changes added to commit (use "git add" and/or "git commit -a")
+    $ git checkout -f
+    $ git status
+    # On branch master
+    nothing to commit (working directory clean)
+
+Now you'd expect these directories to have identical content, but:
+
+    $ diff -x .git -ru /media/hd1s{1,2}/tmp/master/ > /tmp/diff
+    $ ls -l /tmp/diff
+    -rw-r--r-- 1 thomas thomas 613677 10. Jun 19:12 /tmp/diff
+    $ grep '^[^ @+-]' < /tmp/diff
+    diff -x .git -ru /media/hd1s1/tmp/master//ld/configure /media/hd1s2/tmp/master//ld/configure
+
+(Note that this isn't a file that Git had issues with.)
+
+Try again:
+
+    $ diff -x .git -ru /media/hd1s{1,2}/tmp/master/ > /tmp/diff_
+    $ ls -l /tmp/diff*
+    -rw-r--r-- 1 thomas thomas 613677 10. Jun 19:12 /tmp/diff
+    -rw-r--r-- 1 thomas thomas 613677 10. Jun 19:17 /tmp/diff_
+    $ cmp /tmp/diff{,_}; echo $?
+    0
+
+At least it's consistent.  Force a reload:
+
+    # settrans -ag /media/hd1s1
+    # settrans -ag /media/hd1s2
+
+Try again:
+
+    $ diff -x .git -ru /media/hd1s{1,2}/tmp/master/ > /tmp/diff__
+    $ ls -l /tmp/diff*
+    -rw-r--r-- 1 thomas thomas 613677 10. Jun 19:12 /tmp/diff
+    -rw-r--r-- 1 thomas thomas 613677 10. Jun 19:17 /tmp/diff_
+    -rw-r--r-- 1 thomas thomas 613677 10. Jun 19:30 /tmp/diff__
+    $ cmp /tmp/diff{,__}; echo $?
+    0
+
+Consistent; thus very likely corrupt on-disk.
+
+After a few tries, the pattern generally is that for the files where there are
+differences, once the file regularely ends, its content appears once more.
+That is, the files' content appears once (regularely), and then the same again.
+From three retries, I've seen this only ever happen on the `-o
+linux`/`filetype` option file system.
+
+Some more copying:
+
+    $ (cd /media/hd1s1/tmp/ && cp -a master master_)
+    $ (cd /media/hd1s2/tmp/ && cp -a master master_)
+    $ diff -x .git -ru /media/hd1s1/tmp/master{,_}/ > /tmp/diff1
+    $ diff -x .git -ru /media/hd1s2/tmp/master{,_}/ > /tmp/diff2
+    $ ls -l /tmp/diff{1,2}
+    -rw-r--r-- 1 thomas thomas 0 10. Jun 19:46 /tmp/diff1
+    -rw-r--r-- 1 thomas thomas 0 10. Jun 19:46 /tmp/diff2
+
+No further difference.
+
+
+# Conclusion?
+
+The ext2 `filetype` option, as used for Linux-type file systems by default,
+might cause some confusion inside [[hurd/translator/ext2fs]]?
+
+[[!tag open_issue_documentation]] The strange thing is that in
+`[hurd]/ext2fs/dir.c` the `EXT2_FEATURE_INCOMPAT_FILETYPE` is generally masked
+out (is not even considered) when adding a node to a directory in
+`diskfs_direnter_hard` and when reading in `diskfs_get_directs`, so unless
+setting this file system flag has some other effects on the meta data (which it
+doesn't seem to have according to the Linux kernel source code), it shouldn't
+make any difference.  Also, this information would only be used for setting
+directory listing's informational `d_type` field (`readdir`, etc.).  Hrm.
+Perhaps only a red herring?
+
+
+## `e2fsck`
+
+Running `e2fsck` on a file system with the `filetypes` option, there are
+messages that the `filetype` for a lot of files is set (to either 1 (regular
+file, `EXT2_FT_REG_FILE`), or 2 (directory, `EXT2_FT_DIR`)).  The Hurd's ext2fs
+should unconditionally set this field to 0 (`EXT2_FT_UNKNOWN`).
+
+
+# Next
+
+Start over with the `filetype` option reversed:
+
+    # mke2fs -I 128 -o linux -O ^filetype /dev/hd1s1
+    # mke2fs -o hurd -O filetype /dev/hd1s2
+
+In fact, for `-o hurd`, `-O filetype` doesn't have any effect (according to
+`dumpe2fs -h`), so the meta data now only differs in *Filesystem OS type*.
+
+Invoke `git-new-workdir` once more (for each).
+
+Now there is corruption in files on the `-o hurd -O filetype` file system.  But
+this time not caused by `git-new-workdir`, but instead when copying a binutils
+checkout to a new directory.  Huh.  Does `-O filetype` in fact have any effect
+with `-o hurd`?  The `e2fsprogs` source code should tell, but is not available
+right now.