summaryrefslogtreecommitdiff
path: root/open_issues
diff options
context:
space:
mode:
authorSamuel Thibault <samuel.thibault@ens-lyon.org>2011-02-20 22:19:43 +0100
committerSamuel Thibault <samuel.thibault@ens-lyon.org>2011-02-20 22:19:43 +0100
commit7dd4adb5612fa6042d421e8d436a0c7b4facfb22 (patch)
treea0ff209360084aa8de37e9b5a52db6de81103aac /open_issues
parent72f22ab02e662e9e9fed6918ec145fd77584dad1 (diff)
parentd22a3b299d00ce757237f9aee9794d0d4f2758e2 (diff)
Merge branch 'master' of flubber:~hurd-web/hurd-web
Diffstat (limited to 'open_issues')
-rw-r--r--open_issues/binutils.mdwn47
-rw-r--r--open_issues/binutils/log_build-diff43
-rw-r--r--open_issues/binutils/log_install-diff4
-rw-r--r--open_issues/binutils/sum_hurd36
-rw-r--r--open_issues/binutils/sum_linux36
-rw-r--r--open_issues/gnumach_console_timestamp.mdwn27
-rw-r--r--open_issues/implementing_hurd_on_top_of_another_system.mdwn37
-rw-r--r--open_issues/performance/io_system.mdwn11
-rw-r--r--open_issues/performance/io_system/clustered_page_faults.mdwn103
-rw-r--r--open_issues/performance/io_system/read-ahead.mdwn299
10 files changed, 583 insertions, 60 deletions
diff --git a/open_issues/binutils.mdwn b/open_issues/binutils.mdwn
index 81fafaca..ca7496f0 100644
--- a/open_issues/binutils.mdwn
+++ b/open_issues/binutils.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2007, 2008, 2010 Free Software Foundation,
+[[!meta copyright="Copyright © 2007, 2008, 2010, 2011 Free Software Foundation,
Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
@@ -9,7 +9,7 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
-[[!tag open_issue_binutils]]
+[[!tag stable_URL open_issue_binutils]]
Here's what's to be done for maintaining GNU Binutils.
@@ -30,15 +30,14 @@ though, as explained below.
# Configuration
-Last reviewed up to the [[Git mirror's e347ef3b343fc42ed312d5125047d59ae15df795
-(2010-12-20) sources|source_repositories/binutils]].
+Last reviewed up to the [[Git mirror's a446ef2f3862fb5f89c669b34a2b6a2ab943ff96
+(2011-02-10) sources|source_repositories/binutils]].
* Globally
* a.out, COFF, PE image support and 64 bit support are not interesting.
- * In the [[testsuite]]s, `.exp` and `.d` files very likely should not
- only
+ * In the testsuites, `.exp` and `.d` files very likely should not only
care for `*-*-linux*`, but also `*-*-gnu*`. (If the need to be
conditionalized like this at all.)
@@ -96,7 +95,7 @@ Last reviewed up to the [[Git mirror's e347ef3b343fc42ed312d5125047d59ae15df795
* `*-*-gnu*`
TODO: resolve `crt0.o` vs. `crt1.o` issue. [[Testsuite
- failures|testsuite#static]].
+ failures|binutils#static]].
* `configure.tgt`
@@ -109,7 +108,7 @@ Last reviewed up to the [[Git mirror's e347ef3b343fc42ed312d5125047d59ae15df795
# Build
Here's a log of a binutils build run; this is from our [[Git
-repository's 245f62b817ee31135a190793dddb340f04ac95e6 (2010-12-20)
+repository's e8052e7548e0d5523f1764b7d3896ca000bfaed7 (2011-02-10)
sources|source_repositories/binutils]], run on kepler.SCHWINGE and grubber.
$ export LC_ALL=C
@@ -121,7 +120,7 @@ sources|source_repositories/binutils]], run on kepler.SCHWINGE and grubber.
(kepler.SCHWINGE defaults to using /bin/sh for libtool, grubber to /bin/bash;
thus harmonized.)
-On grubber, this takes roughly one hour.
+On grubber, this needs roughly one hour, and takes up around 100 MiB.
## Analysis
@@ -145,7 +144,7 @@ GNU/Linux defining `-DTRAD_CORE`, `-DHAVE_i386linux_vec`
(kepler.SCHWINGE defaults to using /bin/sh, grubber to /bin/bash; thus
harmonized.)
-On grubber, this needs roughly 15 minutes, and takes up around 0.7 GiB.
+On grubber, this needs roughly 5 minutes, and takes up around 60 MiB.
## Analysis
@@ -170,12 +169,12 @@ On grubber, this takes roughly one hour.
Comparing the results files, [[sum_linux]] to [[sum_hurd]]:
$ diff -u -F ^Running open_issues/binutils/sum_linux open_issues/binutils/sum_hurd
- --- open_issues/binutils/sum_linux 2010-12-20 19:01:06.000000000 +0100
- +++ open_issues/binutils/sum_hurd 2010-12-20 19:01:20.000000000 +0100
+ --- open_issues/binutils/sum_linux 2011-02-10 19:01:56.000000000 +0100
+ +++ open_issues/binutils/sum_hurd 2011-02-10 20:27:17.000000000 +0100
@@ -1,5 +1,5 @@
- -Test Run By thomas on Mon Dec 20 11:34:53 2010
+ -Test Run By thomas on Thu Feb 10 18:57:42 2011
-Native configuration is i686-pc-linux-gnu
- +Test Run By tschwinge on Mon Dec 20 11:35:47 2010
+ +Test Run By tschwinge on Thu Feb 10 18:58:16 2011
+Native configuration is i686-unknown-gnu0.3
=== binutils tests ===
@@ -184,9 +183,9 @@ Comparing the results files, [[sum_linux]] to [[sum_hurd]]:
# of expected passes 83
# of unsupported tests 2
- -Test Run By thomas on Mon Dec 20 11:35:19 2010
+ -Test Run By thomas on Thu Feb 10 18:58:10 2011
-Native configuration is i686-pc-linux-gnu
- +Test Run By tschwinge on Mon Dec 20 11:44:29 2010
+ +Test Run By tschwinge on Thu Feb 10 19:06:15 2011
+Native configuration is i686-unknown-gnu0.3
=== ld tests ===
@@ -232,21 +231,21 @@ Comparing the results files, [[sum_linux]] to [[sum_hurd]]:
PASS: ELF DSO small bar (size)
PASS: ELF DSO foo with small bar (size)
PASS: ELF DSO big bar (size)
- @@ -873,13 +873,14 @@ Running [...]/hurd/master/ld/testsuite/l
+ @@ -882,13 +882,14 @@ Running [...]/hurd/master/ld/testsuite/l
=== ld Summary ===
- -# of expected passes 618
+ -# of expected passes 626
-# of expected failures 8
- +# of expected passes 608
+ +# of expected passes 616
+# of unexpected successes 1
+# of expected failures 17
# of untested testcases 6
- /media/data[...]/hurd/master.build/ld/ld-new 2.21.51.20101220
+ /media/data[...]/hurd/master.build/ld/ld-new 2.21.51.20110210
- -Test Run By thomas on Mon Dec 20 11:34:59 2010
+ -Test Run By thomas on Thu Feb 10 18:57:49 2011
-Native configuration is i686-pc-linux-gnu
- +Test Run By tschwinge on Mon Dec 20 11:38:03 2010
+ +Test Run By tschwinge on Thu Feb 10 19:00:16 2011
+Native configuration is i686-unknown-gnu0.3
=== gas tests ===
@@ -255,7 +254,7 @@ Comparing the results files, [[sum_linux]] to [[sum_hurd]]:
## Analysis
- * <a name="static">`FAIL: static [...]`</a>
+ * <a name="static"><!-- stable_URL -->`FAIL: static [...]`</a>
The testsuite isn't prepared for using `crt0.o` instead of `crt1.o`
depending on whether a static or dynamic executable is created. Documented
@@ -269,7 +268,7 @@ Comparing the results files, [[sum_linux]] to [[sum_hurd]]:
weakness|performance/io_system/binutils_ld_64ksec]]), so assuming some
system load variation, the testsuite's timeout may trigger.
- * <a name="weak">`FAIL: ELF weak [...]`</a>
+ * <a name="weak"><!-- stable_URL -->`FAIL: ELF weak [...]`</a>
[[I|tschwinge]] suppose this is due to us having an override w.r.t. weak
symbol handling in glibc, needed for our external [[/libpthread]]. TODO:
diff --git a/open_issues/binutils/log_build-diff b/open_issues/binutils/log_build-diff
index 802d510c..3408d97d 100644
--- a/open_issues/binutils/log_build-diff
+++ b/open_issues/binutils/log_build-diff
@@ -1,5 +1,5 @@
---- /dev/fd/63 2010-12-20 11:34:03.204493002 +0100
-+++ /dev/fd/62 2010-12-20 11:34:03.208493002 +0100
+--- /dev/fd/63 2011-02-10 17:33:04.738225001 +0100
++++ /dev/fd/62 2011-02-10 17:33:04.738225001 +0100
@@ -1,6 +1,6 @@
-checking build system type... i686-pc-linux-gnu
-checking host system type... i686-pc-linux-gnu
@@ -530,7 +530,7 @@
checking whether we are using the GNU C compiler... (cached) yes
checking whether gcc accepts -g... (cached) yes
checking for gcc option to accept ISO C89... (cached) none needed
-@@ -2450,28 +2429,28 @@
+@@ -2453,28 +2432,28 @@
checking for BSD- or MS-compatible name lister (nm)... nm
checking the name lister (nm) interface... BSD nm
checking whether ln -s works... yes
@@ -566,23 +566,37 @@
checking how to hardcode library paths into programs... immediate
checking whether stripping libraries is possible... yes
checking if libtool supports shared libraries... yes
-@@ -2555,13 +2534,13 @@
+@@ -2486,11 +2465,11 @@
+ checking whether the g++ linker (ld) supports shared libraries... yes
+ checking for g++ option to produce PIC... -fPIC -DPIC
+ checking if g++ PIC flag -fPIC -DPIC works... yes
+-checking if g++ static flag -static works... yes
++checking if g++ static flag -static works... no
+ checking if g++ supports -c -o file.o... yes
+ checking if g++ supports -c -o file.o... (cached) yes
+ checking whether the g++ linker (ld) supports shared libraries... yes
+-checking dynamic linker characteristics... (cached) GNU/Linux ld.so
++checking dynamic linker characteristics... gnu0.3 ld.so
+ checking how to hardcode library paths into programs... immediate
+ checking whether NLS is requested... yes
+ checking for catalogs to be installed... bg da es fi fr ga id ja sv tr vi zh_CN zh_TW
+@@ -2570,13 +2549,13 @@
/bin/bash ../../master/ld/../ylwrap ../../master/ld/ldgram.y y.tab.c ldgram.c y.tab.h ldgram.h y.output ldgram.output -- bison -y -d
updating ldgram.h
(echo "/* This file is automatically generated. DO NOT EDIT! */";\
-- for f in `echo " " eelf_i386.o ei386linux.o "" \
+- for f in `echo " " eelf_i386.o ei386linux.o eelf32_x86_64.o "" \
+ for f in `echo " " eelf_i386.o "" \
| sed -e 's/ e/ ld/g' -e 's/ ld/ /g' -e 's/[.]o//g'`; do \
echo "extern ld_emulation_xfer_type ld_${f}_emulation;"; \
done;\
echo "";\
echo "#define EMULATION_LIST \\";\
-- for f in `echo " " eelf_i386.o ei386linux.o "" \
+- for f in `echo " " eelf_i386.o ei386linux.o eelf32_x86_64.o "" \
+ for f in `echo " " eelf_i386.o "" \
| sed -e 's/ e/ ld/g' -e 's/ ld/ /g' -e 's/[.]o//g'`; do \
echo " &ld_${f}_emulation, \\"; \
done;\
-@@ -2650,8 +2629,8 @@
+@@ -2665,8 +2644,8 @@
mv -f .deps/ldctor.Tpo .deps/ldctor.Po
gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT ldmain.o -MD -MP -MF .deps/ldmain.Tpo -c -o ldmain.o \
-DDEFAULT_EMULATION='"elf_i386"' \
@@ -593,7 +607,7 @@
../../master/ld/ldmain.c
mv -f .deps/ldmain.Tpo .deps/ldmain.Po
gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT ldwrite.o -MD -MP -MF .deps/ldwrite.Tpo -c -o ldwrite.o ../../master/ld/ldwrite.c
-@@ -2665,7 +2644,7 @@
+@@ -2680,7 +2659,7 @@
gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT ldmisc.o -MD -MP -MF .deps/ldmisc.Tpo -c -o ldmisc.o ../../master/ld/ldmisc.c
mv -f .deps/ldmisc.Tpo .deps/ldmisc.Po
gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT ldfile.o -MD -MP -MF .deps/ldfile.Tpo -c -o ldfile.o \
@@ -602,19 +616,22 @@
../../master/ld/ldfile.c
mv -f .deps/ldfile.Tpo .deps/ldfile.Po
gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT ldcref.o -MD -MP -MF .deps/ldcref.Tpo -c -o ldcref.o ../../master/ld/ldcref.c
-@@ -2673,14 +2652,11 @@
+@@ -2688,17 +2667,11 @@
gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT plugin.o -MD -MP -MF .deps/plugin.Tpo -c -o plugin.o ../../master/ld/plugin.c
mv -f .deps/plugin.Tpo .deps/plugin.Po
cp ../../master/ld/emultempl/astring.sed stringify.sed
--LIB_PATH='' /bin/bash ../../master/ld/genscripts.sh "../../master/ld" "[...]/hurd/master.build.install/lib" "[...]/hurd/master.build.install" "[...]/hurd/master.build.install" i686-pc-linux-gnu i686-pc-linux-gnu i686-pc-linux-gnu "elf_i386" "/usr/local/lib /lib /usr/lib" no yes elf_i386 "i686-pc-linux-gnu"
+-LIB_PATH='' /bin/bash ../../master/ld/genscripts.sh "../../master/ld" "[...]/hurd/master.build.install/lib" "[...]/hurd/master.build.install" "[...]/hurd/master.build.install" i686-pc-linux-gnu i686-pc-linux-gnu i686-pc-linux-gnu "elf_i386 elf32_x86_64" "/usr/local/lib /lib /usr/lib" no yes elf_i386 "i686-pc-linux-gnu"
+LIB_PATH='' /bin/bash ../../master/ld/genscripts.sh "../../master/ld" "[...]/hurd/master.build.install/lib" "[...]/hurd/master.build.install" "[...]/hurd/master.build.install" i686-unknown-gnu0.3 i686-unknown-gnu0.3 i686-unknown-gnu0.3 "elf_i386" "/usr/local/lib /lib /usr/lib" no yes elf_i386 "i686-unknown-gnu0.3"
gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT eelf_i386.o -MD -MP -MF .deps/eelf_i386.Tpo -c -o eelf_i386.o eelf_i386.c
mv -f .deps/eelf_i386.Tpo .deps/eelf_i386.Po
--LIB_PATH='' /bin/bash ../../master/ld/genscripts.sh "../../master/ld" "[...]/hurd/master.build.install/lib" "[...]/hurd/master.build.install" "[...]/hurd/master.build.install" i686-pc-linux-gnu i686-pc-linux-gnu i686-pc-linux-gnu "elf_i386" "/usr/local/lib /lib /usr/lib" no yes i386linux "i686-pc-linux-gnuaout"
+-LIB_PATH='' /bin/bash ../../master/ld/genscripts.sh "../../master/ld" "[...]/hurd/master.build.install/lib" "[...]/hurd/master.build.install" "[...]/hurd/master.build.install" i686-pc-linux-gnu i686-pc-linux-gnu i686-pc-linux-gnu "elf_i386 elf32_x86_64" "/usr/local/lib /lib /usr/lib" no yes i386linux "i686-pc-linux-gnuaout"
-gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT ei386linux.o -MD -MP -MF .deps/ei386linux.Tpo -c -o ei386linux.o ei386linux.c
-mv -f .deps/ei386linux.Tpo .deps/ei386linux.Po
--/bin/bash ./libtool --tag=CC --mode=link gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -o ld-new ldgram.o ldlex-wrapper.o lexsup.o ldlang.o mri.o ldctor.o ldmain.o ldwrite.o ldexp.o ldemul.o ldver.o ldmisc.o ldfile.o ldcref.o plugin.o eelf_i386.o ei386linux.o ../bfd/libbfd.la ../libiberty/libiberty.a -lz -ldl
--libtool: link: gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -o ld-new ldgram.o ldlex-wrapper.o lexsup.o ldlang.o mri.o ldctor.o ldmain.o ldwrite.o ldexp.o ldemul.o ldver.o ldmisc.o ldfile.o ldcref.o plugin.o eelf_i386.o ei386linux.o ../bfd/.libs/libbfd.a ../libiberty/libiberty.a -lz -ldl
+-LIB_PATH='' /bin/bash ../../master/ld/genscripts.sh "../../master/ld" "[...]/hurd/master.build.install/lib" "[...]/hurd/master.build.install" "[...]/hurd/master.build.install" i686-pc-linux-gnu i686-pc-linux-gnu i686-pc-linux-gnu "elf_i386 elf32_x86_64" "/usr/local/lib /lib /usr/lib" no yes elf32_x86_64 "i686-pc-linux-gnu"
+-gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT eelf32_x86_64.o -MD -MP -MF .deps/eelf32_x86_64.Tpo -c -o eelf32_x86_64.o eelf32_x86_64.c
+-mv -f .deps/eelf32_x86_64.Tpo .deps/eelf32_x86_64.Po
+-/bin/bash ./libtool --tag=CC --mode=link gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -o ld-new ldgram.o ldlex-wrapper.o lexsup.o ldlang.o mri.o ldctor.o ldmain.o ldwrite.o ldexp.o ldemul.o ldver.o ldmisc.o ldfile.o ldcref.o plugin.o eelf_i386.o ei386linux.o eelf32_x86_64.o ../bfd/libbfd.la ../libiberty/libiberty.a -lz -ldl
+-libtool: link: gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -o ld-new ldgram.o ldlex-wrapper.o lexsup.o ldlang.o mri.o ldctor.o ldmain.o ldwrite.o ldexp.o ldemul.o ldver.o ldmisc.o ldfile.o ldcref.o plugin.o eelf_i386.o ei386linux.o eelf32_x86_64.o ../bfd/.libs/libbfd.a ../libiberty/libiberty.a -lz -ldl
+/bin/bash ./libtool --tag=CC --mode=link gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -o ld-new ldgram.o ldlex-wrapper.o lexsup.o ldlang.o mri.o ldctor.o ldmain.o ldwrite.o ldexp.o ldemul.o ldver.o ldmisc.o ldfile.o ldcref.o plugin.o eelf_i386.o ../bfd/libbfd.la ../libiberty/libiberty.a -lz -ldl
+libtool: link: gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -o ld-new ldgram.o ldlex-wrapper.o lexsup.o ldlang.o mri.o ldctor.o ldmain.o ldwrite.o ldexp.o ldemul.o ldver.o ldmisc.o ldfile.o ldcref.o plugin.o eelf_i386.o ../bfd/.libs/libbfd.a ../libiberty/libiberty.a -lz -ldl
touch ld.1
diff --git a/open_issues/binutils/log_install-diff b/open_issues/binutils/log_install-diff
index 83c8d7b6..00496f8b 100644
--- a/open_issues/binutils/log_install-diff
+++ b/open_issues/binutils/log_install-diff
@@ -1,5 +1,5 @@
---- /dev/fd/63 2010-12-20 19:00:16.368493004 +0100
-+++ /dev/fd/62 2010-12-20 19:00:16.368493004 +0100
+--- /dev/fd/63 2011-02-10 18:56:20.086225001 +0100
++++ /dev/fd/62 2011-02-10 18:56:20.086225001 +0100
@@ -68,7 +68,6 @@
libtool: install: /usr/bin/install -c .libs/libbfd.a [...]/hurd/master.build.install/lib/libbfd.a
libtool: install: chmod 644 [...]/hurd/master.build.install/lib/libbfd.a
diff --git a/open_issues/binutils/sum_hurd b/open_issues/binutils/sum_hurd
index 96dd0cb2..15d225f9 100644
--- a/open_issues/binutils/sum_hurd
+++ b/open_issues/binutils/sum_hurd
@@ -1,4 +1,4 @@
-Test Run By tschwinge on Mon Dec 20 11:35:47 2010
+Test Run By tschwinge on Thu Feb 10 18:58:16 2011
Native configuration is i686-unknown-gnu0.3
=== binutils tests ===
@@ -114,7 +114,7 @@ Running [...]/hurd/master/binutils/testsuite/binutils-all/x86-64/x86-64.exp ...
# of expected passes 83
# of unsupported tests 2
-Test Run By tschwinge on Mon Dec 20 11:44:29 2010
+Test Run By tschwinge on Thu Feb 10 19:06:15 2011
Native configuration is i686-unknown-gnu0.3
=== ld tests ===
@@ -640,6 +640,8 @@ PASS: ld-ifunc/ifunc-1-local-x86
PASS: ld-ifunc/ifunc-1-x86
PASS: ld-ifunc/ifunc-10-i386
PASS: ld-ifunc/ifunc-11-i386
+PASS: ld-ifunc/ifunc-12-i386
+PASS: ld-ifunc/ifunc-13-i386
PASS: ld-ifunc/ifunc-2-i386
PASS: ld-ifunc/ifunc-2-local-i386
PASS: ld-ifunc/ifunc-3a-x86
@@ -669,6 +671,8 @@ Running [...]/hurd/master/ld/testsuite/ld-m68k/m68k.exp ...
Running [...]/hurd/master/ld/testsuite/ld-mep/mep.exp ...
Running [...]/hurd/master/ld/testsuite/ld-mips-elf/mips-elf-flags.exp ...
Running [...]/hurd/master/ld/testsuite/ld-mips-elf/mips-elf.exp ...
+Running [...]/hurd/master/ld/testsuite/ld-misc/defsym.exp ...
+PASS: ld-misc/defsym1
Running [...]/hurd/master/ld/testsuite/ld-mmix/mmix.exp ...
Running [...]/hurd/master/ld/testsuite/ld-mn10300/mn10300.exp ...
Running [...]/hurd/master/ld/testsuite/ld-pe/pe-compile.exp ...
@@ -705,6 +709,7 @@ Running [...]/hurd/master/ld/testsuite/ld-scripts/alignof.exp ...
PASS: ALIGNOF
Running [...]/hurd/master/ld/testsuite/ld-scripts/assert.exp ...
PASS: ASSERT
+PASS: ld-scripts/assert2
Running [...]/hurd/master/ld/testsuite/ld-scripts/crossref.exp ...
PASS: NOCROSSREFS 1
PASS: NOCROSSREFS 2
@@ -720,6 +725,8 @@ Running [...]/hurd/master/ld/testsuite/ld-scripts/defined.exp ...
PASS: DEFINED (PRMS 5699)
PASS: ld-scripts/defined2
PASS: ld-scripts/defined3
+PASS: ld-scripts/defined4
+PASS: ld-scripts/defined5
Running [...]/hurd/master/ld/testsuite/ld-scripts/dynamic-sections.exp ...
PASS: dynamic sections
Running [...]/hurd/master/ld/testsuite/ld-scripts/empty-address.exp ...
@@ -735,6 +742,8 @@ Running [...]/hurd/master/ld/testsuite/ld-scripts/empty-orphan.exp ...
PASS: ld-scripts/empty-orphan
Running [...]/hurd/master/ld/testsuite/ld-scripts/expr.exp ...
PASS: ld-scripts/expr1
+PASS: ld-scripts/expr2
+PASS: ld-scripts/sane1
Running [...]/hurd/master/ld/testsuite/ld-scripts/extern.exp ...
PASS: EXTERN
Running [...]/hurd/master/ld/testsuite/ld-scripts/include.exp ...
@@ -873,13 +882,13 @@ Running [...]/hurd/master/ld/testsuite/ld-xtensa/xtensa.exp ...
=== ld Summary ===
-# of expected passes 608
+# of expected passes 616
# of unexpected successes 1
# of expected failures 17
# of untested testcases 6
-/media/data[...]/hurd/master.build/ld/ld-new 2.21.51.20101220
+/media/data[...]/hurd/master.build/ld/ld-new 2.21.51.20110210
-Test Run By tschwinge on Mon Dec 20 11:38:03 2010
+Test Run By tschwinge on Thu Feb 10 19:00:16 2011
Native configuration is i686-unknown-gnu0.3
=== gas tests ===
@@ -961,8 +970,8 @@ PASS: CFI common 2
PASS: CFI common 3
PASS: CFI common 4
PASS: CFI common 5
-PASS: CFI common 7
PASS: CFI common 6
+PASS: CFI common 7
Running [...]/hurd/master/gas/testsuite/gas/cr16/cr16.exp ...
Running [...]/hurd/master/gas/testsuite/gas/cr16/pic.exp ...
Running [...]/hurd/master/gas/testsuite/gas/cris/cris.exp ...
@@ -1001,6 +1010,7 @@ PASS: section flags
PASS: DWARF2 1
PASS: DWARF2 2
PASS: DWARF2 3
+PASS: DWARF2 4
PASS: Check bad section flag
Running [...]/hurd/master/gas/testsuite/gas/fr30/allinsn.exp ...
Running [...]/hurd/master/gas/testsuite/gas/fr30/fr30.exp ...
@@ -1094,8 +1104,10 @@ PASS: i386 -mtune=i686 nops 3
PASS: i386 nops 4
PASS: i386 nops -mtune=i386 4
PASS: i386 -mtune=i686 nops 4
+PASS: i386 -march=i686+nop nops 4a
PASS: i386 nops 5
PASS: i386 -march=i686 nops 5
+PASS: i386 nops 6
PASS: i386 16-bit addressing in 32-bit mode.
PASS: i386 32-bit addressing in 16-bit mode.
PASS: i386 SSE4.1
@@ -1176,6 +1188,10 @@ PASS: i386 FMA scalar insns (Intel disassembly)
PASS: i386 FMA4
PASS: i386 LWP
PASS: i386 XOP
+PASS: i386 BMI insns
+PASS: i386 BMI insns (Intel disassembly)
+PASS: i386 TBM
+PASS: i386 TBM insns (Intel disassembly)
PASS: i386 F16C
PASS: i386 F16C (Intel disassembly)
PASS: i386 FSGSBase
@@ -1217,6 +1233,10 @@ PASS: i386 list-1
PASS: i386 list-2
PASS: i386 list-3
PASS: DWARF2 debugging information 1
+Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/cfi/ilp32.exp ...
+Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/elf/ilp32.exp ...
+Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/ilp32.exp ...
+Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/lns/ilp32.exp ...
Running [...]/hurd/master/gas/testsuite/gas/i860/i860.exp ...
Running [...]/hurd/master/gas/testsuite/gas/ia64/ia64.exp ...
Running [...]/hurd/master/gas/testsuite/gas/ieee-fp/x930509a.exp ...
@@ -1317,6 +1337,6 @@ Running [...]/hurd/master/gas/testsuite/gas/z8k/z8k.exp ...
=== gas Summary ===
-# of expected passes 319
-../as-new 2.21.51.20101220
+# of expected passes 326
+../as-new 2.21.51.20110210
diff --git a/open_issues/binutils/sum_linux b/open_issues/binutils/sum_linux
index c2dae925..49cf53fb 100644
--- a/open_issues/binutils/sum_linux
+++ b/open_issues/binutils/sum_linux
@@ -1,4 +1,4 @@
-Test Run By thomas on Mon Dec 20 11:34:53 2010
+Test Run By thomas on Thu Feb 10 18:57:42 2011
Native configuration is i686-pc-linux-gnu
=== binutils tests ===
@@ -114,7 +114,7 @@ Running [...]/hurd/master/binutils/testsuite/binutils-all/x86-64/x86-64.exp ...
# of expected passes 83
# of unsupported tests 2
-Test Run By thomas on Mon Dec 20 11:35:19 2010
+Test Run By thomas on Thu Feb 10 18:58:10 2011
Native configuration is i686-pc-linux-gnu
=== ld tests ===
@@ -640,6 +640,8 @@ PASS: ld-ifunc/ifunc-1-local-x86
PASS: ld-ifunc/ifunc-1-x86
PASS: ld-ifunc/ifunc-10-i386
PASS: ld-ifunc/ifunc-11-i386
+PASS: ld-ifunc/ifunc-12-i386
+PASS: ld-ifunc/ifunc-13-i386
PASS: ld-ifunc/ifunc-2-i386
PASS: ld-ifunc/ifunc-2-local-i386
PASS: ld-ifunc/ifunc-3a-x86
@@ -669,6 +671,8 @@ Running [...]/hurd/master/ld/testsuite/ld-m68k/m68k.exp ...
Running [...]/hurd/master/ld/testsuite/ld-mep/mep.exp ...
Running [...]/hurd/master/ld/testsuite/ld-mips-elf/mips-elf-flags.exp ...
Running [...]/hurd/master/ld/testsuite/ld-mips-elf/mips-elf.exp ...
+Running [...]/hurd/master/ld/testsuite/ld-misc/defsym.exp ...
+PASS: ld-misc/defsym1
Running [...]/hurd/master/ld/testsuite/ld-mmix/mmix.exp ...
Running [...]/hurd/master/ld/testsuite/ld-mn10300/mn10300.exp ...
Running [...]/hurd/master/ld/testsuite/ld-pe/pe-compile.exp ...
@@ -705,6 +709,7 @@ Running [...]/hurd/master/ld/testsuite/ld-scripts/alignof.exp ...
PASS: ALIGNOF
Running [...]/hurd/master/ld/testsuite/ld-scripts/assert.exp ...
PASS: ASSERT
+PASS: ld-scripts/assert2
Running [...]/hurd/master/ld/testsuite/ld-scripts/crossref.exp ...
PASS: NOCROSSREFS 1
PASS: NOCROSSREFS 2
@@ -720,6 +725,8 @@ Running [...]/hurd/master/ld/testsuite/ld-scripts/defined.exp ...
PASS: DEFINED (PRMS 5699)
PASS: ld-scripts/defined2
PASS: ld-scripts/defined3
+PASS: ld-scripts/defined4
+PASS: ld-scripts/defined5
Running [...]/hurd/master/ld/testsuite/ld-scripts/dynamic-sections.exp ...
PASS: dynamic sections
Running [...]/hurd/master/ld/testsuite/ld-scripts/empty-address.exp ...
@@ -735,6 +742,8 @@ Running [...]/hurd/master/ld/testsuite/ld-scripts/empty-orphan.exp ...
PASS: ld-scripts/empty-orphan
Running [...]/hurd/master/ld/testsuite/ld-scripts/expr.exp ...
PASS: ld-scripts/expr1
+PASS: ld-scripts/expr2
+PASS: ld-scripts/sane1
Running [...]/hurd/master/ld/testsuite/ld-scripts/extern.exp ...
PASS: EXTERN
Running [...]/hurd/master/ld/testsuite/ld-scripts/include.exp ...
@@ -873,12 +882,12 @@ Running [...]/hurd/master/ld/testsuite/ld-xtensa/xtensa.exp ...
=== ld Summary ===
-# of expected passes 618
+# of expected passes 626
# of expected failures 8
# of untested testcases 6
-/media/data[...]/hurd/master.build/ld/ld-new 2.21.51.20101220
+/media/data[...]/hurd/master.build/ld/ld-new 2.21.51.20110210
-Test Run By thomas on Mon Dec 20 11:34:59 2010
+Test Run By thomas on Thu Feb 10 18:57:49 2011
Native configuration is i686-pc-linux-gnu
=== gas tests ===
@@ -960,8 +969,8 @@ PASS: CFI common 2
PASS: CFI common 3
PASS: CFI common 4
PASS: CFI common 5
-PASS: CFI common 7
PASS: CFI common 6
+PASS: CFI common 7
Running [...]/hurd/master/gas/testsuite/gas/cr16/cr16.exp ...
Running [...]/hurd/master/gas/testsuite/gas/cr16/pic.exp ...
Running [...]/hurd/master/gas/testsuite/gas/cris/cris.exp ...
@@ -1000,6 +1009,7 @@ PASS: section flags
PASS: DWARF2 1
PASS: DWARF2 2
PASS: DWARF2 3
+PASS: DWARF2 4
PASS: Check bad section flag
Running [...]/hurd/master/gas/testsuite/gas/fr30/allinsn.exp ...
Running [...]/hurd/master/gas/testsuite/gas/fr30/fr30.exp ...
@@ -1093,8 +1103,10 @@ PASS: i386 -mtune=i686 nops 3
PASS: i386 nops 4
PASS: i386 nops -mtune=i386 4
PASS: i386 -mtune=i686 nops 4
+PASS: i386 -march=i686+nop nops 4a
PASS: i386 nops 5
PASS: i386 -march=i686 nops 5
+PASS: i386 nops 6
PASS: i386 16-bit addressing in 32-bit mode.
PASS: i386 32-bit addressing in 16-bit mode.
PASS: i386 SSE4.1
@@ -1175,6 +1187,10 @@ PASS: i386 FMA scalar insns (Intel disassembly)
PASS: i386 FMA4
PASS: i386 LWP
PASS: i386 XOP
+PASS: i386 BMI insns
+PASS: i386 BMI insns (Intel disassembly)
+PASS: i386 TBM
+PASS: i386 TBM insns (Intel disassembly)
PASS: i386 F16C
PASS: i386 F16C (Intel disassembly)
PASS: i386 FSGSBase
@@ -1216,6 +1232,10 @@ PASS: i386 list-1
PASS: i386 list-2
PASS: i386 list-3
PASS: DWARF2 debugging information 1
+Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/cfi/ilp32.exp ...
+Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/elf/ilp32.exp ...
+Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/ilp32.exp ...
+Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/lns/ilp32.exp ...
Running [...]/hurd/master/gas/testsuite/gas/i860/i860.exp ...
Running [...]/hurd/master/gas/testsuite/gas/ia64/ia64.exp ...
Running [...]/hurd/master/gas/testsuite/gas/ieee-fp/x930509a.exp ...
@@ -1316,6 +1336,6 @@ Running [...]/hurd/master/gas/testsuite/gas/z8k/z8k.exp ...
=== gas Summary ===
-# of expected passes 319
-../as-new 2.21.51.20101220
+# of expected passes 326
+../as-new 2.21.51.20110210
diff --git a/open_issues/gnumach_console_timestamp.mdwn b/open_issues/gnumach_console_timestamp.mdwn
new file mode 100644
index 00000000..b36b47b9
--- /dev/null
+++ b/open_issues/gnumach_console_timestamp.mdwn
@@ -0,0 +1,27 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_gnumach]]
+
+IRC, freenode, #hurd, 2011-02-17
+
+ <azeem> task 39011c10 deallocating an invalid port 349, most probably a
+ bug.
+ <azeem> kernel: Page fault (14), code=6
+ <azeem> Stopped at 0x28b9c7: orb %bh,0(%ecx,%edi,2)
+ <azeem> db>
+ [...]
+ <antrik> tschwinge: I doubt the deallocating warning is related to the
+ later fault
+ <tschwinge> antrik: YOu may be right.
+ <tschwinge> Perhaps it'd be a good idea to add some sort of timestamp to
+ Mach messages.
+ <tschwinge> Like in Linux' dmesg.
+ <tschwinge> Or just RDTSC (internal processor counter).
diff --git a/open_issues/implementing_hurd_on_top_of_another_system.mdwn b/open_issues/implementing_hurd_on_top_of_another_system.mdwn
index 23512aa9..95b71ebb 100644
--- a/open_issues/implementing_hurd_on_top_of_another_system.mdwn
+++ b/open_issues/implementing_hurd_on_top_of_another_system.mdwn
@@ -78,3 +78,40 @@ IRC, #hurd, 2010-12-28
<antrik> though I must say that I'm more and more convinced running the
Hurd on top of a monolithic kernel would actually be a useful approach
for the time being...
+
+---
+
+IRC, #hurd, 2011-02-11
+
+ <neal> marcus and I were discussing how to add Mach to Linux
+ <neal> one could write a module to implement Mach IPC
+ <neal> and another to implement Mach VM
+ <neal> the big thing missing with Mach VM is the ability for a tracing
+ process to easily map or unmap an inferior process's memory
+ <antrik> neal: why would a tracing process need to map the inferior's
+ memory?
+ <neal> the simple answer is that is how it is done on Mach
+ <antrik> neal: is it? not sure we are talking about the same thing
+ here. GDB uses vm_read()/vm_write() to access the inferior's memory AFAIK
+ <neal> on linux?
+ <neal> I think it use /proc/pid/mem
+ <antrik> on Hurd
+ <neal> I'm talking about adding Mach to Linux
+ <neal> by adding some functionality to Linux
+ <neal> and then implementing a bunch in user space
+ <antrik> yeah, but I don't understand the point about mapping inferior's
+ memory :-(
+ <antrik> what would be in user space?
+ <neal> there are a number of different cut points
+ <neal> one could imagine just using Linux's device drivers, CPU scheduler,
+ memory management, etc.
+ <neal> another possibility would be something higher where Hurd processes
+ just use some Hurdish servers
+ <antrik> neal: yeah, these are all options I have been considering... too
+ bad I wasn't able to come to FOSDEM -- I'd love to have participated in
+ this discussion :-(
+ <antrik> neal: BTW, were you just discussing this as a hypothetical idea,
+ or something you are seriously considering?
+ <neal> I'm unlikely to work on it, sorry
+ <antrik> didn't really expect that :-)
+ <antrik> would be nice though if you could write up your conclusions...
diff --git a/open_issues/performance/io_system.mdwn b/open_issues/performance/io_system.mdwn
index 0d41d3c7..4af093ba 100644
--- a/open_issues/performance/io_system.mdwn
+++ b/open_issues/performance/io_system.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation,
+[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation,
Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
@@ -6,8 +6,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
-is included in the section entitled
-[[GNU Free Documentation License|/fdl]]."]]"""]]
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
[[!meta title="I/O System"]]
@@ -20,7 +20,8 @@ slow hard disk access.
The reason for this slowness is lack and/or bad implementation of common
optimization techniques, like scheduling reads and writes to minimize head
movement; effective block caching; effective reads/writes to partial blocks;
-reading/writing multiple blocks at once; and read-ahead. The
+[[reading/writing multiple blocks at once|clustered_page_faults]]; and
+[[read-ahead]]. The
[[ext2_filesystem_server|hurd/translator/ext2fs]] might also need some
optimizations at a higher logical level.
@@ -30,7 +31,7 @@ requires understanding the data flow through the various layers involved in
disk access on the Hurd ([[filesystem|hurd/virtual_file_system]],
[[pager|hurd/libpager]], driver), and general experience with
optimizing complex systems. That said, the killing feature we are definitely
-missing is the read-ahead, and even a very simple implementation would bring
+missing is the [[read-ahead]], and even a very simple implementation would bring
very big performance speedups.
Here are some real testcases:
diff --git a/open_issues/performance/io_system/clustered_page_faults.mdwn b/open_issues/performance/io_system/clustered_page_faults.mdwn
new file mode 100644
index 00000000..3a187523
--- /dev/null
+++ b/open_issues/performance/io_system/clustered_page_faults.mdwn
@@ -0,0 +1,103 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_gnumach open_issue_hurd]]
+
+IRC, freenode, #hurd, 2011-02-16
+
+ <braunr> exceptfor the kernel, everything in an address space is
+ represented with a VM object
+ <braunr> those objects can represent anonymous memory (from malloc() or
+ because of a copy-on-write)
+ <braunr> or files
+ <braunr> on classic Unix systems, these are files
+ <braunr> on the Hurd, these are memory objects, backed by external pagers
+ (like ext2fs)
+ <braunr> so when you read a file
+ <braunr> the kernel maps it from ext2fs in your address space
+ <braunr> and when you access the memory, a fault occurs
+ <braunr> the kernel determines it's a region backed by ext2fs
+ <braunr> so it asks ext2fs to provide the data
+ <braunr> when the fault is resolved, your process goes on
+ <etenil> does the faul occur because Mach doesn't know how to access the
+ memory?
+ <braunr> it occurs because Mach intentionnaly didn't back the region with
+ physical memory
+ <braunr> the MMU is programmed not to know what is present in the memory
+ region
+ <braunr> or because it's read only
+ <braunr> (which is the case for COW faults)
+ <etenil> so that means this bit of memory is a buffer that ext2fs loads the
+ file into and then it is remapped to the application that asked for it
+ <braunr> more or less, yes
+ <braunr> ideally, it's directly written into the right pages
+ <braunr> there is no intermediate buffer
+ <etenil> I see
+ <etenil> and as you told me before, currently the page faults are handled
+ one at a time
+ <etenil> which wastes a lot of time
+ <braunr> a certain amount of time
+ <etenil> enough to bother the user :)
+ <etenil> I've seen pages have a fixed size
+ <braunr> yes
+ <braunr> use the PAGE_SIZE macro
+ <etenil> and when allocating memory, the size that's asked for is rounded
+ up to the page size
+ <etenil> so if I have this correctly, it means that a file ext2fs provides
+ could be split into a lot of pages
+ <braunr> yes
+ <braunr> once in memory, it is managed by the page cache
+ <braunr> so that pages more actively used are kept longer than others
+ <braunr> in order to minimize I/O
+ <etenil> ok
+ <braunr> so a better page cache code would also improve overall performance
+ <braunr> and more RAM would help a lot, since we are strongly limited by
+ the 768 MiB limit
+ <braunr> which reduces the page cache size a lot
+ <etenil> but the problem is that reading a whole file in means trigerring
+ many page faults just for one file
+ <braunr> if you want to stick to the page clustering thing, yes
+ <braunr> you want less page faults, so that there are less IPC between the
+ kernel and the pager
+ <etenil> so either I make pages bigger
+ <etenil> or I modify Mach so it can check up on a range of pages for faults
+ before actually processing
+ <braunr> you *don't* change the page size
+ <etenil> ah
+ <etenil> that's hardware isn't it?
+ <braunr> in Mach, yes
+ <etenil> ok
+ <braunr> and usually, you want the page size to be the CPU page size
+ <etenil> I see
+ <braunr> current CPU can support multiple page sizes, but it becomes quite
+ hard to correctly handle
+ <braunr> and bigger page sizes mean more fragmentation, so it only suits
+ machines with large amounts of RAM, which isn't the case for us
+ <etenil> ok
+ <etenil> so I'll try the second approach then
+ <braunr> that's what i'd recommand
+ <braunr> recommend*
+ <etenil> ok
+
+---
+
+IRC, freenode, #hurd, 2011-02-16
+
+ <antrik> etenil: OSF Mach does have clustered paging BTW; so that's one
+ place to start looking...
+ <antrik> (KAM ported the OSF code to gnumach IIRC)
+ <antrik> there is also an existing patch for clustered paging in libpager,
+ which needs some adaptation
+ <antrik> the biggest part of the task is probably modifying the Hurd
+ servers to use the new interface
+ <antrik> but as I said, KAM's code should be available through google, and
+ can serve as a starting point
+
+<http://lists.gnu.org/archive/html/bug-hurd/2010-06/msg00023.html>
diff --git a/open_issues/performance/io_system/read-ahead.mdwn b/open_issues/performance/io_system/read-ahead.mdwn
new file mode 100644
index 00000000..3ee30b5d
--- /dev/null
+++ b/open_issues/performance/io_system/read-ahead.mdwn
@@ -0,0 +1,299 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_gnumach open_issue_hurd]]
+
+IRC, #hurd, freenode, 2011-02-13:
+
+ <etenil> youpi: Would libdiskfs/diskfs.h be in the right place to make
+ readahead functions?
+ <youpi> etenil: no, it'd rather be at the memory management layer,
+ i.e. mach, unfortunately
+ <youpi> because that's where you see the page faults
+ <etenil> youpi: Linux also provides a readahead() function for higher level
+ applications. I'll probably have to add the same thing in a place that's
+ higher level than mach
+ <youpi> well, that should just be hooked to the same common implementation
+ <etenil> the man page for readahead() also states that portable
+ applications should avoid it, but it could be benefic to have it for
+ portability
+ <youpi> it's not in posix indeed
+
+---
+
+IRC, #hurd, freenode, 2011-02-14:
+
+ <etenil> youpi: I've investigated prefetching (readahead) techniques. One
+ called DiskSeen seems really efficient. I can't tell yet if it's patented
+ etc. but I'll keep you informed
+ <youpi> don't bother with complicated techniques, even the most simple ones
+ will be plenty :)
+ <etenil> it's not complicated really
+ <youpi> the matter is more about how to plug it into mach
+ <etenil> ok
+ <youpi> then don't bother with potential pattents
+ <antrik> etenil: please take a look at the work KAM did for last year's
+ GSoC
+ <youpi> just use a trivial technique :)
+ <etenil> ok, i'll just go the easy way then
+
+ <braunr> antrik: what was etenil referring to when talking about
+ prefetching ?
+ <braunr> oh, madvise() stuff
+ <braunr> i could help him with that
+
+---
+
+[[Etenil]] is now working in this area.
+
+---
+
+IRC, freenode, #hurd, 2011-02-15
+
+ <etenil> oh, I'm looking into prefetching/readahead to improve I/O
+ performance
+ <braunr> etenil: ok
+ <braunr> etenil: that's actually a VM improvement, like samuel told you
+ <etenil> yes
+ <braunr> a true I/O improvement would be I/O scheduling
+ <braunr> and how to implement it in a hurdish way
+ <braunr> (or if it makes sense to have it in the kernel)
+ <etenil> that's what I've been wondering too lately
+ <braunr> concerning the VM, you should look at madvise()
+ <etenil> my understanding is that Mach considers devices without really
+ knowing what they are
+ <braunr> that's roughly the interface used both at the syscall() and the
+ kernel levels in BSD, which made it in many other unix systems
+ <etenil> whereas I/O optimisations are often hard disk drives specific
+ <braunr> that's true for almost any kernel
+ <braunr> the device knowledge is at the driver level
+ <etenil> yes
+ <braunr> (here, I separate kernels from their drivers ofc)
+ <etenil> but Mach also contains some drivers, so I'm going through the code
+ to find the apropriate place for these improvements
+ <braunr> you shouldn't tough the drivers at all
+ <braunr> touch
+ <etenil> true, but I need to understand how it works before fiddling around
+ <braunr> hm
+ <braunr> not at all
+ <braunr> the VM improvement is about pagein clustering
+ <braunr> you don't need to know how pages are fetched
+ <braunr> well, not at the device level
+ <braunr> you need to know about the protocol between the kernel and
+ external pagers
+ <etenil> ok
+ <braunr> you could also implement pageout clustering
+ <etenil> if I understand you well, you say that what I'd need to do is a
+ queuing system for the paging in the VM?
+ <braunr> no
+ <braunr> i'm saying that, when a page fault occurs, the kernel should
+ (depending on what was configured through madvise()) transfer pages in
+ multiple blocks rather than one at a time
+ <braunr> communication with external pagers is already async, made through
+ regular ports
+ <braunr> which already implement message queuing
+ <braunr> you would just need to make the mapped regions larger
+ <braunr> and maybe change the interface so that this size is passed
+ <etenil> mmh
+ <braunr> (also don't forget that page clustering can include pages *before*
+ the page which caused the fault, so you may have to pass the start of
+ that region too)
+ <etenil> I'm not sure I understand the page fault thing
+ <etenil> is it like a segmentation error?
+ <etenil> I can't find a clear definition in Mach's manual
+ <braunr> ah
+ <braunr> it's a fundamental operating system concept
+ <braunr> http://en.wikipedia.org/wiki/Page_fault
+ <etenil> ah ok
+ <etenil> I understand now
+ <etenil> so what's currently happening is that when a page fault occurs,
+ Mach is transfering pages one at a time and wastes time
+ <braunr> sometimes, transferring just one page is what you want
+ <braunr> it depends on the application, which is why there is madvise()
+ <braunr> our rootfs, on the other hand, would benefit much from such an
+ improvement
+ <braunr> in UVM, this optimization is account for around 10% global
+ performance improvement
+ <braunr> accounted*
+ <etenil> not bad
+ <braunr> well, with an improved page cache, I'm sure I/O would matter less
+ on systems with more RAM
+ <braunr> (and another improvement would make mach support more RAM in the
+ first place !)
+ <braunr> an I/O scheduler outside the kernel would be a very good project
+ IMO
+ <braunr> in e.g. libstore/storeio
+ <etenil> yes
+ <braunr> but as i stated in my thesis, a resource scheduler should be as
+ close to its resource as it can
+ <braunr> and since mach can host several operating systems, I/O schedulers
+ should reside near device drivers
+ <braunr> and since current drivers are in the kernel, it makes sens to have
+ it in the kernel too
+ <braunr> so there must be some discussion about this
+ <etenil> doesn't this mean that we'll have to get some optimizations in
+ Mach and have the same outside of Mach for translators that access the
+ hardware directly?
+ <braunr> etenil: why ?
+ <etenil> well as you said Mach contains some drivers, but in principle, it
+ shouldn't, translators should do disk access etc, yes?
+ <braunr> etenil: ok
+ <braunr> etenil: so ?
+ <etenil> well, let's say if one were to introduce SATA support in Hurd,
+ nothing would stop him/her to do so with a translator rather than in Mach
+ <braunr> you should avoid the term translator here
+ <braunr> it's really hurd specific
+ <braunr> let's just say a user space task would be responsible for that
+ job, maybe multiple instances of it, yes
+ <etenil> ok, so in this case, let's say we have some I/O optimization
+ techniques like readahead and I/O scheduling within Mach, would these
+ also apply to the user-space task, or would they need to be
+ reimplemented?
+ <braunr> if you have user space drivers, there is no point having I/O
+ scheduling in the kernel
+ <etenil> but we also have drivers within the kernel
+ <braunr> what you call readahead, and I call pagein/out clustering, is
+ really tied to the VM, so it must be in Mach in any case
+ <braunr> well
+ <braunr> you either have one or the other
+ <braunr> currently we have them in the kernel
+ <braunr> if we switch to DDE, we should have all of them outside
+ <braunr> that's why such things must be discussed
+ <etenil> ok so if I follow you, then future I/O device drivers will need to
+ be implemented for Mach
+ <braunr> currently, yes
+ <braunr> but preferrably, someone should continue the work that has been
+ done on DDe so that drivers are outside the kernel
+ <etenil> so for the time being, I will try and improve I/O in Mach, and if
+ drivers ever get out, then some of the I/O optimizations will need to be
+ moved out of Mach
+ <braunr> let me remind you one of the things i said
+ <braunr> i said I/O scheduling should be close to their resource, because
+ we can host several operating systems
+ <braunr> now, the Hurd is the only system running on top of Mach
+ <braunr> so we could just have I/O scheduling outside too
+ <braunr> then you should consider neighbor hurds
+ <braunr> which can use different partitions, but on the same device
+ <braunr> currently, partitions are managed in the kernel, so file systems
+ (and storeio) can't make good scheduling decisions if it remains that way
+ <braunr> but that can change too
+ <braunr> a single storeio representing a whole disk could be shared by
+ several hurd instances, just as if it were a high level driver
+ <braunr> then you could implement I/O scheduling in storeio, which would be
+ an improvement for the current implementation, and reusable for future
+ work
+ <etenil> yes, that was my first instinct
+ <braunr> and you would be mostly free of the kernel internals that make it
+ a nightmare
+ <etenil> but youpi said that it would be better to modify Mach instead
+ <braunr> he mentioned the page clustering thing
+ <braunr> not I/O scheduling
+ <braunr> theseare really two different things
+ <etenil> ok
+ <braunr> you *can't* implement page clustering outside Mach because Mach
+ implements virtual memory
+ <braunr> both policies and mechanisms
+ <etenil> well, I'd rather think of one thing at a time if that's alright
+ <etenil> so what I'm busy with right now is setting up clustered page-in
+ <etenil> which need to be done within Mach
+ <braunr> keep clustered page-outs in mind too
+ <braunr> although there are more constraints on those
+ <etenil> yes
+ <etenil> I've looked up madvise(). There's a lot of documentation about it
+ in Linux but I couldn't find references to it in Mach (nor Hurd), does it
+ exist?
+ <braunr> well, if it did, you wouldn't be caring about clustered page
+ transfers, would you ?
+ <braunr> be careful about linux specific stuff
+ <etenil> I suppose not
+ <braunr> you should implement at least posix options, and if there are
+ more, consider the bsd variants
+ <braunr> (the Mach VM is the ancestor of all modern BSD VMs)
+ <etenil> madvise() seems to be posix
+ <braunr> there are system specific extensions
+ <braunr> be careful
+ <braunr> CONFORMING TO POSIX.1b. POSIX.1-2001 describes posix_madvise(3)
+ with constants POSIX_MADV_NORMAL, etc., with a behav‐ ior close to that
+ described here. There is a similar posix_fadvise(2) for file access.
+ <braunr> MADV_REMOVE, MADV_DONTFORK, MADV_DOFORK, MADV_HWPOISON,
+ MADV_MERGEABLE, and MADV_UNMERGEABLE are Linux- specific.
+ <etenil> I was about to post these
+ <etenil> ok, so basically madvise() allows tasks etc. to specify a usage
+ type for a chunk of memory, then I could apply the relevant I/O
+ optimization based on this
+ <braunr> that's it
+ <etenil> cool, then I don't need to worry about knowing what the I/O is
+ operating on, I just need to apply the optimizations as advised
+ <etenil> that's convenient
+ <etenil> ok I'll start working on this tonight
+ <etenil> making a basic readahead shouldn't be too hard
+ <braunr> readahead is a misleading name
+ <etenil> is pagein better?
+ <braunr> applies to too many things, doesn't include the case where
+ previous elements could be prefetched
+ <braunr> clustered page transfers is what i would use
+ <braunr> page prefetching maybe
+ <etenil> ok
+ <braunr> you should stick to something that's already used in the
+ literature since you're not inventing something new
+ <etenil> yes I've read a paper about prefetching
+ <etenil> ok
+ <etenil> thanks for your help braunr
+ <braunr> sure
+ <braunr> you're welcome
+ <antrik> braunr: madvise() is really the least important part of the
+ picture...
+ <antrik> very few applications actually use it. but pretty much all
+ applications will profit from clustered paging
+ <antrik> I would consider madvise() an optional goody, not an integral part
+ of the implementation
+ <antrik> etenil: you can find some stuff about KAM's work on
+ http://www.gnu.org/software/hurd/user/kam.html
+ <antrik> not much specific though
+ <etenil> thanks
+ <antrik> I don't remember exactly, but I guess there is also some
+ information on the mailing list. check the archives for last summer
+ <antrik> look for Karim Allah Ahmed
+ <etenil> antrik: I disagree, madvise gives me a good starting point, even
+ if eventually the optimisations should run even without it
+ <antrik> the code he wrote should be available from Google's summer of code
+ page somewhere...
+ <braunr> antrik: right, i was mentioning madvise() because the kernel (VM)
+ interface is pretty similar to the syscall
+ <braunr> but even a default policy would be nice
+ <antrik> etenil: I fear that many bits were discussed only on IRC... so
+ you'd better look through the IRC logs from last April onwards...
+ <etenil> ok
+
+ <etenil> at the beginning I thought I could put that into libstore
+ <etenil> which would have been fine
+
+ <antrik> BTW, I remembered now that KAM's GSoC application should have a
+ pretty good description of the necessary changes... unfortunately, these
+ are not publicly visible IIRC :-(
+
+---
+
+IRC, freenode, #hurd, 2011-02-16
+
+ <etenil> braunr: I've looked in the kernel to see where prefetching would
+ fit best. We talked of the VM yesterday, but I'm not sure about it. It
+ seems to me that the device part of the kernel makes more sense since
+ it's logically what manages devices, am I wrong?
+ <braunr> etenil: you are
+ <braunr> etenil: well
+ <braunr> etenil: drivers should already support clustered sector
+ read/writes
+ <etenil> ah
+ <braunr> but yes, there must be support in the drivers too
+ <braunr> what would really benefit the Hurd mostly concerns page faults, so
+ the right place is the VM subsystem
+
+[[clustered_page_faults]]