diff options
Diffstat (limited to 'open_issues')
| -rw-r--r-- | open_issues/binutils.mdwn | 47 | ||||
| -rw-r--r-- | open_issues/binutils/log_build-diff | 43 | ||||
| -rw-r--r-- | open_issues/binutils/log_install-diff | 4 | ||||
| -rw-r--r-- | open_issues/binutils/sum_hurd | 36 | ||||
| -rw-r--r-- | open_issues/binutils/sum_linux | 36 | ||||
| -rw-r--r-- | open_issues/gnumach_console_timestamp.mdwn | 27 | ||||
| -rw-r--r-- | open_issues/implementing_hurd_on_top_of_another_system.mdwn | 37 | ||||
| -rw-r--r-- | open_issues/performance/io_system.mdwn | 11 | ||||
| -rw-r--r-- | open_issues/performance/io_system/clustered_page_faults.mdwn | 103 | ||||
| -rw-r--r-- | open_issues/performance/io_system/read-ahead.mdwn | 299 |
10 files changed, 583 insertions, 60 deletions
diff --git a/open_issues/binutils.mdwn b/open_issues/binutils.mdwn index 81fafaca..ca7496f0 100644 --- a/open_issues/binutils.mdwn +++ b/open_issues/binutils.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2007, 2008, 2010 Free Software Foundation, +[[!meta copyright="Copyright © 2007, 2008, 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -9,7 +9,7 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -[[!tag open_issue_binutils]] +[[!tag stable_URL open_issue_binutils]] Here's what's to be done for maintaining GNU Binutils. @@ -30,15 +30,14 @@ though, as explained below. # Configuration -Last reviewed up to the [[Git mirror's e347ef3b343fc42ed312d5125047d59ae15df795 -(2010-12-20) sources|source_repositories/binutils]]. +Last reviewed up to the [[Git mirror's a446ef2f3862fb5f89c669b34a2b6a2ab943ff96 +(2011-02-10) sources|source_repositories/binutils]]. * Globally * a.out, COFF, PE image support and 64 bit support are not interesting. - * In the [[testsuite]]s, `.exp` and `.d` files very likely should not - only + * In the testsuites, `.exp` and `.d` files very likely should not only care for `*-*-linux*`, but also `*-*-gnu*`. (If the need to be conditionalized like this at all.) @@ -96,7 +95,7 @@ Last reviewed up to the [[Git mirror's e347ef3b343fc42ed312d5125047d59ae15df795 * `*-*-gnu*` TODO: resolve `crt0.o` vs. `crt1.o` issue. [[Testsuite - failures|testsuite#static]]. + failures|binutils#static]]. * `configure.tgt` @@ -109,7 +108,7 @@ Last reviewed up to the [[Git mirror's e347ef3b343fc42ed312d5125047d59ae15df795 # Build Here's a log of a binutils build run; this is from our [[Git -repository's 245f62b817ee31135a190793dddb340f04ac95e6 (2010-12-20) +repository's e8052e7548e0d5523f1764b7d3896ca000bfaed7 (2011-02-10) sources|source_repositories/binutils]], run on kepler.SCHWINGE and grubber. $ export LC_ALL=C @@ -121,7 +120,7 @@ sources|source_repositories/binutils]], run on kepler.SCHWINGE and grubber. (kepler.SCHWINGE defaults to using /bin/sh for libtool, grubber to /bin/bash; thus harmonized.) -On grubber, this takes roughly one hour. +On grubber, this needs roughly one hour, and takes up around 100 MiB. ## Analysis @@ -145,7 +144,7 @@ GNU/Linux defining `-DTRAD_CORE`, `-DHAVE_i386linux_vec` (kepler.SCHWINGE defaults to using /bin/sh, grubber to /bin/bash; thus harmonized.) -On grubber, this needs roughly 15 minutes, and takes up around 0.7 GiB. +On grubber, this needs roughly 5 minutes, and takes up around 60 MiB. ## Analysis @@ -170,12 +169,12 @@ On grubber, this takes roughly one hour. Comparing the results files, [[sum_linux]] to [[sum_hurd]]: $ diff -u -F ^Running open_issues/binutils/sum_linux open_issues/binutils/sum_hurd - --- open_issues/binutils/sum_linux 2010-12-20 19:01:06.000000000 +0100 - +++ open_issues/binutils/sum_hurd 2010-12-20 19:01:20.000000000 +0100 + --- open_issues/binutils/sum_linux 2011-02-10 19:01:56.000000000 +0100 + +++ open_issues/binutils/sum_hurd 2011-02-10 20:27:17.000000000 +0100 @@ -1,5 +1,5 @@ - -Test Run By thomas on Mon Dec 20 11:34:53 2010 + -Test Run By thomas on Thu Feb 10 18:57:42 2011 -Native configuration is i686-pc-linux-gnu - +Test Run By tschwinge on Mon Dec 20 11:35:47 2010 + +Test Run By tschwinge on Thu Feb 10 18:58:16 2011 +Native configuration is i686-unknown-gnu0.3 === binutils tests === @@ -184,9 +183,9 @@ Comparing the results files, [[sum_linux]] to [[sum_hurd]]: # of expected passes 83 # of unsupported tests 2 - -Test Run By thomas on Mon Dec 20 11:35:19 2010 + -Test Run By thomas on Thu Feb 10 18:58:10 2011 -Native configuration is i686-pc-linux-gnu - +Test Run By tschwinge on Mon Dec 20 11:44:29 2010 + +Test Run By tschwinge on Thu Feb 10 19:06:15 2011 +Native configuration is i686-unknown-gnu0.3 === ld tests === @@ -232,21 +231,21 @@ Comparing the results files, [[sum_linux]] to [[sum_hurd]]: PASS: ELF DSO small bar (size) PASS: ELF DSO foo with small bar (size) PASS: ELF DSO big bar (size) - @@ -873,13 +873,14 @@ Running [...]/hurd/master/ld/testsuite/l + @@ -882,13 +882,14 @@ Running [...]/hurd/master/ld/testsuite/l === ld Summary === - -# of expected passes 618 + -# of expected passes 626 -# of expected failures 8 - +# of expected passes 608 + +# of expected passes 616 +# of unexpected successes 1 +# of expected failures 17 # of untested testcases 6 - /media/data[...]/hurd/master.build/ld/ld-new 2.21.51.20101220 + /media/data[...]/hurd/master.build/ld/ld-new 2.21.51.20110210 - -Test Run By thomas on Mon Dec 20 11:34:59 2010 + -Test Run By thomas on Thu Feb 10 18:57:49 2011 -Native configuration is i686-pc-linux-gnu - +Test Run By tschwinge on Mon Dec 20 11:38:03 2010 + +Test Run By tschwinge on Thu Feb 10 19:00:16 2011 +Native configuration is i686-unknown-gnu0.3 === gas tests === @@ -255,7 +254,7 @@ Comparing the results files, [[sum_linux]] to [[sum_hurd]]: ## Analysis - * <a name="static">`FAIL: static [...]`</a> + * <a name="static"><!-- stable_URL -->`FAIL: static [...]`</a> The testsuite isn't prepared for using `crt0.o` instead of `crt1.o` depending on whether a static or dynamic executable is created. Documented @@ -269,7 +268,7 @@ Comparing the results files, [[sum_linux]] to [[sum_hurd]]: weakness|performance/io_system/binutils_ld_64ksec]]), so assuming some system load variation, the testsuite's timeout may trigger. - * <a name="weak">`FAIL: ELF weak [...]`</a> + * <a name="weak"><!-- stable_URL -->`FAIL: ELF weak [...]`</a> [[I|tschwinge]] suppose this is due to us having an override w.r.t. weak symbol handling in glibc, needed for our external [[/libpthread]]. TODO: diff --git a/open_issues/binutils/log_build-diff b/open_issues/binutils/log_build-diff index 802d510c..3408d97d 100644 --- a/open_issues/binutils/log_build-diff +++ b/open_issues/binutils/log_build-diff @@ -1,5 +1,5 @@ ---- /dev/fd/63 2010-12-20 11:34:03.204493002 +0100 -+++ /dev/fd/62 2010-12-20 11:34:03.208493002 +0100 +--- /dev/fd/63 2011-02-10 17:33:04.738225001 +0100 ++++ /dev/fd/62 2011-02-10 17:33:04.738225001 +0100 @@ -1,6 +1,6 @@ -checking build system type... i686-pc-linux-gnu -checking host system type... i686-pc-linux-gnu @@ -530,7 +530,7 @@ checking whether we are using the GNU C compiler... (cached) yes checking whether gcc accepts -g... (cached) yes checking for gcc option to accept ISO C89... (cached) none needed -@@ -2450,28 +2429,28 @@ +@@ -2453,28 +2432,28 @@ checking for BSD- or MS-compatible name lister (nm)... nm checking the name lister (nm) interface... BSD nm checking whether ln -s works... yes @@ -566,23 +566,37 @@ checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes -@@ -2555,13 +2534,13 @@ +@@ -2486,11 +2465,11 @@ + checking whether the g++ linker (ld) supports shared libraries... yes + checking for g++ option to produce PIC... -fPIC -DPIC + checking if g++ PIC flag -fPIC -DPIC works... yes +-checking if g++ static flag -static works... yes ++checking if g++ static flag -static works... no + checking if g++ supports -c -o file.o... yes + checking if g++ supports -c -o file.o... (cached) yes + checking whether the g++ linker (ld) supports shared libraries... yes +-checking dynamic linker characteristics... (cached) GNU/Linux ld.so ++checking dynamic linker characteristics... gnu0.3 ld.so + checking how to hardcode library paths into programs... immediate + checking whether NLS is requested... yes + checking for catalogs to be installed... bg da es fi fr ga id ja sv tr vi zh_CN zh_TW +@@ -2570,13 +2549,13 @@ /bin/bash ../../master/ld/../ylwrap ../../master/ld/ldgram.y y.tab.c ldgram.c y.tab.h ldgram.h y.output ldgram.output -- bison -y -d updating ldgram.h (echo "/* This file is automatically generated. DO NOT EDIT! */";\ -- for f in `echo " " eelf_i386.o ei386linux.o "" \ +- for f in `echo " " eelf_i386.o ei386linux.o eelf32_x86_64.o "" \ + for f in `echo " " eelf_i386.o "" \ | sed -e 's/ e/ ld/g' -e 's/ ld/ /g' -e 's/[.]o//g'`; do \ echo "extern ld_emulation_xfer_type ld_${f}_emulation;"; \ done;\ echo "";\ echo "#define EMULATION_LIST \\";\ -- for f in `echo " " eelf_i386.o ei386linux.o "" \ +- for f in `echo " " eelf_i386.o ei386linux.o eelf32_x86_64.o "" \ + for f in `echo " " eelf_i386.o "" \ | sed -e 's/ e/ ld/g' -e 's/ ld/ /g' -e 's/[.]o//g'`; do \ echo " &ld_${f}_emulation, \\"; \ done;\ -@@ -2650,8 +2629,8 @@ +@@ -2665,8 +2644,8 @@ mv -f .deps/ldctor.Tpo .deps/ldctor.Po gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT ldmain.o -MD -MP -MF .deps/ldmain.Tpo -c -o ldmain.o \ -DDEFAULT_EMULATION='"elf_i386"' \ @@ -593,7 +607,7 @@ ../../master/ld/ldmain.c mv -f .deps/ldmain.Tpo .deps/ldmain.Po gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT ldwrite.o -MD -MP -MF .deps/ldwrite.Tpo -c -o ldwrite.o ../../master/ld/ldwrite.c -@@ -2665,7 +2644,7 @@ +@@ -2680,7 +2659,7 @@ gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT ldmisc.o -MD -MP -MF .deps/ldmisc.Tpo -c -o ldmisc.o ../../master/ld/ldmisc.c mv -f .deps/ldmisc.Tpo .deps/ldmisc.Po gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT ldfile.o -MD -MP -MF .deps/ldfile.Tpo -c -o ldfile.o \ @@ -602,19 +616,22 @@ ../../master/ld/ldfile.c mv -f .deps/ldfile.Tpo .deps/ldfile.Po gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT ldcref.o -MD -MP -MF .deps/ldcref.Tpo -c -o ldcref.o ../../master/ld/ldcref.c -@@ -2673,14 +2652,11 @@ +@@ -2688,17 +2667,11 @@ gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT plugin.o -MD -MP -MF .deps/plugin.Tpo -c -o plugin.o ../../master/ld/plugin.c mv -f .deps/plugin.Tpo .deps/plugin.Po cp ../../master/ld/emultempl/astring.sed stringify.sed --LIB_PATH='' /bin/bash ../../master/ld/genscripts.sh "../../master/ld" "[...]/hurd/master.build.install/lib" "[...]/hurd/master.build.install" "[...]/hurd/master.build.install" i686-pc-linux-gnu i686-pc-linux-gnu i686-pc-linux-gnu "elf_i386" "/usr/local/lib /lib /usr/lib" no yes elf_i386 "i686-pc-linux-gnu" +-LIB_PATH='' /bin/bash ../../master/ld/genscripts.sh "../../master/ld" "[...]/hurd/master.build.install/lib" "[...]/hurd/master.build.install" "[...]/hurd/master.build.install" i686-pc-linux-gnu i686-pc-linux-gnu i686-pc-linux-gnu "elf_i386 elf32_x86_64" "/usr/local/lib /lib /usr/lib" no yes elf_i386 "i686-pc-linux-gnu" +LIB_PATH='' /bin/bash ../../master/ld/genscripts.sh "../../master/ld" "[...]/hurd/master.build.install/lib" "[...]/hurd/master.build.install" "[...]/hurd/master.build.install" i686-unknown-gnu0.3 i686-unknown-gnu0.3 i686-unknown-gnu0.3 "elf_i386" "/usr/local/lib /lib /usr/lib" no yes elf_i386 "i686-unknown-gnu0.3" gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT eelf_i386.o -MD -MP -MF .deps/eelf_i386.Tpo -c -o eelf_i386.o eelf_i386.c mv -f .deps/eelf_i386.Tpo .deps/eelf_i386.Po --LIB_PATH='' /bin/bash ../../master/ld/genscripts.sh "../../master/ld" "[...]/hurd/master.build.install/lib" "[...]/hurd/master.build.install" "[...]/hurd/master.build.install" i686-pc-linux-gnu i686-pc-linux-gnu i686-pc-linux-gnu "elf_i386" "/usr/local/lib /lib /usr/lib" no yes i386linux "i686-pc-linux-gnuaout" +-LIB_PATH='' /bin/bash ../../master/ld/genscripts.sh "../../master/ld" "[...]/hurd/master.build.install/lib" "[...]/hurd/master.build.install" "[...]/hurd/master.build.install" i686-pc-linux-gnu i686-pc-linux-gnu i686-pc-linux-gnu "elf_i386 elf32_x86_64" "/usr/local/lib /lib /usr/lib" no yes i386linux "i686-pc-linux-gnuaout" -gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT ei386linux.o -MD -MP -MF .deps/ei386linux.Tpo -c -o ei386linux.o ei386linux.c -mv -f .deps/ei386linux.Tpo .deps/ei386linux.Po --/bin/bash ./libtool --tag=CC --mode=link gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -o ld-new ldgram.o ldlex-wrapper.o lexsup.o ldlang.o mri.o ldctor.o ldmain.o ldwrite.o ldexp.o ldemul.o ldver.o ldmisc.o ldfile.o ldcref.o plugin.o eelf_i386.o ei386linux.o ../bfd/libbfd.la ../libiberty/libiberty.a -lz -ldl --libtool: link: gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -o ld-new ldgram.o ldlex-wrapper.o lexsup.o ldlang.o mri.o ldctor.o ldmain.o ldwrite.o ldexp.o ldemul.o ldver.o ldmisc.o ldfile.o ldcref.o plugin.o eelf_i386.o ei386linux.o ../bfd/.libs/libbfd.a ../libiberty/libiberty.a -lz -ldl +-LIB_PATH='' /bin/bash ../../master/ld/genscripts.sh "../../master/ld" "[...]/hurd/master.build.install/lib" "[...]/hurd/master.build.install" "[...]/hurd/master.build.install" i686-pc-linux-gnu i686-pc-linux-gnu i686-pc-linux-gnu "elf_i386 elf32_x86_64" "/usr/local/lib /lib /usr/lib" no yes elf32_x86_64 "i686-pc-linux-gnu" +-gcc -DHAVE_CONFIG_H -I. -I../../master/ld -I. -I../../master/ld -I../bfd -I../../master/ld/../bfd -I../../master/ld/../include -g -O2 -DENABLE_PLUGINS -DLOCALEDIR="\"[...]/hurd/master.build.install/share/locale\"" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT eelf32_x86_64.o -MD -MP -MF .deps/eelf32_x86_64.Tpo -c -o eelf32_x86_64.o eelf32_x86_64.c +-mv -f .deps/eelf32_x86_64.Tpo .deps/eelf32_x86_64.Po +-/bin/bash ./libtool --tag=CC --mode=link gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -o ld-new ldgram.o ldlex-wrapper.o lexsup.o ldlang.o mri.o ldctor.o ldmain.o ldwrite.o ldexp.o ldemul.o ldver.o ldmisc.o ldfile.o ldcref.o plugin.o eelf_i386.o ei386linux.o eelf32_x86_64.o ../bfd/libbfd.la ../libiberty/libiberty.a -lz -ldl +-libtool: link: gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -o ld-new ldgram.o ldlex-wrapper.o lexsup.o ldlang.o mri.o ldctor.o ldmain.o ldwrite.o ldexp.o ldemul.o ldver.o ldmisc.o ldfile.o ldcref.o plugin.o eelf_i386.o ei386linux.o eelf32_x86_64.o ../bfd/.libs/libbfd.a ../libiberty/libiberty.a -lz -ldl +/bin/bash ./libtool --tag=CC --mode=link gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -o ld-new ldgram.o ldlex-wrapper.o lexsup.o ldlang.o mri.o ldctor.o ldmain.o ldwrite.o ldexp.o ldemul.o ldver.o ldmisc.o ldfile.o ldcref.o plugin.o eelf_i386.o ../bfd/libbfd.la ../libiberty/libiberty.a -lz -ldl +libtool: link: gcc -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -o ld-new ldgram.o ldlex-wrapper.o lexsup.o ldlang.o mri.o ldctor.o ldmain.o ldwrite.o ldexp.o ldemul.o ldver.o ldmisc.o ldfile.o ldcref.o plugin.o eelf_i386.o ../bfd/.libs/libbfd.a ../libiberty/libiberty.a -lz -ldl touch ld.1 diff --git a/open_issues/binutils/log_install-diff b/open_issues/binutils/log_install-diff index 83c8d7b6..00496f8b 100644 --- a/open_issues/binutils/log_install-diff +++ b/open_issues/binutils/log_install-diff @@ -1,5 +1,5 @@ ---- /dev/fd/63 2010-12-20 19:00:16.368493004 +0100 -+++ /dev/fd/62 2010-12-20 19:00:16.368493004 +0100 +--- /dev/fd/63 2011-02-10 18:56:20.086225001 +0100 ++++ /dev/fd/62 2011-02-10 18:56:20.086225001 +0100 @@ -68,7 +68,6 @@ libtool: install: /usr/bin/install -c .libs/libbfd.a [...]/hurd/master.build.install/lib/libbfd.a libtool: install: chmod 644 [...]/hurd/master.build.install/lib/libbfd.a diff --git a/open_issues/binutils/sum_hurd b/open_issues/binutils/sum_hurd index 96dd0cb2..15d225f9 100644 --- a/open_issues/binutils/sum_hurd +++ b/open_issues/binutils/sum_hurd @@ -1,4 +1,4 @@ -Test Run By tschwinge on Mon Dec 20 11:35:47 2010 +Test Run By tschwinge on Thu Feb 10 18:58:16 2011 Native configuration is i686-unknown-gnu0.3 === binutils tests === @@ -114,7 +114,7 @@ Running [...]/hurd/master/binutils/testsuite/binutils-all/x86-64/x86-64.exp ... # of expected passes 83 # of unsupported tests 2 -Test Run By tschwinge on Mon Dec 20 11:44:29 2010 +Test Run By tschwinge on Thu Feb 10 19:06:15 2011 Native configuration is i686-unknown-gnu0.3 === ld tests === @@ -640,6 +640,8 @@ PASS: ld-ifunc/ifunc-1-local-x86 PASS: ld-ifunc/ifunc-1-x86 PASS: ld-ifunc/ifunc-10-i386 PASS: ld-ifunc/ifunc-11-i386 +PASS: ld-ifunc/ifunc-12-i386 +PASS: ld-ifunc/ifunc-13-i386 PASS: ld-ifunc/ifunc-2-i386 PASS: ld-ifunc/ifunc-2-local-i386 PASS: ld-ifunc/ifunc-3a-x86 @@ -669,6 +671,8 @@ Running [...]/hurd/master/ld/testsuite/ld-m68k/m68k.exp ... Running [...]/hurd/master/ld/testsuite/ld-mep/mep.exp ... Running [...]/hurd/master/ld/testsuite/ld-mips-elf/mips-elf-flags.exp ... Running [...]/hurd/master/ld/testsuite/ld-mips-elf/mips-elf.exp ... +Running [...]/hurd/master/ld/testsuite/ld-misc/defsym.exp ... +PASS: ld-misc/defsym1 Running [...]/hurd/master/ld/testsuite/ld-mmix/mmix.exp ... Running [...]/hurd/master/ld/testsuite/ld-mn10300/mn10300.exp ... Running [...]/hurd/master/ld/testsuite/ld-pe/pe-compile.exp ... @@ -705,6 +709,7 @@ Running [...]/hurd/master/ld/testsuite/ld-scripts/alignof.exp ... PASS: ALIGNOF Running [...]/hurd/master/ld/testsuite/ld-scripts/assert.exp ... PASS: ASSERT +PASS: ld-scripts/assert2 Running [...]/hurd/master/ld/testsuite/ld-scripts/crossref.exp ... PASS: NOCROSSREFS 1 PASS: NOCROSSREFS 2 @@ -720,6 +725,8 @@ Running [...]/hurd/master/ld/testsuite/ld-scripts/defined.exp ... PASS: DEFINED (PRMS 5699) PASS: ld-scripts/defined2 PASS: ld-scripts/defined3 +PASS: ld-scripts/defined4 +PASS: ld-scripts/defined5 Running [...]/hurd/master/ld/testsuite/ld-scripts/dynamic-sections.exp ... PASS: dynamic sections Running [...]/hurd/master/ld/testsuite/ld-scripts/empty-address.exp ... @@ -735,6 +742,8 @@ Running [...]/hurd/master/ld/testsuite/ld-scripts/empty-orphan.exp ... PASS: ld-scripts/empty-orphan Running [...]/hurd/master/ld/testsuite/ld-scripts/expr.exp ... PASS: ld-scripts/expr1 +PASS: ld-scripts/expr2 +PASS: ld-scripts/sane1 Running [...]/hurd/master/ld/testsuite/ld-scripts/extern.exp ... PASS: EXTERN Running [...]/hurd/master/ld/testsuite/ld-scripts/include.exp ... @@ -873,13 +882,13 @@ Running [...]/hurd/master/ld/testsuite/ld-xtensa/xtensa.exp ... === ld Summary === -# of expected passes 608 +# of expected passes 616 # of unexpected successes 1 # of expected failures 17 # of untested testcases 6 -/media/data[...]/hurd/master.build/ld/ld-new 2.21.51.20101220 +/media/data[...]/hurd/master.build/ld/ld-new 2.21.51.20110210 -Test Run By tschwinge on Mon Dec 20 11:38:03 2010 +Test Run By tschwinge on Thu Feb 10 19:00:16 2011 Native configuration is i686-unknown-gnu0.3 === gas tests === @@ -961,8 +970,8 @@ PASS: CFI common 2 PASS: CFI common 3 PASS: CFI common 4 PASS: CFI common 5 -PASS: CFI common 7 PASS: CFI common 6 +PASS: CFI common 7 Running [...]/hurd/master/gas/testsuite/gas/cr16/cr16.exp ... Running [...]/hurd/master/gas/testsuite/gas/cr16/pic.exp ... Running [...]/hurd/master/gas/testsuite/gas/cris/cris.exp ... @@ -1001,6 +1010,7 @@ PASS: section flags PASS: DWARF2 1 PASS: DWARF2 2 PASS: DWARF2 3 +PASS: DWARF2 4 PASS: Check bad section flag Running [...]/hurd/master/gas/testsuite/gas/fr30/allinsn.exp ... Running [...]/hurd/master/gas/testsuite/gas/fr30/fr30.exp ... @@ -1094,8 +1104,10 @@ PASS: i386 -mtune=i686 nops 3 PASS: i386 nops 4 PASS: i386 nops -mtune=i386 4 PASS: i386 -mtune=i686 nops 4 +PASS: i386 -march=i686+nop nops 4a PASS: i386 nops 5 PASS: i386 -march=i686 nops 5 +PASS: i386 nops 6 PASS: i386 16-bit addressing in 32-bit mode. PASS: i386 32-bit addressing in 16-bit mode. PASS: i386 SSE4.1 @@ -1176,6 +1188,10 @@ PASS: i386 FMA scalar insns (Intel disassembly) PASS: i386 FMA4 PASS: i386 LWP PASS: i386 XOP +PASS: i386 BMI insns +PASS: i386 BMI insns (Intel disassembly) +PASS: i386 TBM +PASS: i386 TBM insns (Intel disassembly) PASS: i386 F16C PASS: i386 F16C (Intel disassembly) PASS: i386 FSGSBase @@ -1217,6 +1233,10 @@ PASS: i386 list-1 PASS: i386 list-2 PASS: i386 list-3 PASS: DWARF2 debugging information 1 +Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/cfi/ilp32.exp ... +Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/elf/ilp32.exp ... +Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/ilp32.exp ... +Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/lns/ilp32.exp ... Running [...]/hurd/master/gas/testsuite/gas/i860/i860.exp ... Running [...]/hurd/master/gas/testsuite/gas/ia64/ia64.exp ... Running [...]/hurd/master/gas/testsuite/gas/ieee-fp/x930509a.exp ... @@ -1317,6 +1337,6 @@ Running [...]/hurd/master/gas/testsuite/gas/z8k/z8k.exp ... === gas Summary === -# of expected passes 319 -../as-new 2.21.51.20101220 +# of expected passes 326 +../as-new 2.21.51.20110210 diff --git a/open_issues/binutils/sum_linux b/open_issues/binutils/sum_linux index c2dae925..49cf53fb 100644 --- a/open_issues/binutils/sum_linux +++ b/open_issues/binutils/sum_linux @@ -1,4 +1,4 @@ -Test Run By thomas on Mon Dec 20 11:34:53 2010 +Test Run By thomas on Thu Feb 10 18:57:42 2011 Native configuration is i686-pc-linux-gnu === binutils tests === @@ -114,7 +114,7 @@ Running [...]/hurd/master/binutils/testsuite/binutils-all/x86-64/x86-64.exp ... # of expected passes 83 # of unsupported tests 2 -Test Run By thomas on Mon Dec 20 11:35:19 2010 +Test Run By thomas on Thu Feb 10 18:58:10 2011 Native configuration is i686-pc-linux-gnu === ld tests === @@ -640,6 +640,8 @@ PASS: ld-ifunc/ifunc-1-local-x86 PASS: ld-ifunc/ifunc-1-x86 PASS: ld-ifunc/ifunc-10-i386 PASS: ld-ifunc/ifunc-11-i386 +PASS: ld-ifunc/ifunc-12-i386 +PASS: ld-ifunc/ifunc-13-i386 PASS: ld-ifunc/ifunc-2-i386 PASS: ld-ifunc/ifunc-2-local-i386 PASS: ld-ifunc/ifunc-3a-x86 @@ -669,6 +671,8 @@ Running [...]/hurd/master/ld/testsuite/ld-m68k/m68k.exp ... Running [...]/hurd/master/ld/testsuite/ld-mep/mep.exp ... Running [...]/hurd/master/ld/testsuite/ld-mips-elf/mips-elf-flags.exp ... Running [...]/hurd/master/ld/testsuite/ld-mips-elf/mips-elf.exp ... +Running [...]/hurd/master/ld/testsuite/ld-misc/defsym.exp ... +PASS: ld-misc/defsym1 Running [...]/hurd/master/ld/testsuite/ld-mmix/mmix.exp ... Running [...]/hurd/master/ld/testsuite/ld-mn10300/mn10300.exp ... Running [...]/hurd/master/ld/testsuite/ld-pe/pe-compile.exp ... @@ -705,6 +709,7 @@ Running [...]/hurd/master/ld/testsuite/ld-scripts/alignof.exp ... PASS: ALIGNOF Running [...]/hurd/master/ld/testsuite/ld-scripts/assert.exp ... PASS: ASSERT +PASS: ld-scripts/assert2 Running [...]/hurd/master/ld/testsuite/ld-scripts/crossref.exp ... PASS: NOCROSSREFS 1 PASS: NOCROSSREFS 2 @@ -720,6 +725,8 @@ Running [...]/hurd/master/ld/testsuite/ld-scripts/defined.exp ... PASS: DEFINED (PRMS 5699) PASS: ld-scripts/defined2 PASS: ld-scripts/defined3 +PASS: ld-scripts/defined4 +PASS: ld-scripts/defined5 Running [...]/hurd/master/ld/testsuite/ld-scripts/dynamic-sections.exp ... PASS: dynamic sections Running [...]/hurd/master/ld/testsuite/ld-scripts/empty-address.exp ... @@ -735,6 +742,8 @@ Running [...]/hurd/master/ld/testsuite/ld-scripts/empty-orphan.exp ... PASS: ld-scripts/empty-orphan Running [...]/hurd/master/ld/testsuite/ld-scripts/expr.exp ... PASS: ld-scripts/expr1 +PASS: ld-scripts/expr2 +PASS: ld-scripts/sane1 Running [...]/hurd/master/ld/testsuite/ld-scripts/extern.exp ... PASS: EXTERN Running [...]/hurd/master/ld/testsuite/ld-scripts/include.exp ... @@ -873,12 +882,12 @@ Running [...]/hurd/master/ld/testsuite/ld-xtensa/xtensa.exp ... === ld Summary === -# of expected passes 618 +# of expected passes 626 # of expected failures 8 # of untested testcases 6 -/media/data[...]/hurd/master.build/ld/ld-new 2.21.51.20101220 +/media/data[...]/hurd/master.build/ld/ld-new 2.21.51.20110210 -Test Run By thomas on Mon Dec 20 11:34:59 2010 +Test Run By thomas on Thu Feb 10 18:57:49 2011 Native configuration is i686-pc-linux-gnu === gas tests === @@ -960,8 +969,8 @@ PASS: CFI common 2 PASS: CFI common 3 PASS: CFI common 4 PASS: CFI common 5 -PASS: CFI common 7 PASS: CFI common 6 +PASS: CFI common 7 Running [...]/hurd/master/gas/testsuite/gas/cr16/cr16.exp ... Running [...]/hurd/master/gas/testsuite/gas/cr16/pic.exp ... Running [...]/hurd/master/gas/testsuite/gas/cris/cris.exp ... @@ -1000,6 +1009,7 @@ PASS: section flags PASS: DWARF2 1 PASS: DWARF2 2 PASS: DWARF2 3 +PASS: DWARF2 4 PASS: Check bad section flag Running [...]/hurd/master/gas/testsuite/gas/fr30/allinsn.exp ... Running [...]/hurd/master/gas/testsuite/gas/fr30/fr30.exp ... @@ -1093,8 +1103,10 @@ PASS: i386 -mtune=i686 nops 3 PASS: i386 nops 4 PASS: i386 nops -mtune=i386 4 PASS: i386 -mtune=i686 nops 4 +PASS: i386 -march=i686+nop nops 4a PASS: i386 nops 5 PASS: i386 -march=i686 nops 5 +PASS: i386 nops 6 PASS: i386 16-bit addressing in 32-bit mode. PASS: i386 32-bit addressing in 16-bit mode. PASS: i386 SSE4.1 @@ -1175,6 +1187,10 @@ PASS: i386 FMA scalar insns (Intel disassembly) PASS: i386 FMA4 PASS: i386 LWP PASS: i386 XOP +PASS: i386 BMI insns +PASS: i386 BMI insns (Intel disassembly) +PASS: i386 TBM +PASS: i386 TBM insns (Intel disassembly) PASS: i386 F16C PASS: i386 F16C (Intel disassembly) PASS: i386 FSGSBase @@ -1216,6 +1232,10 @@ PASS: i386 list-1 PASS: i386 list-2 PASS: i386 list-3 PASS: DWARF2 debugging information 1 +Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/cfi/ilp32.exp ... +Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/elf/ilp32.exp ... +Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/ilp32.exp ... +Running [...]/hurd/master/gas/testsuite/gas/i386/ilp32/lns/ilp32.exp ... Running [...]/hurd/master/gas/testsuite/gas/i860/i860.exp ... Running [...]/hurd/master/gas/testsuite/gas/ia64/ia64.exp ... Running [...]/hurd/master/gas/testsuite/gas/ieee-fp/x930509a.exp ... @@ -1316,6 +1336,6 @@ Running [...]/hurd/master/gas/testsuite/gas/z8k/z8k.exp ... === gas Summary === -# of expected passes 319 -../as-new 2.21.51.20101220 +# of expected passes 326 +../as-new 2.21.51.20110210 diff --git a/open_issues/gnumach_console_timestamp.mdwn b/open_issues/gnumach_console_timestamp.mdwn new file mode 100644 index 00000000..b36b47b9 --- /dev/null +++ b/open_issues/gnumach_console_timestamp.mdwn @@ -0,0 +1,27 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_gnumach]] + +IRC, freenode, #hurd, 2011-02-17 + + <azeem> task 39011c10 deallocating an invalid port 349, most probably a + bug. + <azeem> kernel: Page fault (14), code=6 + <azeem> Stopped at 0x28b9c7: orb %bh,0(%ecx,%edi,2) + <azeem> db> + [...] + <antrik> tschwinge: I doubt the deallocating warning is related to the + later fault + <tschwinge> antrik: YOu may be right. + <tschwinge> Perhaps it'd be a good idea to add some sort of timestamp to + Mach messages. + <tschwinge> Like in Linux' dmesg. + <tschwinge> Or just RDTSC (internal processor counter). diff --git a/open_issues/implementing_hurd_on_top_of_another_system.mdwn b/open_issues/implementing_hurd_on_top_of_another_system.mdwn index 23512aa9..95b71ebb 100644 --- a/open_issues/implementing_hurd_on_top_of_another_system.mdwn +++ b/open_issues/implementing_hurd_on_top_of_another_system.mdwn @@ -78,3 +78,40 @@ IRC, #hurd, 2010-12-28 <antrik> though I must say that I'm more and more convinced running the Hurd on top of a monolithic kernel would actually be a useful approach for the time being... + +--- + +IRC, #hurd, 2011-02-11 + + <neal> marcus and I were discussing how to add Mach to Linux + <neal> one could write a module to implement Mach IPC + <neal> and another to implement Mach VM + <neal> the big thing missing with Mach VM is the ability for a tracing + process to easily map or unmap an inferior process's memory + <antrik> neal: why would a tracing process need to map the inferior's + memory? + <neal> the simple answer is that is how it is done on Mach + <antrik> neal: is it? not sure we are talking about the same thing + here. GDB uses vm_read()/vm_write() to access the inferior's memory AFAIK + <neal> on linux? + <neal> I think it use /proc/pid/mem + <antrik> on Hurd + <neal> I'm talking about adding Mach to Linux + <neal> by adding some functionality to Linux + <neal> and then implementing a bunch in user space + <antrik> yeah, but I don't understand the point about mapping inferior's + memory :-( + <antrik> what would be in user space? + <neal> there are a number of different cut points + <neal> one could imagine just using Linux's device drivers, CPU scheduler, + memory management, etc. + <neal> another possibility would be something higher where Hurd processes + just use some Hurdish servers + <antrik> neal: yeah, these are all options I have been considering... too + bad I wasn't able to come to FOSDEM -- I'd love to have participated in + this discussion :-( + <antrik> neal: BTW, were you just discussing this as a hypothetical idea, + or something you are seriously considering? + <neal> I'm unlikely to work on it, sorry + <antrik> didn't really expect that :-) + <antrik> would be nice though if you could write up your conclusions... diff --git a/open_issues/performance/io_system.mdwn b/open_issues/performance/io_system.mdwn index 0d41d3c7..4af093ba 100644 --- a/open_issues/performance/io_system.mdwn +++ b/open_issues/performance/io_system.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2008, 2009, 2010 Free Software Foundation, +[[!meta copyright="Copyright © 2008, 2009, 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -6,8 +6,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!meta title="I/O System"]] @@ -20,7 +20,8 @@ slow hard disk access. The reason for this slowness is lack and/or bad implementation of common optimization techniques, like scheduling reads and writes to minimize head movement; effective block caching; effective reads/writes to partial blocks; -reading/writing multiple blocks at once; and read-ahead. The +[[reading/writing multiple blocks at once|clustered_page_faults]]; and +[[read-ahead]]. The [[ext2_filesystem_server|hurd/translator/ext2fs]] might also need some optimizations at a higher logical level. @@ -30,7 +31,7 @@ requires understanding the data flow through the various layers involved in disk access on the Hurd ([[filesystem|hurd/virtual_file_system]], [[pager|hurd/libpager]], driver), and general experience with optimizing complex systems. That said, the killing feature we are definitely -missing is the read-ahead, and even a very simple implementation would bring +missing is the [[read-ahead]], and even a very simple implementation would bring very big performance speedups. Here are some real testcases: diff --git a/open_issues/performance/io_system/clustered_page_faults.mdwn b/open_issues/performance/io_system/clustered_page_faults.mdwn new file mode 100644 index 00000000..3a187523 --- /dev/null +++ b/open_issues/performance/io_system/clustered_page_faults.mdwn @@ -0,0 +1,103 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_gnumach open_issue_hurd]] + +IRC, freenode, #hurd, 2011-02-16 + + <braunr> exceptfor the kernel, everything in an address space is + represented with a VM object + <braunr> those objects can represent anonymous memory (from malloc() or + because of a copy-on-write) + <braunr> or files + <braunr> on classic Unix systems, these are files + <braunr> on the Hurd, these are memory objects, backed by external pagers + (like ext2fs) + <braunr> so when you read a file + <braunr> the kernel maps it from ext2fs in your address space + <braunr> and when you access the memory, a fault occurs + <braunr> the kernel determines it's a region backed by ext2fs + <braunr> so it asks ext2fs to provide the data + <braunr> when the fault is resolved, your process goes on + <etenil> does the faul occur because Mach doesn't know how to access the + memory? + <braunr> it occurs because Mach intentionnaly didn't back the region with + physical memory + <braunr> the MMU is programmed not to know what is present in the memory + region + <braunr> or because it's read only + <braunr> (which is the case for COW faults) + <etenil> so that means this bit of memory is a buffer that ext2fs loads the + file into and then it is remapped to the application that asked for it + <braunr> more or less, yes + <braunr> ideally, it's directly written into the right pages + <braunr> there is no intermediate buffer + <etenil> I see + <etenil> and as you told me before, currently the page faults are handled + one at a time + <etenil> which wastes a lot of time + <braunr> a certain amount of time + <etenil> enough to bother the user :) + <etenil> I've seen pages have a fixed size + <braunr> yes + <braunr> use the PAGE_SIZE macro + <etenil> and when allocating memory, the size that's asked for is rounded + up to the page size + <etenil> so if I have this correctly, it means that a file ext2fs provides + could be split into a lot of pages + <braunr> yes + <braunr> once in memory, it is managed by the page cache + <braunr> so that pages more actively used are kept longer than others + <braunr> in order to minimize I/O + <etenil> ok + <braunr> so a better page cache code would also improve overall performance + <braunr> and more RAM would help a lot, since we are strongly limited by + the 768 MiB limit + <braunr> which reduces the page cache size a lot + <etenil> but the problem is that reading a whole file in means trigerring + many page faults just for one file + <braunr> if you want to stick to the page clustering thing, yes + <braunr> you want less page faults, so that there are less IPC between the + kernel and the pager + <etenil> so either I make pages bigger + <etenil> or I modify Mach so it can check up on a range of pages for faults + before actually processing + <braunr> you *don't* change the page size + <etenil> ah + <etenil> that's hardware isn't it? + <braunr> in Mach, yes + <etenil> ok + <braunr> and usually, you want the page size to be the CPU page size + <etenil> I see + <braunr> current CPU can support multiple page sizes, but it becomes quite + hard to correctly handle + <braunr> and bigger page sizes mean more fragmentation, so it only suits + machines with large amounts of RAM, which isn't the case for us + <etenil> ok + <etenil> so I'll try the second approach then + <braunr> that's what i'd recommand + <braunr> recommend* + <etenil> ok + +--- + +IRC, freenode, #hurd, 2011-02-16 + + <antrik> etenil: OSF Mach does have clustered paging BTW; so that's one + place to start looking... + <antrik> (KAM ported the OSF code to gnumach IIRC) + <antrik> there is also an existing patch for clustered paging in libpager, + which needs some adaptation + <antrik> the biggest part of the task is probably modifying the Hurd + servers to use the new interface + <antrik> but as I said, KAM's code should be available through google, and + can serve as a starting point + +<http://lists.gnu.org/archive/html/bug-hurd/2010-06/msg00023.html> diff --git a/open_issues/performance/io_system/read-ahead.mdwn b/open_issues/performance/io_system/read-ahead.mdwn new file mode 100644 index 00000000..3ee30b5d --- /dev/null +++ b/open_issues/performance/io_system/read-ahead.mdwn @@ -0,0 +1,299 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_gnumach open_issue_hurd]] + +IRC, #hurd, freenode, 2011-02-13: + + <etenil> youpi: Would libdiskfs/diskfs.h be in the right place to make + readahead functions? + <youpi> etenil: no, it'd rather be at the memory management layer, + i.e. mach, unfortunately + <youpi> because that's where you see the page faults + <etenil> youpi: Linux also provides a readahead() function for higher level + applications. I'll probably have to add the same thing in a place that's + higher level than mach + <youpi> well, that should just be hooked to the same common implementation + <etenil> the man page for readahead() also states that portable + applications should avoid it, but it could be benefic to have it for + portability + <youpi> it's not in posix indeed + +--- + +IRC, #hurd, freenode, 2011-02-14: + + <etenil> youpi: I've investigated prefetching (readahead) techniques. One + called DiskSeen seems really efficient. I can't tell yet if it's patented + etc. but I'll keep you informed + <youpi> don't bother with complicated techniques, even the most simple ones + will be plenty :) + <etenil> it's not complicated really + <youpi> the matter is more about how to plug it into mach + <etenil> ok + <youpi> then don't bother with potential pattents + <antrik> etenil: please take a look at the work KAM did for last year's + GSoC + <youpi> just use a trivial technique :) + <etenil> ok, i'll just go the easy way then + + <braunr> antrik: what was etenil referring to when talking about + prefetching ? + <braunr> oh, madvise() stuff + <braunr> i could help him with that + +--- + +[[Etenil]] is now working in this area. + +--- + +IRC, freenode, #hurd, 2011-02-15 + + <etenil> oh, I'm looking into prefetching/readahead to improve I/O + performance + <braunr> etenil: ok + <braunr> etenil: that's actually a VM improvement, like samuel told you + <etenil> yes + <braunr> a true I/O improvement would be I/O scheduling + <braunr> and how to implement it in a hurdish way + <braunr> (or if it makes sense to have it in the kernel) + <etenil> that's what I've been wondering too lately + <braunr> concerning the VM, you should look at madvise() + <etenil> my understanding is that Mach considers devices without really + knowing what they are + <braunr> that's roughly the interface used both at the syscall() and the + kernel levels in BSD, which made it in many other unix systems + <etenil> whereas I/O optimisations are often hard disk drives specific + <braunr> that's true for almost any kernel + <braunr> the device knowledge is at the driver level + <etenil> yes + <braunr> (here, I separate kernels from their drivers ofc) + <etenil> but Mach also contains some drivers, so I'm going through the code + to find the apropriate place for these improvements + <braunr> you shouldn't tough the drivers at all + <braunr> touch + <etenil> true, but I need to understand how it works before fiddling around + <braunr> hm + <braunr> not at all + <braunr> the VM improvement is about pagein clustering + <braunr> you don't need to know how pages are fetched + <braunr> well, not at the device level + <braunr> you need to know about the protocol between the kernel and + external pagers + <etenil> ok + <braunr> you could also implement pageout clustering + <etenil> if I understand you well, you say that what I'd need to do is a + queuing system for the paging in the VM? + <braunr> no + <braunr> i'm saying that, when a page fault occurs, the kernel should + (depending on what was configured through madvise()) transfer pages in + multiple blocks rather than one at a time + <braunr> communication with external pagers is already async, made through + regular ports + <braunr> which already implement message queuing + <braunr> you would just need to make the mapped regions larger + <braunr> and maybe change the interface so that this size is passed + <etenil> mmh + <braunr> (also don't forget that page clustering can include pages *before* + the page which caused the fault, so you may have to pass the start of + that region too) + <etenil> I'm not sure I understand the page fault thing + <etenil> is it like a segmentation error? + <etenil> I can't find a clear definition in Mach's manual + <braunr> ah + <braunr> it's a fundamental operating system concept + <braunr> http://en.wikipedia.org/wiki/Page_fault + <etenil> ah ok + <etenil> I understand now + <etenil> so what's currently happening is that when a page fault occurs, + Mach is transfering pages one at a time and wastes time + <braunr> sometimes, transferring just one page is what you want + <braunr> it depends on the application, which is why there is madvise() + <braunr> our rootfs, on the other hand, would benefit much from such an + improvement + <braunr> in UVM, this optimization is account for around 10% global + performance improvement + <braunr> accounted* + <etenil> not bad + <braunr> well, with an improved page cache, I'm sure I/O would matter less + on systems with more RAM + <braunr> (and another improvement would make mach support more RAM in the + first place !) + <braunr> an I/O scheduler outside the kernel would be a very good project + IMO + <braunr> in e.g. libstore/storeio + <etenil> yes + <braunr> but as i stated in my thesis, a resource scheduler should be as + close to its resource as it can + <braunr> and since mach can host several operating systems, I/O schedulers + should reside near device drivers + <braunr> and since current drivers are in the kernel, it makes sens to have + it in the kernel too + <braunr> so there must be some discussion about this + <etenil> doesn't this mean that we'll have to get some optimizations in + Mach and have the same outside of Mach for translators that access the + hardware directly? + <braunr> etenil: why ? + <etenil> well as you said Mach contains some drivers, but in principle, it + shouldn't, translators should do disk access etc, yes? + <braunr> etenil: ok + <braunr> etenil: so ? + <etenil> well, let's say if one were to introduce SATA support in Hurd, + nothing would stop him/her to do so with a translator rather than in Mach + <braunr> you should avoid the term translator here + <braunr> it's really hurd specific + <braunr> let's just say a user space task would be responsible for that + job, maybe multiple instances of it, yes + <etenil> ok, so in this case, let's say we have some I/O optimization + techniques like readahead and I/O scheduling within Mach, would these + also apply to the user-space task, or would they need to be + reimplemented? + <braunr> if you have user space drivers, there is no point having I/O + scheduling in the kernel + <etenil> but we also have drivers within the kernel + <braunr> what you call readahead, and I call pagein/out clustering, is + really tied to the VM, so it must be in Mach in any case + <braunr> well + <braunr> you either have one or the other + <braunr> currently we have them in the kernel + <braunr> if we switch to DDE, we should have all of them outside + <braunr> that's why such things must be discussed + <etenil> ok so if I follow you, then future I/O device drivers will need to + be implemented for Mach + <braunr> currently, yes + <braunr> but preferrably, someone should continue the work that has been + done on DDe so that drivers are outside the kernel + <etenil> so for the time being, I will try and improve I/O in Mach, and if + drivers ever get out, then some of the I/O optimizations will need to be + moved out of Mach + <braunr> let me remind you one of the things i said + <braunr> i said I/O scheduling should be close to their resource, because + we can host several operating systems + <braunr> now, the Hurd is the only system running on top of Mach + <braunr> so we could just have I/O scheduling outside too + <braunr> then you should consider neighbor hurds + <braunr> which can use different partitions, but on the same device + <braunr> currently, partitions are managed in the kernel, so file systems + (and storeio) can't make good scheduling decisions if it remains that way + <braunr> but that can change too + <braunr> a single storeio representing a whole disk could be shared by + several hurd instances, just as if it were a high level driver + <braunr> then you could implement I/O scheduling in storeio, which would be + an improvement for the current implementation, and reusable for future + work + <etenil> yes, that was my first instinct + <braunr> and you would be mostly free of the kernel internals that make it + a nightmare + <etenil> but youpi said that it would be better to modify Mach instead + <braunr> he mentioned the page clustering thing + <braunr> not I/O scheduling + <braunr> theseare really two different things + <etenil> ok + <braunr> you *can't* implement page clustering outside Mach because Mach + implements virtual memory + <braunr> both policies and mechanisms + <etenil> well, I'd rather think of one thing at a time if that's alright + <etenil> so what I'm busy with right now is setting up clustered page-in + <etenil> which need to be done within Mach + <braunr> keep clustered page-outs in mind too + <braunr> although there are more constraints on those + <etenil> yes + <etenil> I've looked up madvise(). There's a lot of documentation about it + in Linux but I couldn't find references to it in Mach (nor Hurd), does it + exist? + <braunr> well, if it did, you wouldn't be caring about clustered page + transfers, would you ? + <braunr> be careful about linux specific stuff + <etenil> I suppose not + <braunr> you should implement at least posix options, and if there are + more, consider the bsd variants + <braunr> (the Mach VM is the ancestor of all modern BSD VMs) + <etenil> madvise() seems to be posix + <braunr> there are system specific extensions + <braunr> be careful + <braunr> CONFORMING TO POSIX.1b. POSIX.1-2001 describes posix_madvise(3) + with constants POSIX_MADV_NORMAL, etc., with a behav‐ ior close to that + described here. There is a similar posix_fadvise(2) for file access. + <braunr> MADV_REMOVE, MADV_DONTFORK, MADV_DOFORK, MADV_HWPOISON, + MADV_MERGEABLE, and MADV_UNMERGEABLE are Linux- specific. + <etenil> I was about to post these + <etenil> ok, so basically madvise() allows tasks etc. to specify a usage + type for a chunk of memory, then I could apply the relevant I/O + optimization based on this + <braunr> that's it + <etenil> cool, then I don't need to worry about knowing what the I/O is + operating on, I just need to apply the optimizations as advised + <etenil> that's convenient + <etenil> ok I'll start working on this tonight + <etenil> making a basic readahead shouldn't be too hard + <braunr> readahead is a misleading name + <etenil> is pagein better? + <braunr> applies to too many things, doesn't include the case where + previous elements could be prefetched + <braunr> clustered page transfers is what i would use + <braunr> page prefetching maybe + <etenil> ok + <braunr> you should stick to something that's already used in the + literature since you're not inventing something new + <etenil> yes I've read a paper about prefetching + <etenil> ok + <etenil> thanks for your help braunr + <braunr> sure + <braunr> you're welcome + <antrik> braunr: madvise() is really the least important part of the + picture... + <antrik> very few applications actually use it. but pretty much all + applications will profit from clustered paging + <antrik> I would consider madvise() an optional goody, not an integral part + of the implementation + <antrik> etenil: you can find some stuff about KAM's work on + http://www.gnu.org/software/hurd/user/kam.html + <antrik> not much specific though + <etenil> thanks + <antrik> I don't remember exactly, but I guess there is also some + information on the mailing list. check the archives for last summer + <antrik> look for Karim Allah Ahmed + <etenil> antrik: I disagree, madvise gives me a good starting point, even + if eventually the optimisations should run even without it + <antrik> the code he wrote should be available from Google's summer of code + page somewhere... + <braunr> antrik: right, i was mentioning madvise() because the kernel (VM) + interface is pretty similar to the syscall + <braunr> but even a default policy would be nice + <antrik> etenil: I fear that many bits were discussed only on IRC... so + you'd better look through the IRC logs from last April onwards... + <etenil> ok + + <etenil> at the beginning I thought I could put that into libstore + <etenil> which would have been fine + + <antrik> BTW, I remembered now that KAM's GSoC application should have a + pretty good description of the necessary changes... unfortunately, these + are not publicly visible IIRC :-( + +--- + +IRC, freenode, #hurd, 2011-02-16 + + <etenil> braunr: I've looked in the kernel to see where prefetching would + fit best. We talked of the VM yesterday, but I'm not sure about it. It + seems to me that the device part of the kernel makes more sense since + it's logically what manages devices, am I wrong? + <braunr> etenil: you are + <braunr> etenil: well + <braunr> etenil: drivers should already support clustered sector + read/writes + <etenil> ah + <braunr> but yes, there must be support in the drivers too + <braunr> what would really benefit the Hurd mostly concerns page faults, so + the right place is the VM subsystem + +[[clustered_page_faults]] |