From a5b95baab6b4512ceaef8749a29ed7b3685b4125 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Wed, 15 Jun 2011 22:54:09 +0200 Subject: Some more bits from IRC and elsewhere. --- hurd/running/qemu/discussion.mdwn | 92 +++++++++++++++++++++++++++++++++ microkernel/mach/gnumach/ports/xen.mdwn | 24 +++++++-- toolchain.mdwn | 8 ++- toolchain/elfosabi_hurd.mdwn | 83 +++++++++++++++++++++++++++++ user/jkoenig/java.mdwn | 5 +- 5 files changed, 204 insertions(+), 8 deletions(-) create mode 100644 toolchain/elfosabi_hurd.mdwn diff --git a/hurd/running/qemu/discussion.mdwn b/hurd/running/qemu/discussion.mdwn index fd0df4c5..1ce14b01 100644 --- a/hurd/running/qemu/discussion.mdwn +++ b/hurd/running/qemu/discussion.mdwn @@ -50,3 +50,95 @@ The problem is actually that the linux block cache doesn't make any consistency between /dev/hda and /dev/hda6, so if you give /dev/hda to qemu, qemu writings won't be consistent with mounting /dev/hda6 in linux. You can give /dev/hda6 directly to qemu and it will be fine. + + +# Host-side Writeback Caching + +IRC, freenode, #hurd, 2011-06-07 + + hm, i guess i should have used cache=writeback with kvm before + starting the debian installer :/ + ah yes, much better + this shows how poor the state of our I/O drivers and subsystem is + :/ + indeed... still no clustered pageout :-( + and no I/O scheduler either + although an I/O scheduler has limited value without clustered + pageouts + since one of its goals is to pack related I/O requests together eh + i wonder if the wiki mentions using cache=writeback to speed up + qemu performances + it would help those unable to use kvm a lot + and even those running kvm too + kvm -m $RAM \ -monitor stdio \ -drive + cache=writeback,index=0,media=disk,file=hd0.img \ + etc.. + the idea is that qemu doesn't open its disk file synchronously + changes are queued in the host page cache before being flushed to + the disk image + but if you brutally close your qemu instance, you're likely to + loose file system consistency + ext2fs will think it has committed its metadata to the disk, but + the disk image won't be updated synchronously + on my machine (which is quite fast), my kvm has installed debian + like 10 times faster than without the option + braunr: I don't think killing qemu should hurt in this + case... probably only matters when the host machine dies + antrik: ah yes, right + it really makes everything faster, even downloading, since I/O + requests aren't interleaved between networking RPCs + regarding I/O sheduler... this discussion came up before, but I + don't remember the outcome -- doesn't the glued Linux driver actually + come with one? + i don't remember either + braunr: err... I don't think interleaving has anything to do with + it... I guess it's simply the fact that downloading writes the result to + disk, which suffers from lacking clustered pageout like everything else + (my internet connection is too slow though to notice :-) ) + well, if there is no I/O during downloading, downloading is faster + :) + +IRC, freenode, #hurd, 2011-06-08 + + youpi: does xen provide disk caching options ? + through a blktap, probably + ok + +([[microkernel/mach/gnumach/ports/Xen]], *Host-side Writeback Caching*.) + + we should find the pages mentioning qemu on the wiki and add the + options to enable disk image caching + it really makes the hurd run a lot faster + as a workaround for emulators until I/O is reworked, ofc + +IRC, freenode, #hurd, 2011-06-09 + + braunr recommends to use writeback caching with kvm. Is this + really recommended with the frequent crashes I experience? + provided that you terminate your kvm normaly (i.e. quitting it, not + killing it), there should be no difference + I think the host's stability is what matters + the data presumably sits in linux's cache even if qemu dies + violently + But the freezes I see force me to kill kvm :-( + maybe kvm doesn't even do caching indeed, I don't know + gnu_srs: you can quit even when frozen + use the console + (the kvm console) + gnu_srs, "Writeback caching will report data writes as completed + as soon as the data is present in the host page cache. This is safe as + long as you trust your host. If your host crashes or loses power, then + the guest may experience data corruption." (from the qemu manpage) + +IRC, freenode, #hurd, 2011-06-11 + + braunr: If you are online. For me setting the parameters -drive + cache=writeback,index=0,media=disk,file=hd0.img does not show any speed + improvement at all compared to the default. + gnu_srs: what's your complete qemu command line ? + kvm -m 1024 -net nic,model=rtl8139 -net + user,hostfwd=tcp::5556-:22 -drive + cache=writeback,index=0,media=disk,file=hd0.img -cdrom netinst.iso + what qemu version ? + qemu-kvm 0.14.1+dfsg-1: Sorry, I cannot be online until + tomorrow again. diff --git a/microkernel/mach/gnumach/ports/xen.mdwn b/microkernel/mach/gnumach/ports/xen.mdwn index bf26410a..af431c92 100644 --- a/microkernel/mach/gnumach/ports/xen.mdwn +++ b/microkernel/mach/gnumach/ports/xen.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2007, 2008, 2009 Free Software Foundation, +[[!meta copyright="Copyright © 2007, 2008, 2009, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -6,11 +6,12 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!toc]] + # Xen dom0, hypervisor /!\ Now that GNU Mach handles PAE you can use a PAE-enabled hypervisor. @@ -30,6 +31,7 @@ If you have a free partition, you can fdisk to type 0x83, create a filesystem us Replace /dev/sda4 with your partition. Install and use crosshurd to setup a GNU/Hurd system on this partition. + # /etc/xen/hurd configuration Here is a sample /etc/xen/hurd configuration @@ -49,6 +51,7 @@ Suggestions about [[networking_configuration]] are available. If you need stable MAC addresses, use a syntax like `vif = [ 'mac=00:16:3e:XX:XX:XX, bridge=br0' ]`. + # Running Hurd with Xen To run Hurd with Xen, use: @@ -63,6 +66,7 @@ and gnumach should get started. Proceed with native-install. - If `xm` complains about networking (`vif could not be connected`), it's Xen scripts' fault, see Xen documentation for how to configure the network. The simplest way is network-bridge with fixed IPs (note that you need the bridge-utils package for this). You can also just disable networking by commenting the vif line in the config. - If `xm` complains `Error: (2, 'Invalid kernel', 'xc_dom_compat_check: guest type xen-3.0-x86_32 not supported by xen kernel, sorry\n')`, you most probably have a PAE-enabled hypervisor and a non-PAE gnumach. Either install and boot non-PAE hypervisor and kernel, or rebuilt gnumach in PAE mode. + # Building from sources If you want to generate these images, first get the `gnumach-1-branch-Xen-branch` branch from gnumach CVS. @@ -75,7 +79,6 @@ Then use The current `hurd-modules` was built from the debian packages `hurd 20070606-2` and `libc0.3 2.6.1-1`. /!\ This means that when using this image, your GNU/Hurd system also needs to be a glibc version 2.6 or later-based one! ---- # Miscellaneous @@ -83,7 +86,6 @@ The current `hurd-modules` was built from the debian packages `hurd 20070606-2` [[!GNU_Savannah_task 5468]], [[!GNU_Savannah_task 6584]]. ---- # `pv-grub` @@ -91,3 +93,15 @@ From Xen 4.0 on you'll be able to run the GNU Hurd directly using `pv-grub`, without the need to [prepare a special bootstrap image](http://youpibouh.thefreecat.org/hurd-xen/build_hurd-modules) (like an initrd). + + +# Host-side Writeback Caching + +Optimization possible as it is with [[QEMU|hurd/running/qemu/discussion]], +*Host-side Writeback Caching*? + +IRC, freenode, #hurd, 2011-06-08 + + youpi: does xen provide disk caching options ? + through a blktap, probably + ok diff --git a/toolchain.mdwn b/toolchain.mdwn index b08dba95..0059317b 100644 --- a/toolchain.mdwn +++ b/toolchain.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2007, 2008, 2010 Free Software Foundation, +[[!meta copyright="Copyright © 2007, 2008, 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -24,4 +24,10 @@ for example, how things are done differently for GNU/Hurd than they are done for GNU/Linux. +--- + + * [[ELFOSABI_HURD]] + +--- + * [[cross-gnu]] diff --git a/toolchain/elfosabi_hurd.mdwn b/toolchain/elfosabi_hurd.mdwn new file mode 100644 index 00000000..4d60f761 --- /dev/null +++ b/toolchain/elfosabi_hurd.mdwn @@ -0,0 +1,83 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="ELFOSABI_HURD"]] + +[[!tag open_issue_binutils open_issue_glibc]] + + +# [[!debbug 630180]] + + +# [[open_issues/binutils]] commit 51b2f560ad035dffad3371093f8e5c80608d196c + +Usage of `ELFOSABI_LINUX`/`STB_GNU_UNIQUE`. Has also been wrong before already +with respect to `STT_GNU_IFUNC`? + + +# IRC + +IRC, freenode, #debian-hurd, 2011-06-11 + + youpi: not that there would be any hope in that, but id you try + asking doko about the gcc miscompiling (wrong elf format) issue? + I didn't + I'm still investigating + maybe it's a binutils change actually + youpi: hm, are you sure it could be binutils? after all, even + some .o files are produced with format gnu/linux, so there's no binutils + involved up to that point of thecompilation yet? + as is + "as", I mean + i see + since it's so unclear, I really prefer to investigate before + bothering doko + youpi: maybe i could be wrong, + in binutils, bfd/elf.c, around lines 9580 + the faulty thing seems to be gnu_unique_object in the source .s + file produced by g++ + that's what that comment (which changed wrt binutils from eg + march) says + seems to concur with my comment above :) + http://paste.debian.net/119542/ ‘¡û extract of diff + ok, that really seems the culprit + starting reportbug + who's the fault then? + binutils + it shouldn't hardcode LINUX + g++ emitting those symbols, or binutil considering them "linux"? + it's a GNU thing, not a Linux thing + ah ok + it's the same dynamic linker actually + youpi: http://sourceware.org/bugzilla/show_bug.cgi?id=10549 + see the reporter :) + heh + youpi: see also gas/config/obj-elf.c:1725 + (another change related to that bug, it seems) + +IRC, freenode, #hurd, 2011-06-15 + + also I still get an "ELF file OS ABI invalid" error with binutils + 2.21.52.20110606-2+hurd.1, is that expected? + tschwinge: oops, the OS ABI invalid is actually due to the file + being marked GNU/Hurd + I guess the linker is simply not aware that it should accept + GNU/Hurd + youpi: So we got to work on glibc'S ld.so to teach it aboput + the Hurd OS ABI? (Or probably simply make that equivalent to the Linux + one?) + probably simply an equivalent + ELFOSABI_HURD is missing from elf/elf.h, for a start... + linux' glibc has tests in lsdodefs.h + the VALID_ELF_OSABI macro + it's thus apparently a matter of providing an ldsodefs.h file with + VALID_ELF_HEADER, VALID_ELF_OSABI and VALID_ELF_ABIVERSION definitions + (and include_next the generic one) + I've prepared a patch for ldsodefs.h, I'll test it diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn index 6aeef9fe..8e1fb097 100644 --- a/user/jkoenig/java.mdwn +++ b/user/jkoenig/java.mdwn @@ -140,7 +140,7 @@ by building on [[pochu]]'s `file_exec_file_name()` I have succeeded in building a Hotspot-enabled `libjvm.so`, although the current toolchain issues -(*Hurd OS ABI*) +([[toolchain/ELFOSABI_HURD]]) have so far prevented me from testing it. @@ -151,7 +151,8 @@ have so far prevented me from testing it. * Assuming it is, continue with getting `$ORIGIN` working. - * [!] [[Samuel|samuelthibault]]/[[tschwinge]]/[[jkoenig]]: *Hurd OS ABI*. + * [!] [[Samuel|samuelthibault]]/[[tschwinge]]/[[jkoenig]]: + [[toolchain/ELFOSABI_HURD]]. * They seem to have a rather heavy-weight process for such projects: confer , -- cgit v1.2.3