From f23e31a673c4bc8b6c9260f640845b16860fc15f Mon Sep 17 00:00:00 2001 From: Jeremie Koenig Date: Tue, 24 May 2011 17:46:58 +0200 Subject: user/jkoenig: reorganize --- user/jkoenig/d-i.mdwn | 358 ++++++++++++++ user/jkoenig/gsoc2011_proposal.mdwn | 628 ------------------------- user/jkoenig/gsoc2011_proposal/discussion.mdwn | 180 ------- user/jkoenig/java.mdwn | 628 +++++++++++++++++++++++++ user/jkoenig/java/discussion.mdwn | 180 +++++++ 5 files changed, 1166 insertions(+), 808 deletions(-) create mode 100644 user/jkoenig/d-i.mdwn delete mode 100644 user/jkoenig/gsoc2011_proposal.mdwn delete mode 100644 user/jkoenig/gsoc2011_proposal/discussion.mdwn create mode 100644 user/jkoenig/java.mdwn create mode 100644 user/jkoenig/java/discussion.mdwn (limited to 'user/jkoenig') diff --git a/user/jkoenig/d-i.mdwn b/user/jkoenig/d-i.mdwn new file mode 100644 index 00000000..0b9f9f7d --- /dev/null +++ b/user/jkoenig/d-i.mdwn @@ -0,0 +1,358 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +## Hurd Debian-Installer + +My [proposal](http://wiki.debian.org/SummerOfCode2010/HurdDebianInstaller/JeremieKoenig) +to work on porting d-i on Hurd +as a [Google Summer of Code](http://code.google.com/soc/) student +has been accepted by the Debian project. + +I will be keeping track of my progress on this page. + +### Links + + * [Modified packages](http://jk.fr.eu.org/debian/unstable) + * [Latest images](http://jk.fr.eu.org/debian/hurd-installer) + * [Debian bugs](http://bugs.debian.org/cgi-bin/pkgreport.cgi?users=jk@jk.fr.eu.org&tag=gsoc2010) + * [BusyBox port](http://lists.debian.org/debian-bsd/2010/05/msg00048.html) + * [GNU Mach initrd](http://lists.gnu.org/archive/html/bug-hurd/2010-06/msg00047.html) + +### Roadmap + +* **mach**: initrd support + * (./) preliminary patch posted and self-built (2010-06-12) + * adjustments will be needed (postponed) + * consider the alternatives discussed on bug-hurd (postponed) + +* **glibc**: fix `mkdir("/")` which returned `EINVAL` + * (./) eglibc 2.11.2-1 includes a quick fix by youpi (2010-06-15) + * (./) more complete patch posted to bug-hurd, + since other calls return incorrect errors under some circumstances + (2010-06-16) + * more work on it will be needed to make it fix the whole thing + (postponed) + +* (./) **partman** (2010-06-23) + * (./) add hurd-i386 to + `partman-partitioning/lib/disk-label.sh` + (2010-06-16, commited by youpi on 2010-06-23) + * (./) short-circuit + `partman-basicfilesystems/init.d/kernelmodules_basicfilesystems` + (2010-06-16) + * (./) partman-auto recipes: + make the default filesystem os-dependent + when it has not been preseeded (ie. the *seen* flag is clear) + * (./) force 4k blocks and 128 bytes inodes + * (./) submit patches to bugs.debian.org + ([[!debbug 586870]] and [[!debbug 586871]]) + * (./) rebuild with responsible version numbers and upload to my repository + +* (./) **libparted** (2010-06-23) + * (./) fix device paths ([[!debbug 586696]]) + * (./) fix crash on exit for part:* stores ([[!debbug 586682]]) + +* **hurd-udeb** (2010-06-23) + * (./) rebuild with the hack suggested by youpi for qemu network configuration + * (./) fix mount to accept `-o defaults` + * cleanup, ask youpi to commit + +* reloading the partition table (2010-06-25) + * User-space part stores + * (./) hurd-udeb now uses `part:N:device:X` for partition devices + (2010-06-23) + * (./) it also provides /lib/partman/commit.d/??hurd\_reloadpart, + which basically does `settrans -ag /dev/[hs]d*`. + (2010-06-24) + * Kernel-based partition devices + * (./) Mach's drivers from Linux support reloading partitions. + With help from youpi this has been made available through a + device\_set\_status() call. + * (./) make libparted use it + * Reminder: + I should file a bug against libparted with the patch sometime. + +* (./) The `/servers/exec` issue (2010-06-26) + * Due to /servers being inexistant, + the bootstrap ext2fs could not register the initial exec server, + meaning that non-bootstrap filesystems used a different one + (started from the passive translator), + which for some reason died on shell scripts, making them stall. + * Adding the `/servers` directory to hurd-udeb fixed it, + as well as the /hurd/proc issue + (failed to be run by init the first time around). + * Reminder: report the non-bootstrap exec servers failure on scripts. + +* (./) **base-installer**: (2010-06-26) + * Work around non-existant /proc/mounts. + * Firmlink /servers into /target after debootstrap + to make the network available. + +* (./) **grub-installer** + * (./) add hurd support (2010-06-27) + * /!\ grub-legacy still needs to be tested + * submit changes as a Debian bug + +**Milestone (2010-06-28): +installer kindof works, with documented manual intervention required** + +* (./) Sort out the situation with dev node creation (2010-07-07): + * Devices and servers used to be set up by debootstrap; + the hurd package would add some missing nodes. + * New strategy implemented in hurd and debootstrap: + * debootstrap uses active firmlinks into the host system + for the target system's /dev and /servers. + * the hurd package now include a `setup-translators` script, + which is used to register the passive translators by the installer's + `/libexec/runsystem` and hurd's postinst script. + +* **busybox**: submit upstream and to [[!debbug 323670]] + (waiting for upstream to review) + * (./) I have mentioned my work on the upstream mailing list, + * (./) merge the recent changes from upstream, + notably to the build system. + (2010-06-23) + * (./) ask upstream for review and merge + (2010-06-25) + * (./) sent as patches as requested + (2010-07-08) + * (./) backport any additional changes onto the debian branch + * (./) hijack [[!debbug 323670]] and submit my patches + +* **aptitude**: + * Currently broken on hurd-i386: + [[!debpkg gtest]] fails to build because of a segfault in one of the test + cases, [[!debpkg google-mock]] and hence [[!debpkg aptitude]] are missing + it as a build-dep. + The older package is not installable anymore because it's linked against + an older version of libept, which has been removed. + * (./) I bypassed the tests and uploaded the 3 packages to my repository + (2010-07-08) + * The segfault will have to be sorted out. (postponed) + +* (./) "Fix" the swap situation. (2010-07-08) + * The device\_close() libstore patch + had the unfortunate effect of making swapon fail, + since the device it activates has to be kept open. + * add options for MAKEDEV and setup-devices + to use the libparted stores + * disable youpi's patch + * make partman-basicfilesystems re-create the device + as a kernel partition, which is needed for swapon + +* (./) netcfg-static: port to hurd (2010-07-09) + * There was some amount of hurd support already + (namely, activating the interface by replacing the socket translator) + * However, this code started an active translator with + di\_exec\_shell\_log("settrans -a ...), + which stalled as a consequence of it capturing libdi's pipe + as its standard output. + * Network devices must be probed by trying to open Mach devices + with predetermined names (currently eth%d, wl%d), + because getifaddrs() does not seem to work on Hurd. + * /!\ netcfg, and configuring the installed system, postponed. + +* **procps** 3.2.7-11 (current hurd-i386 version) has [[!debbug 546888]] + * (./) Submit [[!debbug 588677]] and upload the result to my repository. + (2010-07-11) + +* (./) Set up a Debian mirror with modified packages for installation + * the [mirror](http://jk.fr.eu.org/debian/hurd-install/mirror) + is now up and running (2010-07-06) + * hacked the image build script to include its public key in + debian-archive-keyring at image build time (2010-07-08) + * Apparently debootstrap does not handle multiple versions very well. + Fix by using dpkg-scan{package,sources} rather than apt-ftparchive + to create index files. + (2010-07-10) + * Use the override files from ftp.debian.org, + to avoid debootstrap grabbing inappropriate packages. + * Changed them to make [[!debpkg ifupdown]], + [[!debpkg dhcp3-client]] and [[!debpkg dhcp3-client]] priority extra, + because they're uninstallable at the moment. + (2010-07-12) + +* (./) Put together a "jk-archive-keyring" package, + so that the mirror is authenticated in the target system as well. + (2010-07-12) + +* (./) Fix grub for user-space partitions (2010-07-16) + * grub-probe detects the whole device rather than the partition + as the device behind /boot/grub. + Consequently, grub-install fails. + * One approach would be to replace /dev/hd* by kernel devices + for file systems as well as for swap partitions. + > {X} this makes the installer crash, + > possibly due to cache coherency issue between hdX and hdXsY. + + * (./) GRUB2 kern/emu/getroot.c + [patched](http://lists.gnu.org/archive/html/bug-hurd/2010-07/msg00059.html) + to support part stores. + +* (./) Fix finish-install to skip `finish-install.d/90console` on Hurd + (2010-07-17) + +* (./) Avoid starting unnecessary /dev translators in a burst (2010-07-20) + * Use debootstrap use the extracted /usr/lib/hurd/setup-translators + to create device and server nodes in /target, + then firmlink the whole /target/dev and /target/servers + to the outer system. + * Make hurd.postinst not touch them on initial install. + +* (./) Fix mach-defpager for file and part stores on larger devices + * Use DEVICE\_GET\_RECORDS instead of DEVICE\_GET\_SIZE, which overflows an int + (2010-07-22) + +**Milestone (2010-07-22): +installer works but it's still somewhat ugly and broken** + +* (./) Ship the UTF-8 font for the hurd console + (2010-07-22) + * Upload a version of bogl with youpi's patch for Hurd. + (see [[!debbug 589987]]) + * Fix the hurd console for fonts with 16 pixels wide glyphs + (ie. handle the 8-wide glyph in there correclty) + * Support double-width glyphs (2010-07-24) + * (./) However the reduced font can't be loaded yet, + so make installer/build/Makefile + ship the whole `/usr/src/unifont.bgf` + as `/usr/share/hurd/vga-system.bgf`. + +* (./) Make the installer used the extended capabilities of the Hurd console + (2010-07-23) + * Set an UTF-8 locale in `/lib/debian-installer.d/S41term-hurd`. + * localechooser: set the language display level to 3 + when using the hurd console. + +* (./) **busybox**: cross-platform package uploaded to experimental + (2010-08-03?) + * Aurelien Jarno updated the packaging to busybox 1.17.1, + fixed a whole lot of bugs, + and uploaded a new package with both our changes; + * most patches adopted upstream, and included in the new package; + * (u)mount/swaponoff ported to kFreeBSD; + * per-OS configuration overrides. + +* (./) Update custom packages to the latest versions + and send updated patches to the BTS + (2010-08-11) + * updated partman-base to choose a default filesystem in debian/rules + rather than at runtime, + as suggested by Aurelien Jarno in [[!debbug 586870]] + * patch submitted for debian-installer-utils + ([[!debbug 592684]]). + * patch submitted for locale-chooser + ([[!debbug 592690]]). + * debootstrap, grub-installer and finish-install not yet submitted, + since the details may still change. + +* (./) **partman-target**: fix fstab creation + (2010-08-11) + * See [[!debbug 592671]] + * debian/rules: set `partman/mount_style` to `traditional` on Hurd. + * finish.d/create\_fstab\_header: add a Hurd case. + +* (./) **rootskel**: FTBFS on Hurd and other quirks + (to be fixed very soon) + +* **d-i/installer/build**: (expected soon) + * publish the patch I use + * sort out the changes suitable for inclusion + and ask youpi and/or debian-boot@l.d.o to commit them + +* call for testing and fix the bugs + +* Bug in setup-translators/MAKEDEV: + permissions are broken for nodes re-created through `MAKEDEV -k`, + because MAKEDEV's chmod/chown reaches the pre-existing translator + * Maybe settrans could be made to accept -o/--owner and + -p/--perm, to set the permissions for the underlying node? + +* (./) Silence the "no kernel" warning somehow. + +* Investigate the wget/libc/pfinet/whatever bug which corrupts Packages.gz, + see the IRC log for 2010-07-23, around 1am UTC+0200 + +* Try to resolve problems with udebs which are uninstallable on hurd-i386, + such as installation-locale and partman-whatever. + +* Provide `/proc/cmdline -> 2/cmdline`, or something. + +* Prepare a NMU for genext2fs (which is orphaned), + and ask youpi to sponsor the upload. + +* **busybox**: port + * fix stty/stat/ipcs on kFreeBSD, + * generally port more stuff, + * *ip* is needed (maybe) for network configuration, + * *mount*, *swaponoff* can be from hurd-udeb for now, + though the kFreeBSD people will need them + +* **partman**: further adjustments + * partman-base: handle /dev/hd?s* in lib/base.h + * hide irrelevant mount options? (sync, relatime) + +* Network configuration on the installed system. + This includes porting ifupdown and isc-dhcp-client, + which are currently uninstallable on hurd-i386. +* Also, better DHCP support during and after installation + +* improve the [initrd situation](FIXME: link to bug-hurd post): + ajust the ramdisk support in Mach, + use tmpfs if possible. + +* mklibs{,-copy}: + test library reduction, + make it copy the ld.so -> ld.so.1 symlink. + +* (./) hurd console fonts + +**Milestone (expected 2010-07-19): +it works great and it's beautiful** + +* test, fix, document +* support more types of installation images +* give a shot at the graphical installer if time permits +* integrate wireless drivers with netcfg +* see how [[zhengda]]'s work on DDE could be integrated +* etc.. + +### Mostly done + +#### Week 1 (2010-05-24) + +* genext2fs: patches submitted, [[!debbug 562999]] + which add support for all block sizes and choosing them at runtime. +* busybox: started porting the upstream and Debian package to Hurd and FreeBSD +* rebuilding hurd-udeb from the pkg-hurd version + and adding a ld.so link to the initrd + fixes the exec translator crashing on startup. + (BTW would there be a mean to detect this from the libdiskfs bootstrap code + and report it ?) + +#### Week 2 (2010-05-31 to 2010-06-06) + +* *busybox*: patches [posted](http://lists.debian.org/debian-bsd/2010/05/msg00048.html). +* *libdebian-installer4*: [[!debbug 584538]] +* started working on mach initrd support +* the installation images could boot into the main-menu + with the following changes: + * rebuild hurd-udeb from with the latest pkg-hurd patches + * use busybox from my osports-debian branch (see link above) + * tweak the d-i image build scripts + * the symlink /lib/ld.so -> ld.so.1 needs to be created somehow + (youpi mentionned it being the job of libc0.3-udeb I think) + * fix the poll() issue in libdebian-installer + (patch to be submitted soon), + also there is some hurd doxygen short-circuiting stuff + there which does not apply any more and prevents is to build. + * feed the initrd as a hard drive in qemu + (with some more space added to avoid it from becoming full) + diff --git a/user/jkoenig/gsoc2011_proposal.mdwn b/user/jkoenig/gsoc2011_proposal.mdwn deleted file mode 100644 index 4052f455..00000000 --- a/user/jkoenig/gsoc2011_proposal.mdwn +++ /dev/null @@ -1,628 +0,0 @@ - -# Java for Hurd (and vice versa) - -Contact information: - - * Full name: Jérémie Koenig - * Email: jk@jk.fr.eu.org - * IRC: jkoenig on Freenode and OFTC - -## Introductions - -I am a first year M.Sc. student -in Computer Science at University of Strasbourg (France). -My interests include capability-based security, -programming languages and formal methods -(in particular, object-capability languages and proof-carrying code). - -### Proposal summary - -This project would consist in improving Java support on Hurd. -The first part would consist in -fixing bugs and porting Java-related packages. -The second part would consist in -creating low-level Java bindings for the Hurd interfaces, -as well as libraries to make translator development easier. - -### Previous involvement - -I started contributing to Hurd last summer, -during which I participated to Google Summer of Code -as a student for the Debian project. -I worked on porting Debian-Installer to Hurd. -This project was mostly a success, -although we still have to use a special mirror for installation -with a few modified packages -and tweaked priorities -to work around some uninstallable packages -with Priority: standard. - -Shortly afterwards, -I rewrote the procfs translator -to fix some issues with memory leaks, -make it more reliable, -and improve compatibility with Linux-based tools -such as `procps` or `htop`. - -Although I have not had as much time -as I would have liked to dedicate to the Hurd -since that time, -I have continued to maintain the mirror in question, -and I have started to work -on implementing POSIX threads signal semantics in glibc. - -### Project-related skills and interests - -I have used Java mostly for university assignments. -This includes non-trivial projects -using threads and distributed programming frameworks -such as Java RMI or CORBA. -I have also used it to experiment with -Google App Engine -(web applications) -and Google Web Toolkit -(a compiler from Java to Javascript which helps with AJAX code), -and I have some limited experience with JNI -(the Java Native Interface, to link Java with C code). - -My knowledge of the Hurd and Debian GNU/Hurd is reasonable, -as the Debian-Installer and procfs projects -gave me the opportunity to fiddle with many parts of the system. - -Initially, -I started working on this project because I wanted to use -[Joe-E](http://code.google.com/p/joe-e/) -(a subset of Java) -to investigate the potential -[[applications of object-capability languages|objcap]] -in a Hurd context. -I also believe that improving Java support on Hurd -would be an important milestone. - -### Organisational matters - -I am subscribed to bug-hurd@g.o and -I do have a permanent internet connexion. - -I would be able to attend the regular IRC meetings, -and otherwise communicate with my mentor -through any means they would prefer -(though I expect email and IRC would be the most practical). -Since I'm already familiar with the Hurd, -I don't expect I would require too much time from them. - -My exams end on May 20 so I would be able to start coding -right at the beginning of the GSoC period. -Next year's term would probably begin around September 15, -so that would not be an issue either. -I expect I would work around 40 hours per week, -and my waking hours would be flexible. - -I don't have any other plans for the summer -and would not make any if my project were to be accepted. - -Full disclosure: -I also submitted a proposal to the Jikes RVM project -(which is a research-oriented Java Virtual Machine, -itself written in Java) -for implementing a new garbage collector into the MMTk subsystem. - -## Improve Java support - -### Justification - -Java is a popular language and platform used by many desktop and web -applications (mostly on the server side). As a consequence, competitive Java -support is important for any general-purpose operating system. -Better Java support would also be a prerequisite -for the second part of my proposal. - -### Current situation - -Java is currently supported on Hurd with the GNU Java suite: - - * [GCJ](http://gcc.gnu.org/java/), - the GNU Compiler for Java, is part of GCC and can compile Java - source code to Java bytecode, and both source code and bytecode to - native code; - * libgcj is the implementation of the Java runtime which GCJ uses. - It is based on [GNU Classpath](http://www.gnu.org/software/classpath/). - It includes a bytecode interpreter which enables - Java applications compiled to native code to dynamically load and execute - Java bytecode from class files. - * The gij command is a wrapper around the above-mentioned virtual machine - functionality of libgcj and can be used as a replacement for the java - command. - -However, GCJ does not work flawlessly on Hurd.r -For instance, some parts of libgcj relies on -the POSIX threads signal semantics, which are not yet implemented. -In particular, this makes ant hang waiting for child processes, -which makes some packages fail to build on Hurd -(“ant” is the “make” of the Java world). - -### Tasks - - * **Finish implementing POSIX thread semantics** in glibc (high priority). - According to POSIX, signal dispositions should be global to a process, - while signal blocking masks should be thread-specific. Signals sent to the - process as a whole are to be delivered to any thread which does not block - them. By contrast, Hurd has per-thread signal dispositions and signals - sent to a process are delivered to the main thread only. I have been - working on refactoring the glibc signal code and implementing the POSIX - semantics as a per-thread option. However, due to lack of time I have not - yet been able to test and debug my code properly. Finishing this work - would be my first task. - * **Fix further problems with GCJ on Hurd** (high priority). While I’m not - aware of any other problems with GCJ at the moment, I suspect some might - turn up as I progress with the other tasks. Fixing these problems would - also be a high-priority task. - * **Port OpenJDK 6** (medium priority). While GCJ is fine, it is not yet - 100% complete. It is also slower than OpenJDK on architectures where a - just-in-time compiler is available. Porting OpenJDK would therefore - improve Java support on Hurd in scope and quality. Besides, it would also - be a good way to test GCJ, which is used for bootstrapping by the Debian - OpenJDK packages. Also note that OpenJDK 6 is now the default Java - Runtime Environment on all released Linux-based Debian architectures; - bringing Hurd in line with this would probably be a good thing. - * **Port Eclipse and other Java applications** (low priority). Eclipse is a - popular, state-of-the-art IDE and tool suite used for Java and other - languages. It is a dependency of the Joe-E verifier (see part 3 of this - proposal). Porting Eclipse would be a good opportunity to test GCJ and - OpenJDK. - -### Deliverables - - * The glibc pthreads patch and any other fixes on the Hurd side - would be submitted upstream - * Patches against Debian source packages - required to make them build on Hurd would be submitted - to the [Debian bug tracking system](http://bugs.debian.org/). - - -## Create Java bindings for the Hurd interfaces - -### Justification - -Java is used for many applications and often taught to -introduce object-oriented programming. The fact that Java is a -garbage-collected language makes it easier to use, especially for the less -experienced programmers. Besides, its object-oriented nature is a -natural fit for the capability-based design of Hurd. -The JVM is also used as a target for many other languages, -all of which would benefit from the access provided by these bindings. - -Advantages over other garbage-collected, object-oriented languages include -performance, type safety and the possibility to compile a Java translator to -native code and -[link it statically](http://gcc.gnu.org/wiki/Statically_linking_libgcj) -using GCJ, should anyone want to use a -translator written in Java for booting. -Note that Java is -[being](http://www.linuxjournal.com/article/8757) -[used](http://oss.readytalk.com/avian/) -in this manner for embedded development. -Since GCJ can take bytecode as its input, -this expect this possibility would apply to any JVM-based language. - -Java bindings would lower the bar for newcomers -to begin experimenting with what makes Hurd unique -without being faced right away with the complexity of -low-level systems programming. - -### Tasks summary - - * Implement Java bindings for Mach - * Implement a libports-like library for Java - * Modify MIG to output Java code - * Implement libfoofs-like Java libraries - -### Design principles - -The principles I would use to guide the design -of these Java bindings would be the following ones: - - * The system should be hooked into at a low level, - to ensure that Java is a "first class citizen" - as far as the access to the Hurd's interfaces is concerned. - * At the same time, the memory safety of Java should be maintained - and extended to Mach primitives such as port names and - out-of-line memory regions. - * Higher-level interfaces should be provided as well - in order to make translator development - as easy as possible. - * A minimum amount of JNI code (ie. C code) should be used. - Most of the system should be built using Java itself - on top of a few low-level primitives. - * Hurd objects would map to Java objects. - * Using the same interfaces, - objects corresponding to local ports would be accessed directly, - and remote objects would be accessed over IPC. - -One approach used previously to interface programming languages with the Hurd -has been to create bindings for helper libraries such as libtrivfs. Instead, -for Java I would like to take a lower-level approach by providing access to -Mach primitives and extending MIG to generate Java code from the interface -description files. - -This approach would be initially more involved, and would introduces several -issues related to overcoming the "impedance mismatch" between Java and Mach. -However, once an initial implementation is done it would be easier to maintain -in the long run and we would be able to provide Java bindings for a large -percentage of the Hurd’s interfaces. - -### Bindings for Mach system calls - -In this low-level approach, my intention is to enable Java code to use Mach -system calls (in particular, mach_msg) more or less directly. This would -ensure full access to the system from Java code, but it raises a number of -issues: - - * the Java code must be able to manipulate Mach-level entities, such as port - rights or page-aligned buffers mapped outside of the garbage-collected - heap (for out-of-line transfers); - * putting together IPC messages requires control of the low-level - representation of data. - -In order to address these concerns, classes would be encapsulating these -low-level entities so that they can be referenced through normal, safe objects -from standard Java code. Bindings for Mach system calls can then be provided -in terms of these classes. Their implementation would use C code through the -Java Native Interface (JNI). - -More specifically, this functionality would be provided by the `org.gnu.mach` -package, which would contain at least the following classes: - - * `MachPort` would encapsulate a `mach_port_t`. (Some of) its constructors - would act as an interface for the `mach_port_allocate()` system call. - `MachPort` objects would also be instantiated from other parts of the JNI - C code to represent port rights received through IPC. The `deallocate()` - method would call `mach_port_deallocate()` and replace the encapsulated - port name with `MACH_PORT_DEAD`. We would recommend that users call it - when a port is no longer used, but the finalizer would also deallocate the - port when the `MachPort` object is garbage collected. - * `Buffer` would represent a page-aligned buffer allocated outside of the - Java heap, to be transferred (or having been received) as out-of-line - memory. The JNI code would would provide methods to read and write data at - an arbitrary offset (but within bounds) and would use `vm_allocate()` and - `vm_deallocate()` in the same spirit as for `MachPort` objects. - * `Message` would allow Java code to put together Mach messages. The - constructor would allocate a `byte[]` member array of a given size. - Additional methods would be provided to fill in or query the information - in the message header and additional data items, including `MachPort` and - `Buffer` objects which would be translated to the corresponding port names - and out-of-line pointers. - A global map from port names to the corresponding `MachPort` object - would probably be needed to ensure that there is a one-to-one - correspondence. - * `Syscall` would provide static JNI methods for performing system calls not - covered by the above classes, such as `mach_msg()` or - `mach_thread_self()`. These methods would accept or return `MachPort`, - `Buffer` and `Message` objects when appropriate. The associated C code - would access the contents of such objects directly in order to perform the - required unsafe operations, such as constructing `MachPort` and `Buffer` - objects directly from port names and C pointers. - -Note that careful consideration should be given to the interfaces of these -classes to avoid “safety leaks” which would compromise the safety guarantees -provided by Java. Potential problematic scenarios include the following -examples: - - * It must not be possible to write an integer at some position in a - `Message` object, and to read it back as a `MachPort` or `Buffer` object, - since this would allow unsafe access to arbitrary memory addresses and - mach port names. - * Providing the `mach_task_self()` system call would also provide access to - arbitrary addresses and ports by using the `vm_*` family of RPC operations - with the returned `MachPort` object. This means that the relevant task - operations should be provided by the `Syscall` class instead. - -Finally, access should be provided to the initial ports and file descriptors -in `_hurd_ports` and provided by the `getdport()` function, -for instance through static methods such as -`getCRDir()`, `getCWDir()`, `getProc()`, ... in a dedicated class such as -`org.gnu.hurd.InitPorts`. - -A realistic example of code based on such interfaces would be: - - import org.gnu.mach.MsgType; - import org.gnu.mach.MachPort; - import org.gnu.mach.Buffer; - import org.gnu.mach.Message; - import org.gnu.mach.Syscall; - import org.gnu.hurd.InitPorts; - - public class Hello - { - public static main(String argv[]) - /* Parent class for all Mach-related exceptions */ - throws org.gnu.mach.MachException - { - /* Allocate a reply port */ - MachPort reply = new MachPort(); - - /* Allocate an out-of-line buffer */ - Buffer data = new Buffer(MsgType.CHAR, 13); - data.writeString(0, "Hello, World!"); - - /* Craft an io_write message */ - Message msg = new Message(1024); - msg.setRemotePort(InitPorts.getdport(1)); - msg.setLocalPort(reply, Message.Type.MAKE_SEND_ONCE); - msg.setId(21000); - msg.addBuffer(data); - - /* Make the call, MACH_MSG_SEND | MACH_MSG_RECEIVE */ - Syscall.machMsg(msg, true, true, reply); - - /* Extract the returned value */ - msg.assertId(21100); - int retCode = msg.readInt(0); - int amount = msg.readInt(1); - } - } - -Should this paradigm prove insufficient, -more ideas could be borrowed from the -[`org.vmmagic`](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.151.5253&rep=rep1&type=pdf) -package used by [Jikes RVM](http://jikesrvm.org/), -a research Java virtual machine itself written in Java. - -### Generating Java stubs with MIG - -Once the basic machinery is in place to interface with Mach, Java programs -have more or less equal access to the system functionality without resorting -to more JNI code. However, as illustrated above, this access is far from -convenient. - -As a solution I would modify MIG to add the option to output Java code. MIG -would emit a Java interface, a client class able to implement the interface -given a Mach port send right, an a server class which would be able to handle -incoming messages. The class diagram below, although it is by no means -complete or exempt of any problem, illustrates the general idea: - -[[gsoc2011_classes.png]] - -This structure is somewhat reminiscent of -[Java RMI](http://en.wikipedia.org/wiki/Java_remote_method_invocation) -or similar systems, -which aim to provide more or less transparent access to remote objects. -The exact way the Java code would be generated still needs to be determined, -but basically: - - * An interface, corresponding to the header files generated by MIG, would - enumerate the operations listed in a given .defs files. Method names would - be transformed to adhere to Java conventions (for instance, - `some_random_identifier` would become `someRandomIdentifier`). - * A user class, corresponding to the `*User.c` files, - would implement this interface by doing RPC over a given MachPort object. - * A server class, corresponding to `*Server.c`, would be able to handle - incoming messages using a user-provided implementation of the interface. - (Possibly, a skeleton class providing methods which would raise - `NotImplementedException`s would be provided as well. - Users would derive from this class and override the relevant methods. - This would allow them not to implement some operations, - and would avoid pre-existing code from breaking when new operations are - introduced.) - -In order to help with the implementation of servers, some kind of library -would be needed to associate Mach receive rights with server objects and to -handle incoming messages on dedicated threads, in the spirit of libports. -This would probably require support for port sets at the level of the Mach -primitives described in the previous section. - -When possible, operations involving the transmission of send rights -of some kind would be expressed in terms of the MIG-generated interfaces -instead of `MachPort` objects. -Upon reception of a send right, -a `FooUser` object would be created -and associated with the corresponding `MachPort` object. -If the received send right corresponds to a local port -to which a server object has been associated, -this object would be used instead. -This way, -subsequent operations on the received send right -would be handled as direct method calls -instead of going through RPC mechanisms. - -Some issues will still need to be solved regarding how MIG will convert -interface description files to Java interfaces. For instance: - - * `.defs` files are not explicitly associated with a type. For instance in - the example above, MIG would have to somehow infer that io_t corresponds - to `this` in the `Io` interface. - * More generally, a correspondence between MIG and Java types would have - to be determined. Ideally this would be automated and not hardcoded - too much. - * Initially, reply port parameters would be ignored. However they may be - needed for some applications. - -So the details would need to be flushed out during the community bonding -period and as the implementation progresses. However I’m confident that a -satisfactory solution can be designed. - -Using these new features, the example above could be rewritten as: - - import org.gnu.hurd.InitPorts; - import org.gnu.hurd.Io; - import org.gnu.hurd.IoUser; - - class Hello { - static void main(String argv[]) throws ... - { - Io stdout = new IoUser(InitPorts.getdport(1)); - String hello = “Hello, World!\n”; - - int amount = stdout.write(hello.getBytes(), -1); - - /* (A retCode corresponding to an error - would be signalled as an exception.) */ - } - } - -An example of server implementation would be: - - import org.gnu.hurd.Io; - import java.util.Arrays; - - class HelloIo implements Io { - final byte[] contents = “Hello, World!\n”.getBytes(); - - int write(byte[] data, int offset) { - return SOME_ERROR_CODE; - } - - byte[] read(int offset, int amount) { - return Arrays.copyOfRange(contents, offset, - offset + amount - 1); - } - - /* ... */ - } - -A new server object could then be created with `new IoServer(new HelloIo())`, -and associated with some receive right at the level of the ports management -library. - -### Base classes for common types of translators - -Once MIG can target Java code, and a libports equivalent is available, -creating new translators in Java would be greatly facilitated. However, -we would probably want to introduce basic implementations of file system -translators in the spirit of libtrivfs or libnetfs. They could take the form -of base classes implementing the relevant MIG-generated interfaces which -would then be derived by users, -or could define a simpler interface -which would then be used by adapter classes -to implement the required ones. - -I would draw inspiration from libtrivfs and libnetfs -to design and implement similar solutions for Java. - -### Deliverables - - * A hurd-java package would contain the Java code developed - in the context of this project. - * The Java code would be documented using javadoc - and a tutorial for writing translators would be written as well. - * Modifications to MIG would be submitted upstream, - or a patched MIG package would be made available. - -The Java libraries resulting from this work, -including any MIG support classes -as well as the class files built from the MIG-generated code -for the Mach and Hurd interface definition files, -would be provided as single `hurd-java` package for -Debian GNU/Hurd. -This package would be separate from both Hurd and Mach, -so as not to impose unreasonable build dependencies on them. - -I expect I would be able to act as its maintainer in the foreseeable future, -either as an individual or as a part of the Hurd team. -Hopefully, -my code would be claimed by the Hurd project as their own, -and consequently the modifications to MIG -(which would at least conceptually depend on the Mach Java package) -could be integrated upstream. - -Since by design, -the Java code would use only a small number of stable interfaces, -it would not be subject to excessive amounts of bitrot. -Consequently, -maintenance would primarily consist in -fixing bugs as they are reported, -and adding new features as they are requested. -A large number of such requests -would mean the package is useful, -so I expect that the overall amount of work -would be correlated with the willingness of more people -to help with maintenance -should I become overwhelmed or get hit by a bus. - - -## Timeline - -The dates listed are deadlines for the associated tasks. - - * *Community bonding period.* - Discuss, refine and complete the design of the Java bindings - (in particular the MIG and "libports" parts) - * *May 23.* - Coding starts. - * *May 30.* - Finish implementing pthread signal semantics. - * *June 5.* - Port OpenJDK - * *June 12.* - Fix the remaining problems with GCJ and/or OpenJDK, - possibly port Eclipse or other big Java packages. - * *June 19.* - Create the bindings for Mach. - * *June 26.* - Work on some kind of basic Java libports - to handle receive rights. - * *July 3.* - Test, write some documentation and examples. - * *July 17 (two weeks).* - Add the Java target to MIG. - * *July 24.* - Test, write some documentation and examples. - * *August 7 (two weeks).* - Implement a modular libfoofs to help with translator development. - Try to write a basic but non-trivial translator - to evaluate the performance and ease of use of the result, - rectify any rough edges this would uncover. - * *August 22. (last two weeks)* - Polish the code and packaging, - finish writing the documentation. - - -## Conclusion - -This project is arguably ambitious. -However, I have been thinking about it for some time now -and I'm confident I would be able to accomplish most of it. - -In the event multiple language bindings projects -would be accepted, -some work could probably be done in common. -In particular, -[ArneBab](http://www.bddebian.com/~hurd-web/community/weblogs/ArneBab/2011-04-06-application-pyhurd/) -seems to favor a low-level approach for his Python bindings as I do for Java, -and I would be happy to discuss API design and coordinate MIG changes with him. -I would also have an extra month after the end of the GSoC period -before I go back to school, -which I would be able to use to finish the project -if there is some remaining work. -(Last year's rewrite of procfs was done during this period.) - -As for the project's benefits, -I believe that good support for Java -is a must-have for the Hurd. -Java bindings would also further the Hurd's agenda -of user freedom by extending this freedom to more people: -I expect the set of developers -who would be able to write Java code against a well-written libfoofs -is much larger than -those who master the intricacies of low-level systems C programming. -From a more strategic point of view, -this would also help recruit new contributors -by providing an easier path to learning the inner workings of the Hurd. - -Further developments -which would build on the results of this project -include my planned [[experiment with Joe-E|objcap]] -(which I would possibly take on as a university project next year). -Another possibility would be to reimplement some parts -of the Java standard library -directly in terms of the Hurd interfaces -instead of using the POSIX ones through glibc. -This would possibly improve the performance -of some Java applications (though probably not by much), -and would otherwise be a good project -for someone trying to get acquainted with Hurd. - -Overall, I believe this project would be fun, interesting and useful. -I hope that you will share this sentiment -and give me the opportunity to spend another summer working on Hurd. - diff --git a/user/jkoenig/gsoc2011_proposal/discussion.mdwn b/user/jkoenig/gsoc2011_proposal/discussion.mdwn deleted file mode 100644 index 0131d8d5..00000000 --- a/user/jkoenig/gsoc2011_proposal/discussion.mdwn +++ /dev/null @@ -1,180 +0,0 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -Some [[tschwinge]] comments regarding your proposal. Which is very good, if I -may say so again! :-) - -Of course, everyone is invited to contribute here! - -I want to give the following methodology a try, instead of only having -email/IRC discussions -- for the latter are again and again showing a tendency -to be dumped and deposited into their respective archives, and be forgotten -there. Of course, email/IRC discussions have their usefulness too, so we're -not going to replace them totally. For example, for conducting discussions -with a bunch of people (who may not even be following these pages here), email -(or, as applicable, the even more interactive IRC) will still be the medium of -choice. (And then, the executive summary should be posted here, or -incorporated into your proposal.) - -Also, if you disagree with this suggested procedure right away, or at some -later point begin to feel that this thing doesn't work out, or simply takes too -much time (I don't think so: writing emails takes time, too), just say so, and -we can reconsider. - -Of course, as this wiki is a passive medium rather than an active one as IRC -and email are, it is fine to send notices like: *I have updated the wiki page, -please have a look*. - -One idea is that your proposal evolves alongside with the ongoing work, and -represents (in more or less detail) what has been done and what will be done. -Also, we can hopefully use parts of it for documentation purposes, or as -recipes for similar work (enabling other programming languages on the Hurd, for -example). - -For this, I suggest the following procedure: as applicable, you can either -address any comments in here (for example, if they're wrong :-), or if they -require further discussion; think: *email discussion*), or you can address them -directly in your propoal and remove the comments from here at the same time -(think: *bug fix*). - -Generally, you can assume that for things I didn't comment on (within some -reasonable timeframe/upon asking me again) that I'm fine with them. Otherwise, -I might say: *I don't like this as is, but I'll need more time to think about -it.* - -There is also a possibility that parts of your proposal will be split off; in -cases where we think they're valuable to follow, but not at this time. (As you -know, your proposal is not really a trivial one, so it may just be too much for -one person's summer.) Such bits could be moved to [[open_issues]] pages, -either new ones or existing ones, as applicable. - - -# POSIX Threads Signal Semantics - - * Great! [[tschwinge]] had a brief look, and should have a deeper one. - - * If [[jkoenig]] thinks it's mature enough: should ask Samuel to test this - (that is, only the refactoring patches for starters?) on the buildds. - - * Then: should ask Roland to review. - - * Documentations bits should probably be moved to [[glibc/signal]]. - - -## libthreads (cthreads) Integration - - * [[tschwinge]] suggests to leave them as-is? - - -## [[libpthread]] integration - - * To be done. - - -# Java - - * [[tschwinge]] has to read about RMI and CORBA. - - -# Joe-E - - * For later. - - -# GCJ - - * [[tschwinge]] has the feeling that Java in GCC (that is, GCJ) is mostly - dead? (True?) - - * Thus perhaps not too much effort should be spent with it. - - If the POSIX threads signal semantics makes it going, then great, otherwise - we should get a feeling what else is missing. - - -# OpenJDK - - * All in all, [[tschwinge]] has the feeling that a working OpenJDK will be - more useful/powerful than GCJ. - - * We need to get a feeling how difficult such an OS port will be. - - * [[jkoenig]] suggests OpenJDK 6 -- should we directly go for version 7 - instead? - - * What are the differences (regarding the OS port) between the two - versions? Or this there something even more recent to be worked upon, - for new OS ports? - - * Perhaps the different versions' OS port specific stuff is not at - all very different, so that both v6 and v7 could be done? - - * They seem to have a rather heavy-weight process for such projects: confer - , - for example. Do we need this, too? - - -# Eclipse - -OK for testing -- but I'd very much hope that it *just works* as soon as we -provide the required Java platform. - - -# Java Bindings - - -## Design Principles - - * Generally ack. - - -### MIG - - * Hacking [[microkernel/mach/MIG]] shouldn't be too difficult. - - * (Unless you want to make MIG's own code (that is, not the generated - code, but MIG itself) look a bit more nice, too.) ;-) - - * There are also alternatives to MIG. If there is interest, the following - could be considered: - - * FLICK ([[!GNU_Savannah_task 5723]]). [[tschwinge]] has no idea yet if - there would be any benefits over MIG, like better modularity (for the - backends)? If we feel like it, we could spend a little bit of time on - this. - - * For [[microkernel/Viengoos]], Neal has written a RPC stub generator - entirely in C Preprocessor macros. While this is obviously not - directly applicable, perhaps we can get some ideas from it. - - * Anything else that would be worth having a look at? (What are other - microkernels using?) - - -### `mach_msg` - - * Seems like the right approach to [[tschwinge]], but hasn't digested all the - pecularities yet. Will definitely need more time. - - -# GSoC Site Discussion - - * Discussion items from - - should be copied here: - - * technical bits (obviously); - - * also the *why do we want Java bindings* reasoning; - - * CLISP findings should also be documented somewhere permanently. - - * We should probaby open up a *languages for Hurd* section on the web - pages ([[!taglink open_issue_documentation]]). diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn new file mode 100644 index 00000000..4052f455 --- /dev/null +++ b/user/jkoenig/java.mdwn @@ -0,0 +1,628 @@ + +# Java for Hurd (and vice versa) + +Contact information: + + * Full name: Jérémie Koenig + * Email: jk@jk.fr.eu.org + * IRC: jkoenig on Freenode and OFTC + +## Introductions + +I am a first year M.Sc. student +in Computer Science at University of Strasbourg (France). +My interests include capability-based security, +programming languages and formal methods +(in particular, object-capability languages and proof-carrying code). + +### Proposal summary + +This project would consist in improving Java support on Hurd. +The first part would consist in +fixing bugs and porting Java-related packages. +The second part would consist in +creating low-level Java bindings for the Hurd interfaces, +as well as libraries to make translator development easier. + +### Previous involvement + +I started contributing to Hurd last summer, +during which I participated to Google Summer of Code +as a student for the Debian project. +I worked on porting Debian-Installer to Hurd. +This project was mostly a success, +although we still have to use a special mirror for installation +with a few modified packages +and tweaked priorities +to work around some uninstallable packages +with Priority: standard. + +Shortly afterwards, +I rewrote the procfs translator +to fix some issues with memory leaks, +make it more reliable, +and improve compatibility with Linux-based tools +such as `procps` or `htop`. + +Although I have not had as much time +as I would have liked to dedicate to the Hurd +since that time, +I have continued to maintain the mirror in question, +and I have started to work +on implementing POSIX threads signal semantics in glibc. + +### Project-related skills and interests + +I have used Java mostly for university assignments. +This includes non-trivial projects +using threads and distributed programming frameworks +such as Java RMI or CORBA. +I have also used it to experiment with +Google App Engine +(web applications) +and Google Web Toolkit +(a compiler from Java to Javascript which helps with AJAX code), +and I have some limited experience with JNI +(the Java Native Interface, to link Java with C code). + +My knowledge of the Hurd and Debian GNU/Hurd is reasonable, +as the Debian-Installer and procfs projects +gave me the opportunity to fiddle with many parts of the system. + +Initially, +I started working on this project because I wanted to use +[Joe-E](http://code.google.com/p/joe-e/) +(a subset of Java) +to investigate the potential +[[applications of object-capability languages|objcap]] +in a Hurd context. +I also believe that improving Java support on Hurd +would be an important milestone. + +### Organisational matters + +I am subscribed to bug-hurd@g.o and +I do have a permanent internet connexion. + +I would be able to attend the regular IRC meetings, +and otherwise communicate with my mentor +through any means they would prefer +(though I expect email and IRC would be the most practical). +Since I'm already familiar with the Hurd, +I don't expect I would require too much time from them. + +My exams end on May 20 so I would be able to start coding +right at the beginning of the GSoC period. +Next year's term would probably begin around September 15, +so that would not be an issue either. +I expect I would work around 40 hours per week, +and my waking hours would be flexible. + +I don't have any other plans for the summer +and would not make any if my project were to be accepted. + +Full disclosure: +I also submitted a proposal to the Jikes RVM project +(which is a research-oriented Java Virtual Machine, +itself written in Java) +for implementing a new garbage collector into the MMTk subsystem. + +## Improve Java support + +### Justification + +Java is a popular language and platform used by many desktop and web +applications (mostly on the server side). As a consequence, competitive Java +support is important for any general-purpose operating system. +Better Java support would also be a prerequisite +for the second part of my proposal. + +### Current situation + +Java is currently supported on Hurd with the GNU Java suite: + + * [GCJ](http://gcc.gnu.org/java/), + the GNU Compiler for Java, is part of GCC and can compile Java + source code to Java bytecode, and both source code and bytecode to + native code; + * libgcj is the implementation of the Java runtime which GCJ uses. + It is based on [GNU Classpath](http://www.gnu.org/software/classpath/). + It includes a bytecode interpreter which enables + Java applications compiled to native code to dynamically load and execute + Java bytecode from class files. + * The gij command is a wrapper around the above-mentioned virtual machine + functionality of libgcj and can be used as a replacement for the java + command. + +However, GCJ does not work flawlessly on Hurd.r +For instance, some parts of libgcj relies on +the POSIX threads signal semantics, which are not yet implemented. +In particular, this makes ant hang waiting for child processes, +which makes some packages fail to build on Hurd +(“ant” is the “make” of the Java world). + +### Tasks + + * **Finish implementing POSIX thread semantics** in glibc (high priority). + According to POSIX, signal dispositions should be global to a process, + while signal blocking masks should be thread-specific. Signals sent to the + process as a whole are to be delivered to any thread which does not block + them. By contrast, Hurd has per-thread signal dispositions and signals + sent to a process are delivered to the main thread only. I have been + working on refactoring the glibc signal code and implementing the POSIX + semantics as a per-thread option. However, due to lack of time I have not + yet been able to test and debug my code properly. Finishing this work + would be my first task. + * **Fix further problems with GCJ on Hurd** (high priority). While I’m not + aware of any other problems with GCJ at the moment, I suspect some might + turn up as I progress with the other tasks. Fixing these problems would + also be a high-priority task. + * **Port OpenJDK 6** (medium priority). While GCJ is fine, it is not yet + 100% complete. It is also slower than OpenJDK on architectures where a + just-in-time compiler is available. Porting OpenJDK would therefore + improve Java support on Hurd in scope and quality. Besides, it would also + be a good way to test GCJ, which is used for bootstrapping by the Debian + OpenJDK packages. Also note that OpenJDK 6 is now the default Java + Runtime Environment on all released Linux-based Debian architectures; + bringing Hurd in line with this would probably be a good thing. + * **Port Eclipse and other Java applications** (low priority). Eclipse is a + popular, state-of-the-art IDE and tool suite used for Java and other + languages. It is a dependency of the Joe-E verifier (see part 3 of this + proposal). Porting Eclipse would be a good opportunity to test GCJ and + OpenJDK. + +### Deliverables + + * The glibc pthreads patch and any other fixes on the Hurd side + would be submitted upstream + * Patches against Debian source packages + required to make them build on Hurd would be submitted + to the [Debian bug tracking system](http://bugs.debian.org/). + + +## Create Java bindings for the Hurd interfaces + +### Justification + +Java is used for many applications and often taught to +introduce object-oriented programming. The fact that Java is a +garbage-collected language makes it easier to use, especially for the less +experienced programmers. Besides, its object-oriented nature is a +natural fit for the capability-based design of Hurd. +The JVM is also used as a target for many other languages, +all of which would benefit from the access provided by these bindings. + +Advantages over other garbage-collected, object-oriented languages include +performance, type safety and the possibility to compile a Java translator to +native code and +[link it statically](http://gcc.gnu.org/wiki/Statically_linking_libgcj) +using GCJ, should anyone want to use a +translator written in Java for booting. +Note that Java is +[being](http://www.linuxjournal.com/article/8757) +[used](http://oss.readytalk.com/avian/) +in this manner for embedded development. +Since GCJ can take bytecode as its input, +this expect this possibility would apply to any JVM-based language. + +Java bindings would lower the bar for newcomers +to begin experimenting with what makes Hurd unique +without being faced right away with the complexity of +low-level systems programming. + +### Tasks summary + + * Implement Java bindings for Mach + * Implement a libports-like library for Java + * Modify MIG to output Java code + * Implement libfoofs-like Java libraries + +### Design principles + +The principles I would use to guide the design +of these Java bindings would be the following ones: + + * The system should be hooked into at a low level, + to ensure that Java is a "first class citizen" + as far as the access to the Hurd's interfaces is concerned. + * At the same time, the memory safety of Java should be maintained + and extended to Mach primitives such as port names and + out-of-line memory regions. + * Higher-level interfaces should be provided as well + in order to make translator development + as easy as possible. + * A minimum amount of JNI code (ie. C code) should be used. + Most of the system should be built using Java itself + on top of a few low-level primitives. + * Hurd objects would map to Java objects. + * Using the same interfaces, + objects corresponding to local ports would be accessed directly, + and remote objects would be accessed over IPC. + +One approach used previously to interface programming languages with the Hurd +has been to create bindings for helper libraries such as libtrivfs. Instead, +for Java I would like to take a lower-level approach by providing access to +Mach primitives and extending MIG to generate Java code from the interface +description files. + +This approach would be initially more involved, and would introduces several +issues related to overcoming the "impedance mismatch" between Java and Mach. +However, once an initial implementation is done it would be easier to maintain +in the long run and we would be able to provide Java bindings for a large +percentage of the Hurd’s interfaces. + +### Bindings for Mach system calls + +In this low-level approach, my intention is to enable Java code to use Mach +system calls (in particular, mach_msg) more or less directly. This would +ensure full access to the system from Java code, but it raises a number of +issues: + + * the Java code must be able to manipulate Mach-level entities, such as port + rights or page-aligned buffers mapped outside of the garbage-collected + heap (for out-of-line transfers); + * putting together IPC messages requires control of the low-level + representation of data. + +In order to address these concerns, classes would be encapsulating these +low-level entities so that they can be referenced through normal, safe objects +from standard Java code. Bindings for Mach system calls can then be provided +in terms of these classes. Their implementation would use C code through the +Java Native Interface (JNI). + +More specifically, this functionality would be provided by the `org.gnu.mach` +package, which would contain at least the following classes: + + * `MachPort` would encapsulate a `mach_port_t`. (Some of) its constructors + would act as an interface for the `mach_port_allocate()` system call. + `MachPort` objects would also be instantiated from other parts of the JNI + C code to represent port rights received through IPC. The `deallocate()` + method would call `mach_port_deallocate()` and replace the encapsulated + port name with `MACH_PORT_DEAD`. We would recommend that users call it + when a port is no longer used, but the finalizer would also deallocate the + port when the `MachPort` object is garbage collected. + * `Buffer` would represent a page-aligned buffer allocated outside of the + Java heap, to be transferred (or having been received) as out-of-line + memory. The JNI code would would provide methods to read and write data at + an arbitrary offset (but within bounds) and would use `vm_allocate()` and + `vm_deallocate()` in the same spirit as for `MachPort` objects. + * `Message` would allow Java code to put together Mach messages. The + constructor would allocate a `byte[]` member array of a given size. + Additional methods would be provided to fill in or query the information + in the message header and additional data items, including `MachPort` and + `Buffer` objects which would be translated to the corresponding port names + and out-of-line pointers. + A global map from port names to the corresponding `MachPort` object + would probably be needed to ensure that there is a one-to-one + correspondence. + * `Syscall` would provide static JNI methods for performing system calls not + covered by the above classes, such as `mach_msg()` or + `mach_thread_self()`. These methods would accept or return `MachPort`, + `Buffer` and `Message` objects when appropriate. The associated C code + would access the contents of such objects directly in order to perform the + required unsafe operations, such as constructing `MachPort` and `Buffer` + objects directly from port names and C pointers. + +Note that careful consideration should be given to the interfaces of these +classes to avoid “safety leaks” which would compromise the safety guarantees +provided by Java. Potential problematic scenarios include the following +examples: + + * It must not be possible to write an integer at some position in a + `Message` object, and to read it back as a `MachPort` or `Buffer` object, + since this would allow unsafe access to arbitrary memory addresses and + mach port names. + * Providing the `mach_task_self()` system call would also provide access to + arbitrary addresses and ports by using the `vm_*` family of RPC operations + with the returned `MachPort` object. This means that the relevant task + operations should be provided by the `Syscall` class instead. + +Finally, access should be provided to the initial ports and file descriptors +in `_hurd_ports` and provided by the `getdport()` function, +for instance through static methods such as +`getCRDir()`, `getCWDir()`, `getProc()`, ... in a dedicated class such as +`org.gnu.hurd.InitPorts`. + +A realistic example of code based on such interfaces would be: + + import org.gnu.mach.MsgType; + import org.gnu.mach.MachPort; + import org.gnu.mach.Buffer; + import org.gnu.mach.Message; + import org.gnu.mach.Syscall; + import org.gnu.hurd.InitPorts; + + public class Hello + { + public static main(String argv[]) + /* Parent class for all Mach-related exceptions */ + throws org.gnu.mach.MachException + { + /* Allocate a reply port */ + MachPort reply = new MachPort(); + + /* Allocate an out-of-line buffer */ + Buffer data = new Buffer(MsgType.CHAR, 13); + data.writeString(0, "Hello, World!"); + + /* Craft an io_write message */ + Message msg = new Message(1024); + msg.setRemotePort(InitPorts.getdport(1)); + msg.setLocalPort(reply, Message.Type.MAKE_SEND_ONCE); + msg.setId(21000); + msg.addBuffer(data); + + /* Make the call, MACH_MSG_SEND | MACH_MSG_RECEIVE */ + Syscall.machMsg(msg, true, true, reply); + + /* Extract the returned value */ + msg.assertId(21100); + int retCode = msg.readInt(0); + int amount = msg.readInt(1); + } + } + +Should this paradigm prove insufficient, +more ideas could be borrowed from the +[`org.vmmagic`](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.151.5253&rep=rep1&type=pdf) +package used by [Jikes RVM](http://jikesrvm.org/), +a research Java virtual machine itself written in Java. + +### Generating Java stubs with MIG + +Once the basic machinery is in place to interface with Mach, Java programs +have more or less equal access to the system functionality without resorting +to more JNI code. However, as illustrated above, this access is far from +convenient. + +As a solution I would modify MIG to add the option to output Java code. MIG +would emit a Java interface, a client class able to implement the interface +given a Mach port send right, an a server class which would be able to handle +incoming messages. The class diagram below, although it is by no means +complete or exempt of any problem, illustrates the general idea: + +[[gsoc2011_classes.png]] + +This structure is somewhat reminiscent of +[Java RMI](http://en.wikipedia.org/wiki/Java_remote_method_invocation) +or similar systems, +which aim to provide more or less transparent access to remote objects. +The exact way the Java code would be generated still needs to be determined, +but basically: + + * An interface, corresponding to the header files generated by MIG, would + enumerate the operations listed in a given .defs files. Method names would + be transformed to adhere to Java conventions (for instance, + `some_random_identifier` would become `someRandomIdentifier`). + * A user class, corresponding to the `*User.c` files, + would implement this interface by doing RPC over a given MachPort object. + * A server class, corresponding to `*Server.c`, would be able to handle + incoming messages using a user-provided implementation of the interface. + (Possibly, a skeleton class providing methods which would raise + `NotImplementedException`s would be provided as well. + Users would derive from this class and override the relevant methods. + This would allow them not to implement some operations, + and would avoid pre-existing code from breaking when new operations are + introduced.) + +In order to help with the implementation of servers, some kind of library +would be needed to associate Mach receive rights with server objects and to +handle incoming messages on dedicated threads, in the spirit of libports. +This would probably require support for port sets at the level of the Mach +primitives described in the previous section. + +When possible, operations involving the transmission of send rights +of some kind would be expressed in terms of the MIG-generated interfaces +instead of `MachPort` objects. +Upon reception of a send right, +a `FooUser` object would be created +and associated with the corresponding `MachPort` object. +If the received send right corresponds to a local port +to which a server object has been associated, +this object would be used instead. +This way, +subsequent operations on the received send right +would be handled as direct method calls +instead of going through RPC mechanisms. + +Some issues will still need to be solved regarding how MIG will convert +interface description files to Java interfaces. For instance: + + * `.defs` files are not explicitly associated with a type. For instance in + the example above, MIG would have to somehow infer that io_t corresponds + to `this` in the `Io` interface. + * More generally, a correspondence between MIG and Java types would have + to be determined. Ideally this would be automated and not hardcoded + too much. + * Initially, reply port parameters would be ignored. However they may be + needed for some applications. + +So the details would need to be flushed out during the community bonding +period and as the implementation progresses. However I’m confident that a +satisfactory solution can be designed. + +Using these new features, the example above could be rewritten as: + + import org.gnu.hurd.InitPorts; + import org.gnu.hurd.Io; + import org.gnu.hurd.IoUser; + + class Hello { + static void main(String argv[]) throws ... + { + Io stdout = new IoUser(InitPorts.getdport(1)); + String hello = “Hello, World!\n”; + + int amount = stdout.write(hello.getBytes(), -1); + + /* (A retCode corresponding to an error + would be signalled as an exception.) */ + } + } + +An example of server implementation would be: + + import org.gnu.hurd.Io; + import java.util.Arrays; + + class HelloIo implements Io { + final byte[] contents = “Hello, World!\n”.getBytes(); + + int write(byte[] data, int offset) { + return SOME_ERROR_CODE; + } + + byte[] read(int offset, int amount) { + return Arrays.copyOfRange(contents, offset, + offset + amount - 1); + } + + /* ... */ + } + +A new server object could then be created with `new IoServer(new HelloIo())`, +and associated with some receive right at the level of the ports management +library. + +### Base classes for common types of translators + +Once MIG can target Java code, and a libports equivalent is available, +creating new translators in Java would be greatly facilitated. However, +we would probably want to introduce basic implementations of file system +translators in the spirit of libtrivfs or libnetfs. They could take the form +of base classes implementing the relevant MIG-generated interfaces which +would then be derived by users, +or could define a simpler interface +which would then be used by adapter classes +to implement the required ones. + +I would draw inspiration from libtrivfs and libnetfs +to design and implement similar solutions for Java. + +### Deliverables + + * A hurd-java package would contain the Java code developed + in the context of this project. + * The Java code would be documented using javadoc + and a tutorial for writing translators would be written as well. + * Modifications to MIG would be submitted upstream, + or a patched MIG package would be made available. + +The Java libraries resulting from this work, +including any MIG support classes +as well as the class files built from the MIG-generated code +for the Mach and Hurd interface definition files, +would be provided as single `hurd-java` package for +Debian GNU/Hurd. +This package would be separate from both Hurd and Mach, +so as not to impose unreasonable build dependencies on them. + +I expect I would be able to act as its maintainer in the foreseeable future, +either as an individual or as a part of the Hurd team. +Hopefully, +my code would be claimed by the Hurd project as their own, +and consequently the modifications to MIG +(which would at least conceptually depend on the Mach Java package) +could be integrated upstream. + +Since by design, +the Java code would use only a small number of stable interfaces, +it would not be subject to excessive amounts of bitrot. +Consequently, +maintenance would primarily consist in +fixing bugs as they are reported, +and adding new features as they are requested. +A large number of such requests +would mean the package is useful, +so I expect that the overall amount of work +would be correlated with the willingness of more people +to help with maintenance +should I become overwhelmed or get hit by a bus. + + +## Timeline + +The dates listed are deadlines for the associated tasks. + + * *Community bonding period.* + Discuss, refine and complete the design of the Java bindings + (in particular the MIG and "libports" parts) + * *May 23.* + Coding starts. + * *May 30.* + Finish implementing pthread signal semantics. + * *June 5.* + Port OpenJDK + * *June 12.* + Fix the remaining problems with GCJ and/or OpenJDK, + possibly port Eclipse or other big Java packages. + * *June 19.* + Create the bindings for Mach. + * *June 26.* + Work on some kind of basic Java libports + to handle receive rights. + * *July 3.* + Test, write some documentation and examples. + * *July 17 (two weeks).* + Add the Java target to MIG. + * *July 24.* + Test, write some documentation and examples. + * *August 7 (two weeks).* + Implement a modular libfoofs to help with translator development. + Try to write a basic but non-trivial translator + to evaluate the performance and ease of use of the result, + rectify any rough edges this would uncover. + * *August 22. (last two weeks)* + Polish the code and packaging, + finish writing the documentation. + + +## Conclusion + +This project is arguably ambitious. +However, I have been thinking about it for some time now +and I'm confident I would be able to accomplish most of it. + +In the event multiple language bindings projects +would be accepted, +some work could probably be done in common. +In particular, +[ArneBab](http://www.bddebian.com/~hurd-web/community/weblogs/ArneBab/2011-04-06-application-pyhurd/) +seems to favor a low-level approach for his Python bindings as I do for Java, +and I would be happy to discuss API design and coordinate MIG changes with him. +I would also have an extra month after the end of the GSoC period +before I go back to school, +which I would be able to use to finish the project +if there is some remaining work. +(Last year's rewrite of procfs was done during this period.) + +As for the project's benefits, +I believe that good support for Java +is a must-have for the Hurd. +Java bindings would also further the Hurd's agenda +of user freedom by extending this freedom to more people: +I expect the set of developers +who would be able to write Java code against a well-written libfoofs +is much larger than +those who master the intricacies of low-level systems C programming. +From a more strategic point of view, +this would also help recruit new contributors +by providing an easier path to learning the inner workings of the Hurd. + +Further developments +which would build on the results of this project +include my planned [[experiment with Joe-E|objcap]] +(which I would possibly take on as a university project next year). +Another possibility would be to reimplement some parts +of the Java standard library +directly in terms of the Hurd interfaces +instead of using the POSIX ones through glibc. +This would possibly improve the performance +of some Java applications (though probably not by much), +and would otherwise be a good project +for someone trying to get acquainted with Hurd. + +Overall, I believe this project would be fun, interesting and useful. +I hope that you will share this sentiment +and give me the opportunity to spend another summer working on Hurd. + diff --git a/user/jkoenig/java/discussion.mdwn b/user/jkoenig/java/discussion.mdwn new file mode 100644 index 00000000..0131d8d5 --- /dev/null +++ b/user/jkoenig/java/discussion.mdwn @@ -0,0 +1,180 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +Some [[tschwinge]] comments regarding your proposal. Which is very good, if I +may say so again! :-) + +Of course, everyone is invited to contribute here! + +I want to give the following methodology a try, instead of only having +email/IRC discussions -- for the latter are again and again showing a tendency +to be dumped and deposited into their respective archives, and be forgotten +there. Of course, email/IRC discussions have their usefulness too, so we're +not going to replace them totally. For example, for conducting discussions +with a bunch of people (who may not even be following these pages here), email +(or, as applicable, the even more interactive IRC) will still be the medium of +choice. (And then, the executive summary should be posted here, or +incorporated into your proposal.) + +Also, if you disagree with this suggested procedure right away, or at some +later point begin to feel that this thing doesn't work out, or simply takes too +much time (I don't think so: writing emails takes time, too), just say so, and +we can reconsider. + +Of course, as this wiki is a passive medium rather than an active one as IRC +and email are, it is fine to send notices like: *I have updated the wiki page, +please have a look*. + +One idea is that your proposal evolves alongside with the ongoing work, and +represents (in more or less detail) what has been done and what will be done. +Also, we can hopefully use parts of it for documentation purposes, or as +recipes for similar work (enabling other programming languages on the Hurd, for +example). + +For this, I suggest the following procedure: as applicable, you can either +address any comments in here (for example, if they're wrong :-), or if they +require further discussion; think: *email discussion*), or you can address them +directly in your propoal and remove the comments from here at the same time +(think: *bug fix*). + +Generally, you can assume that for things I didn't comment on (within some +reasonable timeframe/upon asking me again) that I'm fine with them. Otherwise, +I might say: *I don't like this as is, but I'll need more time to think about +it.* + +There is also a possibility that parts of your proposal will be split off; in +cases where we think they're valuable to follow, but not at this time. (As you +know, your proposal is not really a trivial one, so it may just be too much for +one person's summer.) Such bits could be moved to [[open_issues]] pages, +either new ones or existing ones, as applicable. + + +# POSIX Threads Signal Semantics + + * Great! [[tschwinge]] had a brief look, and should have a deeper one. + + * If [[jkoenig]] thinks it's mature enough: should ask Samuel to test this + (that is, only the refactoring patches for starters?) on the buildds. + + * Then: should ask Roland to review. + + * Documentations bits should probably be moved to [[glibc/signal]]. + + +## libthreads (cthreads) Integration + + * [[tschwinge]] suggests to leave them as-is? + + +## [[libpthread]] integration + + * To be done. + + +# Java + + * [[tschwinge]] has to read about RMI and CORBA. + + +# Joe-E + + * For later. + + +# GCJ + + * [[tschwinge]] has the feeling that Java in GCC (that is, GCJ) is mostly + dead? (True?) + + * Thus perhaps not too much effort should be spent with it. + + If the POSIX threads signal semantics makes it going, then great, otherwise + we should get a feeling what else is missing. + + +# OpenJDK + + * All in all, [[tschwinge]] has the feeling that a working OpenJDK will be + more useful/powerful than GCJ. + + * We need to get a feeling how difficult such an OS port will be. + + * [[jkoenig]] suggests OpenJDK 6 -- should we directly go for version 7 + instead? + + * What are the differences (regarding the OS port) between the two + versions? Or this there something even more recent to be worked upon, + for new OS ports? + + * Perhaps the different versions' OS port specific stuff is not at + all very different, so that both v6 and v7 could be done? + + * They seem to have a rather heavy-weight process for such projects: confer + , + for example. Do we need this, too? + + +# Eclipse + +OK for testing -- but I'd very much hope that it *just works* as soon as we +provide the required Java platform. + + +# Java Bindings + + +## Design Principles + + * Generally ack. + + +### MIG + + * Hacking [[microkernel/mach/MIG]] shouldn't be too difficult. + + * (Unless you want to make MIG's own code (that is, not the generated + code, but MIG itself) look a bit more nice, too.) ;-) + + * There are also alternatives to MIG. If there is interest, the following + could be considered: + + * FLICK ([[!GNU_Savannah_task 5723]]). [[tschwinge]] has no idea yet if + there would be any benefits over MIG, like better modularity (for the + backends)? If we feel like it, we could spend a little bit of time on + this. + + * For [[microkernel/Viengoos]], Neal has written a RPC stub generator + entirely in C Preprocessor macros. While this is obviously not + directly applicable, perhaps we can get some ideas from it. + + * Anything else that would be worth having a look at? (What are other + microkernels using?) + + +### `mach_msg` + + * Seems like the right approach to [[tschwinge]], but hasn't digested all the + pecularities yet. Will definitely need more time. + + +# GSoC Site Discussion + + * Discussion items from + + should be copied here: + + * technical bits (obviously); + + * also the *why do we want Java bindings* reasoning; + + * CLISP findings should also be documented somewhere permanently. + + * We should probaby open up a *languages for Hurd* section on the web + pages ([[!taglink open_issue_documentation]]). -- cgit v1.2.3 From 254a5b6dccdc28d6b04b8864135cf1ada850dea9 Mon Sep 17 00:00:00 2001 From: Jeremie Koenig Date: Tue, 24 May 2011 17:57:22 +0200 Subject: user/jkoenig: link to the new location of the java proposal --- user/jkoenig/gsoc2011_proposal.mdwn | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 user/jkoenig/gsoc2011_proposal.mdwn (limited to 'user/jkoenig') diff --git a/user/jkoenig/gsoc2011_proposal.mdwn b/user/jkoenig/gsoc2011_proposal.mdwn new file mode 100644 index 00000000..ccdde659 --- /dev/null +++ b/user/jkoenig/gsoc2011_proposal.mdwn @@ -0,0 +1,14 @@ +[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +## Hurd Debian-Installer + +This page has moved [[here|java]]. + -- cgit v1.2.3 From 3f23831b22f83329908d36ea11a921113ffc9a6d Mon Sep 17 00:00:00 2001 From: "http://jeremie.koenig.myopenid.com/" Date: Sat, 28 May 2011 12:15:55 +0000 Subject: remove erroneous title --- user/jkoenig/gsoc2011_proposal.mdwn | 2 -- 1 file changed, 2 deletions(-) (limited to 'user/jkoenig') diff --git a/user/jkoenig/gsoc2011_proposal.mdwn b/user/jkoenig/gsoc2011_proposal.mdwn index ccdde659..9840f14f 100644 --- a/user/jkoenig/gsoc2011_proposal.mdwn +++ b/user/jkoenig/gsoc2011_proposal.mdwn @@ -8,7 +8,5 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] -## Hurd Debian-Installer - This page has moved [[here|java]]. -- cgit v1.2.3 From e24c06d392601c1c3a5ead08aea237a4bfa79d03 Mon Sep 17 00:00:00 2001 From: Jeremie Koenig Date: Wed, 15 Jun 2011 16:27:00 +0200 Subject: user/jkoenig: java status report --- user/jkoenig/java.mdwn | 738 ++++++---------------------------------- user/jkoenig/java/proposal.mdwn | 628 ++++++++++++++++++++++++++++++++++ 2 files changed, 740 insertions(+), 626 deletions(-) create mode 100644 user/jkoenig/java/proposal.mdwn (limited to 'user/jkoenig') diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn index 4052f455..90f51028 100644 --- a/user/jkoenig/java.mdwn +++ b/user/jkoenig/java.mdwn @@ -1,628 +1,114 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +# Improve Java on Hurd (GSoC 2011) + + +## Description + +The project consists in improving Java support on Hurd. +This includes porting OpenJDK, +creating low-level Java bindings for Mach and Hurd, +as well as creating Java libraries to help with translator development. + +For details, see my original [[proposal]]. + + +## Current status + + +### Apt repository + +Modified Debian packages are available in this repository: + + deb http://jk.fr.eu.org/debian experimental/ + deb-src http://jk.fr.eu.org/debian experimental/ + + +### Glibc signal code improvements + +I have submitted +[preliminary patches](http://lists.gnu.org/archive/html/bug-hurd/2011-05/msg00182.html) +for global signal dispositions, +which I'm currently testing. +I have since fixed a few thinks and implemented `SA_SIGINFO` +(which is required by OpenJDK.) +My latest code is available on +[github](http://github.com/jeremie-koenig/glibc/commits/master-beware-rebase), +and modified Debian packages +are available in my apt repository. + +One question is how the new symbols introduced by my patches +should be handled. +Weak symbols turned out to be impractical, +so I'm currently considering using a Debian-specific +symbol version in the interim period (`GLIBC_2.13_DEBIAN_7` so far). +The ultimate symbol version to be used will depend on +the time at which the patches get integrated upstream, +at which point we will alias the interim version +to the new one in debian packages. + +I have modified libc0.3 to include a `deb-symbols(5)` file +so that we get an accurate libc dependency in `hurd` and other packages +when the symbols in question are pulled in. + +Another issue which came up with OpenJDK is the expansion +by the dynamic linker of `$ORIGIN` in the `RPATH` header, +see below. + +#### Plans + +I will submit revised series for review later this week, +as well as matching Debian patches. +I expect only the last patch (implement global dispositions) will change, +and new ones will be added on top of it. + + +### Port OpenJDK + +As suggested by [[tschwinge]], I have targeted OpenJDK 7 at first. +I don't expect it will be too hard to backport my patches to OpenJDK 6. +I have succeeded in building a working JIT-less ("zero") version, +although the dynamic linker issue must be worked around. +Porting Hotspot (the original just-in-time compiler of OpenJDK) +should not be too hard. +If that fails we can fall back on Shark +(a portable alternative JIT which uses LLVM). + +The dynamic linker issue is as follows. +An executable-specific search path can be provided in the ELF RPATH header. +RPATH directories can include the special string `$ORIGIN`, +which is to be expanded to the directory the executable was loaded from. +OpenJDK's `java` command uses this feature to locate +the right `libjli.so` at runtime. +However, +on Hurd this information is not available to the dynamic linker +and as a consequence RPATH components which include `$ORIGIN` +are silently discarded. + +This can be worked around by defining +the `LD_ORIGIN_PATH` environment variable. +(which have I used to build and test OpenJDK so far.) + +#### Plans + +I intend to fix the RPATH issue +by building on [[pochu]]'s `file_exec_file_name()` patches. + +I have succeeded in building a Hotspot-enabled `libjvm.so`, +although the current toolchain issues +have so far prevented me from testing it. + + +### Java bindings for Mach + +(just started.) -# Java for Hurd (and vice versa) - -Contact information: - - * Full name: Jérémie Koenig - * Email: jk@jk.fr.eu.org - * IRC: jkoenig on Freenode and OFTC - -## Introductions - -I am a first year M.Sc. student -in Computer Science at University of Strasbourg (France). -My interests include capability-based security, -programming languages and formal methods -(in particular, object-capability languages and proof-carrying code). - -### Proposal summary - -This project would consist in improving Java support on Hurd. -The first part would consist in -fixing bugs and porting Java-related packages. -The second part would consist in -creating low-level Java bindings for the Hurd interfaces, -as well as libraries to make translator development easier. - -### Previous involvement - -I started contributing to Hurd last summer, -during which I participated to Google Summer of Code -as a student for the Debian project. -I worked on porting Debian-Installer to Hurd. -This project was mostly a success, -although we still have to use a special mirror for installation -with a few modified packages -and tweaked priorities -to work around some uninstallable packages -with Priority: standard. - -Shortly afterwards, -I rewrote the procfs translator -to fix some issues with memory leaks, -make it more reliable, -and improve compatibility with Linux-based tools -such as `procps` or `htop`. - -Although I have not had as much time -as I would have liked to dedicate to the Hurd -since that time, -I have continued to maintain the mirror in question, -and I have started to work -on implementing POSIX threads signal semantics in glibc. - -### Project-related skills and interests - -I have used Java mostly for university assignments. -This includes non-trivial projects -using threads and distributed programming frameworks -such as Java RMI or CORBA. -I have also used it to experiment with -Google App Engine -(web applications) -and Google Web Toolkit -(a compiler from Java to Javascript which helps with AJAX code), -and I have some limited experience with JNI -(the Java Native Interface, to link Java with C code). - -My knowledge of the Hurd and Debian GNU/Hurd is reasonable, -as the Debian-Installer and procfs projects -gave me the opportunity to fiddle with many parts of the system. - -Initially, -I started working on this project because I wanted to use -[Joe-E](http://code.google.com/p/joe-e/) -(a subset of Java) -to investigate the potential -[[applications of object-capability languages|objcap]] -in a Hurd context. -I also believe that improving Java support on Hurd -would be an important milestone. - -### Organisational matters - -I am subscribed to bug-hurd@g.o and -I do have a permanent internet connexion. - -I would be able to attend the regular IRC meetings, -and otherwise communicate with my mentor -through any means they would prefer -(though I expect email and IRC would be the most practical). -Since I'm already familiar with the Hurd, -I don't expect I would require too much time from them. - -My exams end on May 20 so I would be able to start coding -right at the beginning of the GSoC period. -Next year's term would probably begin around September 15, -so that would not be an issue either. -I expect I would work around 40 hours per week, -and my waking hours would be flexible. - -I don't have any other plans for the summer -and would not make any if my project were to be accepted. - -Full disclosure: -I also submitted a proposal to the Jikes RVM project -(which is a research-oriented Java Virtual Machine, -itself written in Java) -for implementing a new garbage collector into the MMTk subsystem. - -## Improve Java support - -### Justification - -Java is a popular language and platform used by many desktop and web -applications (mostly on the server side). As a consequence, competitive Java -support is important for any general-purpose operating system. -Better Java support would also be a prerequisite -for the second part of my proposal. - -### Current situation - -Java is currently supported on Hurd with the GNU Java suite: - - * [GCJ](http://gcc.gnu.org/java/), - the GNU Compiler for Java, is part of GCC and can compile Java - source code to Java bytecode, and both source code and bytecode to - native code; - * libgcj is the implementation of the Java runtime which GCJ uses. - It is based on [GNU Classpath](http://www.gnu.org/software/classpath/). - It includes a bytecode interpreter which enables - Java applications compiled to native code to dynamically load and execute - Java bytecode from class files. - * The gij command is a wrapper around the above-mentioned virtual machine - functionality of libgcj and can be used as a replacement for the java - command. - -However, GCJ does not work flawlessly on Hurd.r -For instance, some parts of libgcj relies on -the POSIX threads signal semantics, which are not yet implemented. -In particular, this makes ant hang waiting for child processes, -which makes some packages fail to build on Hurd -(“ant” is the “make” of the Java world). - -### Tasks - - * **Finish implementing POSIX thread semantics** in glibc (high priority). - According to POSIX, signal dispositions should be global to a process, - while signal blocking masks should be thread-specific. Signals sent to the - process as a whole are to be delivered to any thread which does not block - them. By contrast, Hurd has per-thread signal dispositions and signals - sent to a process are delivered to the main thread only. I have been - working on refactoring the glibc signal code and implementing the POSIX - semantics as a per-thread option. However, due to lack of time I have not - yet been able to test and debug my code properly. Finishing this work - would be my first task. - * **Fix further problems with GCJ on Hurd** (high priority). While I’m not - aware of any other problems with GCJ at the moment, I suspect some might - turn up as I progress with the other tasks. Fixing these problems would - also be a high-priority task. - * **Port OpenJDK 6** (medium priority). While GCJ is fine, it is not yet - 100% complete. It is also slower than OpenJDK on architectures where a - just-in-time compiler is available. Porting OpenJDK would therefore - improve Java support on Hurd in scope and quality. Besides, it would also - be a good way to test GCJ, which is used for bootstrapping by the Debian - OpenJDK packages. Also note that OpenJDK 6 is now the default Java - Runtime Environment on all released Linux-based Debian architectures; - bringing Hurd in line with this would probably be a good thing. - * **Port Eclipse and other Java applications** (low priority). Eclipse is a - popular, state-of-the-art IDE and tool suite used for Java and other - languages. It is a dependency of the Joe-E verifier (see part 3 of this - proposal). Porting Eclipse would be a good opportunity to test GCJ and - OpenJDK. - -### Deliverables - - * The glibc pthreads patch and any other fixes on the Hurd side - would be submitted upstream - * Patches against Debian source packages - required to make them build on Hurd would be submitted - to the [Debian bug tracking system](http://bugs.debian.org/). - - -## Create Java bindings for the Hurd interfaces - -### Justification - -Java is used for many applications and often taught to -introduce object-oriented programming. The fact that Java is a -garbage-collected language makes it easier to use, especially for the less -experienced programmers. Besides, its object-oriented nature is a -natural fit for the capability-based design of Hurd. -The JVM is also used as a target for many other languages, -all of which would benefit from the access provided by these bindings. - -Advantages over other garbage-collected, object-oriented languages include -performance, type safety and the possibility to compile a Java translator to -native code and -[link it statically](http://gcc.gnu.org/wiki/Statically_linking_libgcj) -using GCJ, should anyone want to use a -translator written in Java for booting. -Note that Java is -[being](http://www.linuxjournal.com/article/8757) -[used](http://oss.readytalk.com/avian/) -in this manner for embedded development. -Since GCJ can take bytecode as its input, -this expect this possibility would apply to any JVM-based language. - -Java bindings would lower the bar for newcomers -to begin experimenting with what makes Hurd unique -without being faced right away with the complexity of -low-level systems programming. - -### Tasks summary - - * Implement Java bindings for Mach - * Implement a libports-like library for Java - * Modify MIG to output Java code - * Implement libfoofs-like Java libraries - -### Design principles - -The principles I would use to guide the design -of these Java bindings would be the following ones: - - * The system should be hooked into at a low level, - to ensure that Java is a "first class citizen" - as far as the access to the Hurd's interfaces is concerned. - * At the same time, the memory safety of Java should be maintained - and extended to Mach primitives such as port names and - out-of-line memory regions. - * Higher-level interfaces should be provided as well - in order to make translator development - as easy as possible. - * A minimum amount of JNI code (ie. C code) should be used. - Most of the system should be built using Java itself - on top of a few low-level primitives. - * Hurd objects would map to Java objects. - * Using the same interfaces, - objects corresponding to local ports would be accessed directly, - and remote objects would be accessed over IPC. - -One approach used previously to interface programming languages with the Hurd -has been to create bindings for helper libraries such as libtrivfs. Instead, -for Java I would like to take a lower-level approach by providing access to -Mach primitives and extending MIG to generate Java code from the interface -description files. - -This approach would be initially more involved, and would introduces several -issues related to overcoming the "impedance mismatch" between Java and Mach. -However, once an initial implementation is done it would be easier to maintain -in the long run and we would be able to provide Java bindings for a large -percentage of the Hurd’s interfaces. - -### Bindings for Mach system calls - -In this low-level approach, my intention is to enable Java code to use Mach -system calls (in particular, mach_msg) more or less directly. This would -ensure full access to the system from Java code, but it raises a number of -issues: - - * the Java code must be able to manipulate Mach-level entities, such as port - rights or page-aligned buffers mapped outside of the garbage-collected - heap (for out-of-line transfers); - * putting together IPC messages requires control of the low-level - representation of data. - -In order to address these concerns, classes would be encapsulating these -low-level entities so that they can be referenced through normal, safe objects -from standard Java code. Bindings for Mach system calls can then be provided -in terms of these classes. Their implementation would use C code through the -Java Native Interface (JNI). - -More specifically, this functionality would be provided by the `org.gnu.mach` -package, which would contain at least the following classes: - - * `MachPort` would encapsulate a `mach_port_t`. (Some of) its constructors - would act as an interface for the `mach_port_allocate()` system call. - `MachPort` objects would also be instantiated from other parts of the JNI - C code to represent port rights received through IPC. The `deallocate()` - method would call `mach_port_deallocate()` and replace the encapsulated - port name with `MACH_PORT_DEAD`. We would recommend that users call it - when a port is no longer used, but the finalizer would also deallocate the - port when the `MachPort` object is garbage collected. - * `Buffer` would represent a page-aligned buffer allocated outside of the - Java heap, to be transferred (or having been received) as out-of-line - memory. The JNI code would would provide methods to read and write data at - an arbitrary offset (but within bounds) and would use `vm_allocate()` and - `vm_deallocate()` in the same spirit as for `MachPort` objects. - * `Message` would allow Java code to put together Mach messages. The - constructor would allocate a `byte[]` member array of a given size. - Additional methods would be provided to fill in or query the information - in the message header and additional data items, including `MachPort` and - `Buffer` objects which would be translated to the corresponding port names - and out-of-line pointers. - A global map from port names to the corresponding `MachPort` object - would probably be needed to ensure that there is a one-to-one - correspondence. - * `Syscall` would provide static JNI methods for performing system calls not - covered by the above classes, such as `mach_msg()` or - `mach_thread_self()`. These methods would accept or return `MachPort`, - `Buffer` and `Message` objects when appropriate. The associated C code - would access the contents of such objects directly in order to perform the - required unsafe operations, such as constructing `MachPort` and `Buffer` - objects directly from port names and C pointers. - -Note that careful consideration should be given to the interfaces of these -classes to avoid “safety leaks” which would compromise the safety guarantees -provided by Java. Potential problematic scenarios include the following -examples: - - * It must not be possible to write an integer at some position in a - `Message` object, and to read it back as a `MachPort` or `Buffer` object, - since this would allow unsafe access to arbitrary memory addresses and - mach port names. - * Providing the `mach_task_self()` system call would also provide access to - arbitrary addresses and ports by using the `vm_*` family of RPC operations - with the returned `MachPort` object. This means that the relevant task - operations should be provided by the `Syscall` class instead. - -Finally, access should be provided to the initial ports and file descriptors -in `_hurd_ports` and provided by the `getdport()` function, -for instance through static methods such as -`getCRDir()`, `getCWDir()`, `getProc()`, ... in a dedicated class such as -`org.gnu.hurd.InitPorts`. - -A realistic example of code based on such interfaces would be: - - import org.gnu.mach.MsgType; - import org.gnu.mach.MachPort; - import org.gnu.mach.Buffer; - import org.gnu.mach.Message; - import org.gnu.mach.Syscall; - import org.gnu.hurd.InitPorts; - - public class Hello - { - public static main(String argv[]) - /* Parent class for all Mach-related exceptions */ - throws org.gnu.mach.MachException - { - /* Allocate a reply port */ - MachPort reply = new MachPort(); - - /* Allocate an out-of-line buffer */ - Buffer data = new Buffer(MsgType.CHAR, 13); - data.writeString(0, "Hello, World!"); - - /* Craft an io_write message */ - Message msg = new Message(1024); - msg.setRemotePort(InitPorts.getdport(1)); - msg.setLocalPort(reply, Message.Type.MAKE_SEND_ONCE); - msg.setId(21000); - msg.addBuffer(data); - - /* Make the call, MACH_MSG_SEND | MACH_MSG_RECEIVE */ - Syscall.machMsg(msg, true, true, reply); - - /* Extract the returned value */ - msg.assertId(21100); - int retCode = msg.readInt(0); - int amount = msg.readInt(1); - } - } - -Should this paradigm prove insufficient, -more ideas could be borrowed from the -[`org.vmmagic`](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.151.5253&rep=rep1&type=pdf) -package used by [Jikes RVM](http://jikesrvm.org/), -a research Java virtual machine itself written in Java. - -### Generating Java stubs with MIG - -Once the basic machinery is in place to interface with Mach, Java programs -have more or less equal access to the system functionality without resorting -to more JNI code. However, as illustrated above, this access is far from -convenient. - -As a solution I would modify MIG to add the option to output Java code. MIG -would emit a Java interface, a client class able to implement the interface -given a Mach port send right, an a server class which would be able to handle -incoming messages. The class diagram below, although it is by no means -complete or exempt of any problem, illustrates the general idea: - -[[gsoc2011_classes.png]] - -This structure is somewhat reminiscent of -[Java RMI](http://en.wikipedia.org/wiki/Java_remote_method_invocation) -or similar systems, -which aim to provide more or less transparent access to remote objects. -The exact way the Java code would be generated still needs to be determined, -but basically: - - * An interface, corresponding to the header files generated by MIG, would - enumerate the operations listed in a given .defs files. Method names would - be transformed to adhere to Java conventions (for instance, - `some_random_identifier` would become `someRandomIdentifier`). - * A user class, corresponding to the `*User.c` files, - would implement this interface by doing RPC over a given MachPort object. - * A server class, corresponding to `*Server.c`, would be able to handle - incoming messages using a user-provided implementation of the interface. - (Possibly, a skeleton class providing methods which would raise - `NotImplementedException`s would be provided as well. - Users would derive from this class and override the relevant methods. - This would allow them not to implement some operations, - and would avoid pre-existing code from breaking when new operations are - introduced.) - -In order to help with the implementation of servers, some kind of library -would be needed to associate Mach receive rights with server objects and to -handle incoming messages on dedicated threads, in the spirit of libports. -This would probably require support for port sets at the level of the Mach -primitives described in the previous section. - -When possible, operations involving the transmission of send rights -of some kind would be expressed in terms of the MIG-generated interfaces -instead of `MachPort` objects. -Upon reception of a send right, -a `FooUser` object would be created -and associated with the corresponding `MachPort` object. -If the received send right corresponds to a local port -to which a server object has been associated, -this object would be used instead. -This way, -subsequent operations on the received send right -would be handled as direct method calls -instead of going through RPC mechanisms. - -Some issues will still need to be solved regarding how MIG will convert -interface description files to Java interfaces. For instance: - - * `.defs` files are not explicitly associated with a type. For instance in - the example above, MIG would have to somehow infer that io_t corresponds - to `this` in the `Io` interface. - * More generally, a correspondence between MIG and Java types would have - to be determined. Ideally this would be automated and not hardcoded - too much. - * Initially, reply port parameters would be ignored. However they may be - needed for some applications. - -So the details would need to be flushed out during the community bonding -period and as the implementation progresses. However I’m confident that a -satisfactory solution can be designed. - -Using these new features, the example above could be rewritten as: - - import org.gnu.hurd.InitPorts; - import org.gnu.hurd.Io; - import org.gnu.hurd.IoUser; - - class Hello { - static void main(String argv[]) throws ... - { - Io stdout = new IoUser(InitPorts.getdport(1)); - String hello = “Hello, World!\n”; - - int amount = stdout.write(hello.getBytes(), -1); - - /* (A retCode corresponding to an error - would be signalled as an exception.) */ - } - } - -An example of server implementation would be: - - import org.gnu.hurd.Io; - import java.util.Arrays; - - class HelloIo implements Io { - final byte[] contents = “Hello, World!\n”.getBytes(); - - int write(byte[] data, int offset) { - return SOME_ERROR_CODE; - } - - byte[] read(int offset, int amount) { - return Arrays.copyOfRange(contents, offset, - offset + amount - 1); - } - - /* ... */ - } - -A new server object could then be created with `new IoServer(new HelloIo())`, -and associated with some receive right at the level of the ports management -library. - -### Base classes for common types of translators - -Once MIG can target Java code, and a libports equivalent is available, -creating new translators in Java would be greatly facilitated. However, -we would probably want to introduce basic implementations of file system -translators in the spirit of libtrivfs or libnetfs. They could take the form -of base classes implementing the relevant MIG-generated interfaces which -would then be derived by users, -or could define a simpler interface -which would then be used by adapter classes -to implement the required ones. - -I would draw inspiration from libtrivfs and libnetfs -to design and implement similar solutions for Java. - -### Deliverables - - * A hurd-java package would contain the Java code developed - in the context of this project. - * The Java code would be documented using javadoc - and a tutorial for writing translators would be written as well. - * Modifications to MIG would be submitted upstream, - or a patched MIG package would be made available. - -The Java libraries resulting from this work, -including any MIG support classes -as well as the class files built from the MIG-generated code -for the Mach and Hurd interface definition files, -would be provided as single `hurd-java` package for -Debian GNU/Hurd. -This package would be separate from both Hurd and Mach, -so as not to impose unreasonable build dependencies on them. - -I expect I would be able to act as its maintainer in the foreseeable future, -either as an individual or as a part of the Hurd team. -Hopefully, -my code would be claimed by the Hurd project as their own, -and consequently the modifications to MIG -(which would at least conceptually depend on the Mach Java package) -could be integrated upstream. - -Since by design, -the Java code would use only a small number of stable interfaces, -it would not be subject to excessive amounts of bitrot. -Consequently, -maintenance would primarily consist in -fixing bugs as they are reported, -and adding new features as they are requested. -A large number of such requests -would mean the package is useful, -so I expect that the overall amount of work -would be correlated with the willingness of more people -to help with maintenance -should I become overwhelmed or get hit by a bus. - - -## Timeline - -The dates listed are deadlines for the associated tasks. - - * *Community bonding period.* - Discuss, refine and complete the design of the Java bindings - (in particular the MIG and "libports" parts) - * *May 23.* - Coding starts. - * *May 30.* - Finish implementing pthread signal semantics. - * *June 5.* - Port OpenJDK - * *June 12.* - Fix the remaining problems with GCJ and/or OpenJDK, - possibly port Eclipse or other big Java packages. - * *June 19.* - Create the bindings for Mach. - * *June 26.* - Work on some kind of basic Java libports - to handle receive rights. - * *July 3.* - Test, write some documentation and examples. - * *July 17 (two weeks).* - Add the Java target to MIG. - * *July 24.* - Test, write some documentation and examples. - * *August 7 (two weeks).* - Implement a modular libfoofs to help with translator development. - Try to write a basic but non-trivial translator - to evaluate the performance and ease of use of the result, - rectify any rough edges this would uncover. - * *August 22. (last two weeks)* - Polish the code and packaging, - finish writing the documentation. - - -## Conclusion - -This project is arguably ambitious. -However, I have been thinking about it for some time now -and I'm confident I would be able to accomplish most of it. - -In the event multiple language bindings projects -would be accepted, -some work could probably be done in common. -In particular, -[ArneBab](http://www.bddebian.com/~hurd-web/community/weblogs/ArneBab/2011-04-06-application-pyhurd/) -seems to favor a low-level approach for his Python bindings as I do for Java, -and I would be happy to discuss API design and coordinate MIG changes with him. -I would also have an extra month after the end of the GSoC period -before I go back to school, -which I would be able to use to finish the project -if there is some remaining work. -(Last year's rewrite of procfs was done during this period.) - -As for the project's benefits, -I believe that good support for Java -is a must-have for the Hurd. -Java bindings would also further the Hurd's agenda -of user freedom by extending this freedom to more people: -I expect the set of developers -who would be able to write Java code against a well-written libfoofs -is much larger than -those who master the intricacies of low-level systems C programming. -From a more strategic point of view, -this would also help recruit new contributors -by providing an easier path to learning the inner workings of the Hurd. - -Further developments -which would build on the results of this project -include my planned [[experiment with Joe-E|objcap]] -(which I would possibly take on as a university project next year). -Another possibility would be to reimplement some parts -of the Java standard library -directly in terms of the Hurd interfaces -instead of using the POSIX ones through glibc. -This would possibly improve the performance -of some Java applications (though probably not by much), -and would otherwise be a good project -for someone trying to get acquainted with Hurd. - -Overall, I believe this project would be fun, interesting and useful. -I hope that you will share this sentiment -and give me the opportunity to spend another summer working on Hurd. diff --git a/user/jkoenig/java/proposal.mdwn b/user/jkoenig/java/proposal.mdwn new file mode 100644 index 00000000..4052f455 --- /dev/null +++ b/user/jkoenig/java/proposal.mdwn @@ -0,0 +1,628 @@ + +# Java for Hurd (and vice versa) + +Contact information: + + * Full name: Jérémie Koenig + * Email: jk@jk.fr.eu.org + * IRC: jkoenig on Freenode and OFTC + +## Introductions + +I am a first year M.Sc. student +in Computer Science at University of Strasbourg (France). +My interests include capability-based security, +programming languages and formal methods +(in particular, object-capability languages and proof-carrying code). + +### Proposal summary + +This project would consist in improving Java support on Hurd. +The first part would consist in +fixing bugs and porting Java-related packages. +The second part would consist in +creating low-level Java bindings for the Hurd interfaces, +as well as libraries to make translator development easier. + +### Previous involvement + +I started contributing to Hurd last summer, +during which I participated to Google Summer of Code +as a student for the Debian project. +I worked on porting Debian-Installer to Hurd. +This project was mostly a success, +although we still have to use a special mirror for installation +with a few modified packages +and tweaked priorities +to work around some uninstallable packages +with Priority: standard. + +Shortly afterwards, +I rewrote the procfs translator +to fix some issues with memory leaks, +make it more reliable, +and improve compatibility with Linux-based tools +such as `procps` or `htop`. + +Although I have not had as much time +as I would have liked to dedicate to the Hurd +since that time, +I have continued to maintain the mirror in question, +and I have started to work +on implementing POSIX threads signal semantics in glibc. + +### Project-related skills and interests + +I have used Java mostly for university assignments. +This includes non-trivial projects +using threads and distributed programming frameworks +such as Java RMI or CORBA. +I have also used it to experiment with +Google App Engine +(web applications) +and Google Web Toolkit +(a compiler from Java to Javascript which helps with AJAX code), +and I have some limited experience with JNI +(the Java Native Interface, to link Java with C code). + +My knowledge of the Hurd and Debian GNU/Hurd is reasonable, +as the Debian-Installer and procfs projects +gave me the opportunity to fiddle with many parts of the system. + +Initially, +I started working on this project because I wanted to use +[Joe-E](http://code.google.com/p/joe-e/) +(a subset of Java) +to investigate the potential +[[applications of object-capability languages|objcap]] +in a Hurd context. +I also believe that improving Java support on Hurd +would be an important milestone. + +### Organisational matters + +I am subscribed to bug-hurd@g.o and +I do have a permanent internet connexion. + +I would be able to attend the regular IRC meetings, +and otherwise communicate with my mentor +through any means they would prefer +(though I expect email and IRC would be the most practical). +Since I'm already familiar with the Hurd, +I don't expect I would require too much time from them. + +My exams end on May 20 so I would be able to start coding +right at the beginning of the GSoC period. +Next year's term would probably begin around September 15, +so that would not be an issue either. +I expect I would work around 40 hours per week, +and my waking hours would be flexible. + +I don't have any other plans for the summer +and would not make any if my project were to be accepted. + +Full disclosure: +I also submitted a proposal to the Jikes RVM project +(which is a research-oriented Java Virtual Machine, +itself written in Java) +for implementing a new garbage collector into the MMTk subsystem. + +## Improve Java support + +### Justification + +Java is a popular language and platform used by many desktop and web +applications (mostly on the server side). As a consequence, competitive Java +support is important for any general-purpose operating system. +Better Java support would also be a prerequisite +for the second part of my proposal. + +### Current situation + +Java is currently supported on Hurd with the GNU Java suite: + + * [GCJ](http://gcc.gnu.org/java/), + the GNU Compiler for Java, is part of GCC and can compile Java + source code to Java bytecode, and both source code and bytecode to + native code; + * libgcj is the implementation of the Java runtime which GCJ uses. + It is based on [GNU Classpath](http://www.gnu.org/software/classpath/). + It includes a bytecode interpreter which enables + Java applications compiled to native code to dynamically load and execute + Java bytecode from class files. + * The gij command is a wrapper around the above-mentioned virtual machine + functionality of libgcj and can be used as a replacement for the java + command. + +However, GCJ does not work flawlessly on Hurd.r +For instance, some parts of libgcj relies on +the POSIX threads signal semantics, which are not yet implemented. +In particular, this makes ant hang waiting for child processes, +which makes some packages fail to build on Hurd +(“ant” is the “make” of the Java world). + +### Tasks + + * **Finish implementing POSIX thread semantics** in glibc (high priority). + According to POSIX, signal dispositions should be global to a process, + while signal blocking masks should be thread-specific. Signals sent to the + process as a whole are to be delivered to any thread which does not block + them. By contrast, Hurd has per-thread signal dispositions and signals + sent to a process are delivered to the main thread only. I have been + working on refactoring the glibc signal code and implementing the POSIX + semantics as a per-thread option. However, due to lack of time I have not + yet been able to test and debug my code properly. Finishing this work + would be my first task. + * **Fix further problems with GCJ on Hurd** (high priority). While I’m not + aware of any other problems with GCJ at the moment, I suspect some might + turn up as I progress with the other tasks. Fixing these problems would + also be a high-priority task. + * **Port OpenJDK 6** (medium priority). While GCJ is fine, it is not yet + 100% complete. It is also slower than OpenJDK on architectures where a + just-in-time compiler is available. Porting OpenJDK would therefore + improve Java support on Hurd in scope and quality. Besides, it would also + be a good way to test GCJ, which is used for bootstrapping by the Debian + OpenJDK packages. Also note that OpenJDK 6 is now the default Java + Runtime Environment on all released Linux-based Debian architectures; + bringing Hurd in line with this would probably be a good thing. + * **Port Eclipse and other Java applications** (low priority). Eclipse is a + popular, state-of-the-art IDE and tool suite used for Java and other + languages. It is a dependency of the Joe-E verifier (see part 3 of this + proposal). Porting Eclipse would be a good opportunity to test GCJ and + OpenJDK. + +### Deliverables + + * The glibc pthreads patch and any other fixes on the Hurd side + would be submitted upstream + * Patches against Debian source packages + required to make them build on Hurd would be submitted + to the [Debian bug tracking system](http://bugs.debian.org/). + + +## Create Java bindings for the Hurd interfaces + +### Justification + +Java is used for many applications and often taught to +introduce object-oriented programming. The fact that Java is a +garbage-collected language makes it easier to use, especially for the less +experienced programmers. Besides, its object-oriented nature is a +natural fit for the capability-based design of Hurd. +The JVM is also used as a target for many other languages, +all of which would benefit from the access provided by these bindings. + +Advantages over other garbage-collected, object-oriented languages include +performance, type safety and the possibility to compile a Java translator to +native code and +[link it statically](http://gcc.gnu.org/wiki/Statically_linking_libgcj) +using GCJ, should anyone want to use a +translator written in Java for booting. +Note that Java is +[being](http://www.linuxjournal.com/article/8757) +[used](http://oss.readytalk.com/avian/) +in this manner for embedded development. +Since GCJ can take bytecode as its input, +this expect this possibility would apply to any JVM-based language. + +Java bindings would lower the bar for newcomers +to begin experimenting with what makes Hurd unique +without being faced right away with the complexity of +low-level systems programming. + +### Tasks summary + + * Implement Java bindings for Mach + * Implement a libports-like library for Java + * Modify MIG to output Java code + * Implement libfoofs-like Java libraries + +### Design principles + +The principles I would use to guide the design +of these Java bindings would be the following ones: + + * The system should be hooked into at a low level, + to ensure that Java is a "first class citizen" + as far as the access to the Hurd's interfaces is concerned. + * At the same time, the memory safety of Java should be maintained + and extended to Mach primitives such as port names and + out-of-line memory regions. + * Higher-level interfaces should be provided as well + in order to make translator development + as easy as possible. + * A minimum amount of JNI code (ie. C code) should be used. + Most of the system should be built using Java itself + on top of a few low-level primitives. + * Hurd objects would map to Java objects. + * Using the same interfaces, + objects corresponding to local ports would be accessed directly, + and remote objects would be accessed over IPC. + +One approach used previously to interface programming languages with the Hurd +has been to create bindings for helper libraries such as libtrivfs. Instead, +for Java I would like to take a lower-level approach by providing access to +Mach primitives and extending MIG to generate Java code from the interface +description files. + +This approach would be initially more involved, and would introduces several +issues related to overcoming the "impedance mismatch" between Java and Mach. +However, once an initial implementation is done it would be easier to maintain +in the long run and we would be able to provide Java bindings for a large +percentage of the Hurd’s interfaces. + +### Bindings for Mach system calls + +In this low-level approach, my intention is to enable Java code to use Mach +system calls (in particular, mach_msg) more or less directly. This would +ensure full access to the system from Java code, but it raises a number of +issues: + + * the Java code must be able to manipulate Mach-level entities, such as port + rights or page-aligned buffers mapped outside of the garbage-collected + heap (for out-of-line transfers); + * putting together IPC messages requires control of the low-level + representation of data. + +In order to address these concerns, classes would be encapsulating these +low-level entities so that they can be referenced through normal, safe objects +from standard Java code. Bindings for Mach system calls can then be provided +in terms of these classes. Their implementation would use C code through the +Java Native Interface (JNI). + +More specifically, this functionality would be provided by the `org.gnu.mach` +package, which would contain at least the following classes: + + * `MachPort` would encapsulate a `mach_port_t`. (Some of) its constructors + would act as an interface for the `mach_port_allocate()` system call. + `MachPort` objects would also be instantiated from other parts of the JNI + C code to represent port rights received through IPC. The `deallocate()` + method would call `mach_port_deallocate()` and replace the encapsulated + port name with `MACH_PORT_DEAD`. We would recommend that users call it + when a port is no longer used, but the finalizer would also deallocate the + port when the `MachPort` object is garbage collected. + * `Buffer` would represent a page-aligned buffer allocated outside of the + Java heap, to be transferred (or having been received) as out-of-line + memory. The JNI code would would provide methods to read and write data at + an arbitrary offset (but within bounds) and would use `vm_allocate()` and + `vm_deallocate()` in the same spirit as for `MachPort` objects. + * `Message` would allow Java code to put together Mach messages. The + constructor would allocate a `byte[]` member array of a given size. + Additional methods would be provided to fill in or query the information + in the message header and additional data items, including `MachPort` and + `Buffer` objects which would be translated to the corresponding port names + and out-of-line pointers. + A global map from port names to the corresponding `MachPort` object + would probably be needed to ensure that there is a one-to-one + correspondence. + * `Syscall` would provide static JNI methods for performing system calls not + covered by the above classes, such as `mach_msg()` or + `mach_thread_self()`. These methods would accept or return `MachPort`, + `Buffer` and `Message` objects when appropriate. The associated C code + would access the contents of such objects directly in order to perform the + required unsafe operations, such as constructing `MachPort` and `Buffer` + objects directly from port names and C pointers. + +Note that careful consideration should be given to the interfaces of these +classes to avoid “safety leaks” which would compromise the safety guarantees +provided by Java. Potential problematic scenarios include the following +examples: + + * It must not be possible to write an integer at some position in a + `Message` object, and to read it back as a `MachPort` or `Buffer` object, + since this would allow unsafe access to arbitrary memory addresses and + mach port names. + * Providing the `mach_task_self()` system call would also provide access to + arbitrary addresses and ports by using the `vm_*` family of RPC operations + with the returned `MachPort` object. This means that the relevant task + operations should be provided by the `Syscall` class instead. + +Finally, access should be provided to the initial ports and file descriptors +in `_hurd_ports` and provided by the `getdport()` function, +for instance through static methods such as +`getCRDir()`, `getCWDir()`, `getProc()`, ... in a dedicated class such as +`org.gnu.hurd.InitPorts`. + +A realistic example of code based on such interfaces would be: + + import org.gnu.mach.MsgType; + import org.gnu.mach.MachPort; + import org.gnu.mach.Buffer; + import org.gnu.mach.Message; + import org.gnu.mach.Syscall; + import org.gnu.hurd.InitPorts; + + public class Hello + { + public static main(String argv[]) + /* Parent class for all Mach-related exceptions */ + throws org.gnu.mach.MachException + { + /* Allocate a reply port */ + MachPort reply = new MachPort(); + + /* Allocate an out-of-line buffer */ + Buffer data = new Buffer(MsgType.CHAR, 13); + data.writeString(0, "Hello, World!"); + + /* Craft an io_write message */ + Message msg = new Message(1024); + msg.setRemotePort(InitPorts.getdport(1)); + msg.setLocalPort(reply, Message.Type.MAKE_SEND_ONCE); + msg.setId(21000); + msg.addBuffer(data); + + /* Make the call, MACH_MSG_SEND | MACH_MSG_RECEIVE */ + Syscall.machMsg(msg, true, true, reply); + + /* Extract the returned value */ + msg.assertId(21100); + int retCode = msg.readInt(0); + int amount = msg.readInt(1); + } + } + +Should this paradigm prove insufficient, +more ideas could be borrowed from the +[`org.vmmagic`](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.151.5253&rep=rep1&type=pdf) +package used by [Jikes RVM](http://jikesrvm.org/), +a research Java virtual machine itself written in Java. + +### Generating Java stubs with MIG + +Once the basic machinery is in place to interface with Mach, Java programs +have more or less equal access to the system functionality without resorting +to more JNI code. However, as illustrated above, this access is far from +convenient. + +As a solution I would modify MIG to add the option to output Java code. MIG +would emit a Java interface, a client class able to implement the interface +given a Mach port send right, an a server class which would be able to handle +incoming messages. The class diagram below, although it is by no means +complete or exempt of any problem, illustrates the general idea: + +[[gsoc2011_classes.png]] + +This structure is somewhat reminiscent of +[Java RMI](http://en.wikipedia.org/wiki/Java_remote_method_invocation) +or similar systems, +which aim to provide more or less transparent access to remote objects. +The exact way the Java code would be generated still needs to be determined, +but basically: + + * An interface, corresponding to the header files generated by MIG, would + enumerate the operations listed in a given .defs files. Method names would + be transformed to adhere to Java conventions (for instance, + `some_random_identifier` would become `someRandomIdentifier`). + * A user class, corresponding to the `*User.c` files, + would implement this interface by doing RPC over a given MachPort object. + * A server class, corresponding to `*Server.c`, would be able to handle + incoming messages using a user-provided implementation of the interface. + (Possibly, a skeleton class providing methods which would raise + `NotImplementedException`s would be provided as well. + Users would derive from this class and override the relevant methods. + This would allow them not to implement some operations, + and would avoid pre-existing code from breaking when new operations are + introduced.) + +In order to help with the implementation of servers, some kind of library +would be needed to associate Mach receive rights with server objects and to +handle incoming messages on dedicated threads, in the spirit of libports. +This would probably require support for port sets at the level of the Mach +primitives described in the previous section. + +When possible, operations involving the transmission of send rights +of some kind would be expressed in terms of the MIG-generated interfaces +instead of `MachPort` objects. +Upon reception of a send right, +a `FooUser` object would be created +and associated with the corresponding `MachPort` object. +If the received send right corresponds to a local port +to which a server object has been associated, +this object would be used instead. +This way, +subsequent operations on the received send right +would be handled as direct method calls +instead of going through RPC mechanisms. + +Some issues will still need to be solved regarding how MIG will convert +interface description files to Java interfaces. For instance: + + * `.defs` files are not explicitly associated with a type. For instance in + the example above, MIG would have to somehow infer that io_t corresponds + to `this` in the `Io` interface. + * More generally, a correspondence between MIG and Java types would have + to be determined. Ideally this would be automated and not hardcoded + too much. + * Initially, reply port parameters would be ignored. However they may be + needed for some applications. + +So the details would need to be flushed out during the community bonding +period and as the implementation progresses. However I’m confident that a +satisfactory solution can be designed. + +Using these new features, the example above could be rewritten as: + + import org.gnu.hurd.InitPorts; + import org.gnu.hurd.Io; + import org.gnu.hurd.IoUser; + + class Hello { + static void main(String argv[]) throws ... + { + Io stdout = new IoUser(InitPorts.getdport(1)); + String hello = “Hello, World!\n”; + + int amount = stdout.write(hello.getBytes(), -1); + + /* (A retCode corresponding to an error + would be signalled as an exception.) */ + } + } + +An example of server implementation would be: + + import org.gnu.hurd.Io; + import java.util.Arrays; + + class HelloIo implements Io { + final byte[] contents = “Hello, World!\n”.getBytes(); + + int write(byte[] data, int offset) { + return SOME_ERROR_CODE; + } + + byte[] read(int offset, int amount) { + return Arrays.copyOfRange(contents, offset, + offset + amount - 1); + } + + /* ... */ + } + +A new server object could then be created with `new IoServer(new HelloIo())`, +and associated with some receive right at the level of the ports management +library. + +### Base classes for common types of translators + +Once MIG can target Java code, and a libports equivalent is available, +creating new translators in Java would be greatly facilitated. However, +we would probably want to introduce basic implementations of file system +translators in the spirit of libtrivfs or libnetfs. They could take the form +of base classes implementing the relevant MIG-generated interfaces which +would then be derived by users, +or could define a simpler interface +which would then be used by adapter classes +to implement the required ones. + +I would draw inspiration from libtrivfs and libnetfs +to design and implement similar solutions for Java. + +### Deliverables + + * A hurd-java package would contain the Java code developed + in the context of this project. + * The Java code would be documented using javadoc + and a tutorial for writing translators would be written as well. + * Modifications to MIG would be submitted upstream, + or a patched MIG package would be made available. + +The Java libraries resulting from this work, +including any MIG support classes +as well as the class files built from the MIG-generated code +for the Mach and Hurd interface definition files, +would be provided as single `hurd-java` package for +Debian GNU/Hurd. +This package would be separate from both Hurd and Mach, +so as not to impose unreasonable build dependencies on them. + +I expect I would be able to act as its maintainer in the foreseeable future, +either as an individual or as a part of the Hurd team. +Hopefully, +my code would be claimed by the Hurd project as their own, +and consequently the modifications to MIG +(which would at least conceptually depend on the Mach Java package) +could be integrated upstream. + +Since by design, +the Java code would use only a small number of stable interfaces, +it would not be subject to excessive amounts of bitrot. +Consequently, +maintenance would primarily consist in +fixing bugs as they are reported, +and adding new features as they are requested. +A large number of such requests +would mean the package is useful, +so I expect that the overall amount of work +would be correlated with the willingness of more people +to help with maintenance +should I become overwhelmed or get hit by a bus. + + +## Timeline + +The dates listed are deadlines for the associated tasks. + + * *Community bonding period.* + Discuss, refine and complete the design of the Java bindings + (in particular the MIG and "libports" parts) + * *May 23.* + Coding starts. + * *May 30.* + Finish implementing pthread signal semantics. + * *June 5.* + Port OpenJDK + * *June 12.* + Fix the remaining problems with GCJ and/or OpenJDK, + possibly port Eclipse or other big Java packages. + * *June 19.* + Create the bindings for Mach. + * *June 26.* + Work on some kind of basic Java libports + to handle receive rights. + * *July 3.* + Test, write some documentation and examples. + * *July 17 (two weeks).* + Add the Java target to MIG. + * *July 24.* + Test, write some documentation and examples. + * *August 7 (two weeks).* + Implement a modular libfoofs to help with translator development. + Try to write a basic but non-trivial translator + to evaluate the performance and ease of use of the result, + rectify any rough edges this would uncover. + * *August 22. (last two weeks)* + Polish the code and packaging, + finish writing the documentation. + + +## Conclusion + +This project is arguably ambitious. +However, I have been thinking about it for some time now +and I'm confident I would be able to accomplish most of it. + +In the event multiple language bindings projects +would be accepted, +some work could probably be done in common. +In particular, +[ArneBab](http://www.bddebian.com/~hurd-web/community/weblogs/ArneBab/2011-04-06-application-pyhurd/) +seems to favor a low-level approach for his Python bindings as I do for Java, +and I would be happy to discuss API design and coordinate MIG changes with him. +I would also have an extra month after the end of the GSoC period +before I go back to school, +which I would be able to use to finish the project +if there is some remaining work. +(Last year's rewrite of procfs was done during this period.) + +As for the project's benefits, +I believe that good support for Java +is a must-have for the Hurd. +Java bindings would also further the Hurd's agenda +of user freedom by extending this freedom to more people: +I expect the set of developers +who would be able to write Java code against a well-written libfoofs +is much larger than +those who master the intricacies of low-level systems C programming. +From a more strategic point of view, +this would also help recruit new contributors +by providing an easier path to learning the inner workings of the Hurd. + +Further developments +which would build on the results of this project +include my planned [[experiment with Joe-E|objcap]] +(which I would possibly take on as a university project next year). +Another possibility would be to reimplement some parts +of the Java standard library +directly in terms of the Hurd interfaces +instead of using the POSIX ones through glibc. +This would possibly improve the performance +of some Java applications (though probably not by much), +and would otherwise be a good project +for someone trying to get acquainted with Hurd. + +Overall, I believe this project would be fun, interesting and useful. +I hope that you will share this sentiment +and give me the opportunity to spend another summer working on Hurd. + -- cgit v1.2.3 From 97ef5d15ef5e44806b92da486c5f06311db14727 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Wed, 15 Jun 2011 19:01:24 +0200 Subject: user/jkoenig/java: Integrate most of the action items from the discussion page. --- user/jkoenig/java.mdwn | 91 +++++++++++++++++++++++++++++++- user/jkoenig/java/discussion.mdwn | 108 -------------------------------------- 2 files changed, 90 insertions(+), 109 deletions(-) (limited to 'user/jkoenig') diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn index 90f51028..fcd316b7 100644 --- a/user/jkoenig/java.mdwn +++ b/user/jkoenig/java.mdwn @@ -38,7 +38,7 @@ I have submitted [preliminary patches](http://lists.gnu.org/archive/html/bug-hurd/2011-05/msg00182.html) for global signal dispositions, which I'm currently testing. -I have since fixed a few thinks and implemented `SA_SIGINFO` +I have since fixed a few things and implemented `SA_SIGINFO` (which is required by OpenJDK.) My latest code is available on [github](http://github.com/jeremie-koenig/glibc/commits/master-beware-rebase), @@ -56,6 +56,7 @@ at which point we will alias the interim version to the new one in debian packages. I have modified libc0.3 to include a `deb-symbols(5)` file +(alternatively see ) so that we get an accurate libc dependency in `hurd` and other packages when the symbols in question are pulled in. @@ -71,6 +72,28 @@ I expect only the last patch (implement global dispositions) will change, and new ones will be added on top of it. +##### Open Items + + * Test patches: in progress, [[jkoenig]], Svante. More volunteers welcome, + of course. + + * If [[jkoenig]] thinks it's mature enough: should ask Samuel to test this + (that is, only the refactoring patches for starters?) on the buildds. + + * Get patches reviewed (Roland?), and integrated into official sources: [!] + [[tschwinge]]. + + * Documentations bits (from [[proposal]] and elsewhere) should probably be + moved either into the appropriate glibc or Hurd documentation + files/reference manuals, or to [[glibc/signal]]. + + * libthreads (cthreads) integration. + + * [[tschwinge]] suggests to leave them as-is? + + * [[libpthread]] integration. + + ### Port OpenJDK As suggested by [[tschwinge]], I have targeted OpenJDK 7 at first. @@ -107,8 +130,74 @@ although the current toolchain issues have so far prevented me from testing it. +##### Open Items + + * They seem to have a rather heavy-weight process for such projects: confer + , + for example. Do we need this, too? + + * Eclipse + + OK for testing -- but I'd very much hope that it *just works* as soon as we + provide the required Java platform. But it may perhaps have some + Linux-specifics (needlessly?) in its basement. Is it available for Debian + GNU/kFreeBSD already? + + ### Java bindings for Mach + +#### Plans + (just started.) +##### Open Items + + * [[tschwinge]] has to read about RMI and CORBA. + + * MIG + + * Hacking [[microkernel/mach/MIG]] shouldn't be too difficult. + + * (Unless you want to make MIG's own code (that is, not the generated + code, but MIG itself) look a bit more nice, too.) ;-) + + * There are also alternatives to MIG. If there is interest, the following + could be considered: + + * FLICK ([[!GNU_Savannah_task 5723]]). [[tschwinge]] has no idea yet if + there would be any benefits over MIG, like better modularity (for the + backends)? If we feel like it, we could spend a little bit of time on + this. + + * For [[microkernel/Viengoos]], Neal has written a RPC stub generator + entirely in C Preprocessor macros. While this is obviously not + directly applicable, perhaps we can get some ideas from it. + + * Anything else that would be worth having a look at? (What are other + microkernels using?) + + * `mach_msg` + + * Seems like the right approach to [[tschwinge]], but he hasn't digested + all the pecularities yet. Will definitely need more time. + + +## Postponed + +Might get back to these as time/interest permits. + + +### GCJ + + * [[tschwinge]] has the feeling that Java in GCC (that is, GCJ) is mostly + dead? (True?) + + * Thus perhaps not too much effort should be spent with it. + + If the POSIX threads signal semantics makes it going, then great, otherwise + we should get a feeling what else is missing. + + +### Joe-E. diff --git a/user/jkoenig/java/discussion.mdwn b/user/jkoenig/java/discussion.mdwn index 0131d8d5..266a7bcc 100644 --- a/user/jkoenig/java/discussion.mdwn +++ b/user/jkoenig/java/discussion.mdwn @@ -56,114 +56,6 @@ one person's summer.) Such bits could be moved to [[open_issues]] pages, either new ones or existing ones, as applicable. -# POSIX Threads Signal Semantics - - * Great! [[tschwinge]] had a brief look, and should have a deeper one. - - * If [[jkoenig]] thinks it's mature enough: should ask Samuel to test this - (that is, only the refactoring patches for starters?) on the buildds. - - * Then: should ask Roland to review. - - * Documentations bits should probably be moved to [[glibc/signal]]. - - -## libthreads (cthreads) Integration - - * [[tschwinge]] suggests to leave them as-is? - - -## [[libpthread]] integration - - * To be done. - - -# Java - - * [[tschwinge]] has to read about RMI and CORBA. - - -# Joe-E - - * For later. - - -# GCJ - - * [[tschwinge]] has the feeling that Java in GCC (that is, GCJ) is mostly - dead? (True?) - - * Thus perhaps not too much effort should be spent with it. - - If the POSIX threads signal semantics makes it going, then great, otherwise - we should get a feeling what else is missing. - - -# OpenJDK - - * All in all, [[tschwinge]] has the feeling that a working OpenJDK will be - more useful/powerful than GCJ. - - * We need to get a feeling how difficult such an OS port will be. - - * [[jkoenig]] suggests OpenJDK 6 -- should we directly go for version 7 - instead? - - * What are the differences (regarding the OS port) between the two - versions? Or this there something even more recent to be worked upon, - for new OS ports? - - * Perhaps the different versions' OS port specific stuff is not at - all very different, so that both v6 and v7 could be done? - - * They seem to have a rather heavy-weight process for such projects: confer - , - for example. Do we need this, too? - - -# Eclipse - -OK for testing -- but I'd very much hope that it *just works* as soon as we -provide the required Java platform. - - -# Java Bindings - - -## Design Principles - - * Generally ack. - - -### MIG - - * Hacking [[microkernel/mach/MIG]] shouldn't be too difficult. - - * (Unless you want to make MIG's own code (that is, not the generated - code, but MIG itself) look a bit more nice, too.) ;-) - - * There are also alternatives to MIG. If there is interest, the following - could be considered: - - * FLICK ([[!GNU_Savannah_task 5723]]). [[tschwinge]] has no idea yet if - there would be any benefits over MIG, like better modularity (for the - backends)? If we feel like it, we could spend a little bit of time on - this. - - * For [[microkernel/Viengoos]], Neal has written a RPC stub generator - entirely in C Preprocessor macros. While this is obviously not - directly applicable, perhaps we can get some ideas from it. - - * Anything else that would be worth having a look at? (What are other - microkernels using?) - - -### `mach_msg` - - * Seems like the right approach to [[tschwinge]], but hasn't digested all the - pecularities yet. Will definitely need more time. - - # GSoC Site Discussion * Discussion items from -- cgit v1.2.3 From 5693033a406e3c5f1f536435b8d6c40bd2ef4df3 Mon Sep 17 00:00:00 2001 From: Jeremie Koenig Date: Wed, 15 Jun 2011 21:10:03 +0200 Subject: link to pochu's patches --- user/jkoenig/java.mdwn | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) (limited to 'user/jkoenig') diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn index fcd316b7..5702a0f2 100644 --- a/user/jkoenig/java.mdwn +++ b/user/jkoenig/java.mdwn @@ -123,7 +123,8 @@ the `LD_ORIGIN_PATH` environment variable. #### Plans I intend to fix the RPATH issue -by building on [[pochu]]'s `file_exec_file_name()` patches. +by building on [[pochu]]'s `file_exec_file_name()` +[patches](http://lists.gnu.org/archive/html/bug-hurd/2010-08/msg00023.html). I have succeeded in building a Hotspot-enabled `libjvm.so`, although the current toolchain issues -- cgit v1.2.3 From c0391c22faaca583d91a5ddb88fb59d409a1d855 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Wed, 15 Jun 2011 22:06:16 +0200 Subject: user/jkoenig/java: IRC meeting updates. --- user/jkoenig/java.mdwn | 30 +++++++++++++++++++++++++----- 1 file changed, 25 insertions(+), 5 deletions(-) (limited to 'user/jkoenig') diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn index fcd316b7..af2b908e 100644 --- a/user/jkoenig/java.mdwn +++ b/user/jkoenig/java.mdwn @@ -60,6 +60,12 @@ I have modified libc0.3 to include a `deb-symbols(5)` file so that we get an accurate libc dependency in `hurd` and other packages when the symbols in question are pulled in. +[[hurd/libthreads]] (cthreads library) will not be changed. There's no reason +why its behavior should change, whereas for [[libpthread]] it's needed for +conformance. Patches posted on 2011-05-25, but there's a more recent one in +the modified hurd package (adds `_hurd_sigstate_delete` and removes the weak +symbols). + Another issue which came up with OpenJDK is the expansion by the dynamic linker of `$ORIGIN` in the `RPATH` header, see below. @@ -77,8 +83,8 @@ and new ones will be added on top of it. * Test patches: in progress, [[jkoenig]], Svante. More volunteers welcome, of course. - * If [[jkoenig]] thinks it's mature enough: should ask Samuel to test this - (that is, only the refactoring patches for starters?) on the buildds. + * If [[jkoenig]] thinks it's mature enough: should ask + [[Samuel|samuelthibault]] to test these patches on the buildds. * Get patches reviewed (Roland?), and integrated into official sources: [!] [[tschwinge]]. @@ -87,11 +93,12 @@ and new ones will be added on top of it. moved either into the appropriate glibc or Hurd documentation files/reference manuals, or to [[glibc/signal]]. - * libthreads (cthreads) integration. + * `SA_SIGINFO` patche is based on [[Samuel|samuelthibault]]'s earlier work. + Thus, have him review the new patch? - * [[tschwinge]] suggests to leave them as-is? + * Perhaps have a look at `SA_NOCLDWAIT`. - * [[libpthread]] integration. + * Which numeric values to use for `SA_SIGINFO` (and `SA_NOCLDWAIT`)? ### Port OpenJDK @@ -105,6 +112,11 @@ should not be too hard. If that fails we can fall back on Shark (a portable alternative JIT which uses LLVM). +Complexity of porting HotSpot: probably low. The complex things should be +arch- rather than OS-specific. Not many Linux-specific interfaces used. +Garbage collection/memory management, etc. and/or most of other Linux-specific +interfaces are already dealt with for the zero build. + The dynamic linker issue is as follows. An executable-specific search path can be provided in the ELF RPATH header. RPATH directories can include the special string `$ORIGIN`, @@ -127,11 +139,19 @@ by building on [[pochu]]'s `file_exec_file_name()` patches. I have succeeded in building a Hotspot-enabled `libjvm.so`, although the current toolchain issues +(*Hurd OS ABI*) have so far prevented me from testing it. ##### Open Items + * [!] [[tschwinge]] to have a look at [[pochu]]'s `file_exec_file_name()` + patches (2010-05), whether it's generally the right idea. + + * Assuming it is, continue with getting `$ORIGIN` working. + + * [!] [[Samuel|samuelthibault]]/[[tschwinge]]/[[jkoenig]]: *Hurd OS ABI*. + * They seem to have a rather heavy-weight process for such projects: confer , for example. Do we need this, too? -- cgit v1.2.3 From a5b95baab6b4512ceaef8749a29ed7b3685b4125 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Wed, 15 Jun 2011 22:54:09 +0200 Subject: Some more bits from IRC and elsewhere. --- hurd/running/qemu/discussion.mdwn | 92 +++++++++++++++++++++++++++++++++ microkernel/mach/gnumach/ports/xen.mdwn | 24 +++++++-- toolchain.mdwn | 8 ++- toolchain/elfosabi_hurd.mdwn | 83 +++++++++++++++++++++++++++++ user/jkoenig/java.mdwn | 5 +- 5 files changed, 204 insertions(+), 8 deletions(-) create mode 100644 toolchain/elfosabi_hurd.mdwn (limited to 'user/jkoenig') diff --git a/hurd/running/qemu/discussion.mdwn b/hurd/running/qemu/discussion.mdwn index fd0df4c5..1ce14b01 100644 --- a/hurd/running/qemu/discussion.mdwn +++ b/hurd/running/qemu/discussion.mdwn @@ -50,3 +50,95 @@ The problem is actually that the linux block cache doesn't make any consistency between /dev/hda and /dev/hda6, so if you give /dev/hda to qemu, qemu writings won't be consistent with mounting /dev/hda6 in linux. You can give /dev/hda6 directly to qemu and it will be fine. + + +# Host-side Writeback Caching + +IRC, freenode, #hurd, 2011-06-07 + + hm, i guess i should have used cache=writeback with kvm before + starting the debian installer :/ + ah yes, much better + this shows how poor the state of our I/O drivers and subsystem is + :/ + indeed... still no clustered pageout :-( + and no I/O scheduler either + although an I/O scheduler has limited value without clustered + pageouts + since one of its goals is to pack related I/O requests together eh + i wonder if the wiki mentions using cache=writeback to speed up + qemu performances + it would help those unable to use kvm a lot + and even those running kvm too + kvm -m $RAM \ -monitor stdio \ -drive + cache=writeback,index=0,media=disk,file=hd0.img \ + etc.. + the idea is that qemu doesn't open its disk file synchronously + changes are queued in the host page cache before being flushed to + the disk image + but if you brutally close your qemu instance, you're likely to + loose file system consistency + ext2fs will think it has committed its metadata to the disk, but + the disk image won't be updated synchronously + on my machine (which is quite fast), my kvm has installed debian + like 10 times faster than without the option + braunr: I don't think killing qemu should hurt in this + case... probably only matters when the host machine dies + antrik: ah yes, right + it really makes everything faster, even downloading, since I/O + requests aren't interleaved between networking RPCs + regarding I/O sheduler... this discussion came up before, but I + don't remember the outcome -- doesn't the glued Linux driver actually + come with one? + i don't remember either + braunr: err... I don't think interleaving has anything to do with + it... I guess it's simply the fact that downloading writes the result to + disk, which suffers from lacking clustered pageout like everything else + (my internet connection is too slow though to notice :-) ) + well, if there is no I/O during downloading, downloading is faster + :) + +IRC, freenode, #hurd, 2011-06-08 + + youpi: does xen provide disk caching options ? + through a blktap, probably + ok + +([[microkernel/mach/gnumach/ports/Xen]], *Host-side Writeback Caching*.) + + we should find the pages mentioning qemu on the wiki and add the + options to enable disk image caching + it really makes the hurd run a lot faster + as a workaround for emulators until I/O is reworked, ofc + +IRC, freenode, #hurd, 2011-06-09 + + braunr recommends to use writeback caching with kvm. Is this + really recommended with the frequent crashes I experience? + provided that you terminate your kvm normaly (i.e. quitting it, not + killing it), there should be no difference + I think the host's stability is what matters + the data presumably sits in linux's cache even if qemu dies + violently + But the freezes I see force me to kill kvm :-( + maybe kvm doesn't even do caching indeed, I don't know + gnu_srs: you can quit even when frozen + use the console + (the kvm console) + gnu_srs, "Writeback caching will report data writes as completed + as soon as the data is present in the host page cache. This is safe as + long as you trust your host. If your host crashes or loses power, then + the guest may experience data corruption." (from the qemu manpage) + +IRC, freenode, #hurd, 2011-06-11 + + braunr: If you are online. For me setting the parameters -drive + cache=writeback,index=0,media=disk,file=hd0.img does not show any speed + improvement at all compared to the default. + gnu_srs: what's your complete qemu command line ? + kvm -m 1024 -net nic,model=rtl8139 -net + user,hostfwd=tcp::5556-:22 -drive + cache=writeback,index=0,media=disk,file=hd0.img -cdrom netinst.iso + what qemu version ? + qemu-kvm 0.14.1+dfsg-1: Sorry, I cannot be online until + tomorrow again. diff --git a/microkernel/mach/gnumach/ports/xen.mdwn b/microkernel/mach/gnumach/ports/xen.mdwn index bf26410a..af431c92 100644 --- a/microkernel/mach/gnumach/ports/xen.mdwn +++ b/microkernel/mach/gnumach/ports/xen.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2007, 2008, 2009 Free Software Foundation, +[[!meta copyright="Copyright © 2007, 2008, 2009, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -6,11 +6,12 @@ id="license" text="Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.2 or any later version published by the Free Software Foundation; with no Invariant Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled -[[GNU Free Documentation License|/fdl]]."]]"""]] +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] [[!toc]] + # Xen dom0, hypervisor /!\ Now that GNU Mach handles PAE you can use a PAE-enabled hypervisor. @@ -30,6 +31,7 @@ If you have a free partition, you can fdisk to type 0x83, create a filesystem us Replace /dev/sda4 with your partition. Install and use crosshurd to setup a GNU/Hurd system on this partition. + # /etc/xen/hurd configuration Here is a sample /etc/xen/hurd configuration @@ -49,6 +51,7 @@ Suggestions about [[networking_configuration]] are available. If you need stable MAC addresses, use a syntax like `vif = [ 'mac=00:16:3e:XX:XX:XX, bridge=br0' ]`. + # Running Hurd with Xen To run Hurd with Xen, use: @@ -63,6 +66,7 @@ and gnumach should get started. Proceed with native-install. - If `xm` complains about networking (`vif could not be connected`), it's Xen scripts' fault, see Xen documentation for how to configure the network. The simplest way is network-bridge with fixed IPs (note that you need the bridge-utils package for this). You can also just disable networking by commenting the vif line in the config. - If `xm` complains `Error: (2, 'Invalid kernel', 'xc_dom_compat_check: guest type xen-3.0-x86_32 not supported by xen kernel, sorry\n')`, you most probably have a PAE-enabled hypervisor and a non-PAE gnumach. Either install and boot non-PAE hypervisor and kernel, or rebuilt gnumach in PAE mode. + # Building from sources If you want to generate these images, first get the `gnumach-1-branch-Xen-branch` branch from gnumach CVS. @@ -75,7 +79,6 @@ Then use The current `hurd-modules` was built from the debian packages `hurd 20070606-2` and `libc0.3 2.6.1-1`. /!\ This means that when using this image, your GNU/Hurd system also needs to be a glibc version 2.6 or later-based one! ---- # Miscellaneous @@ -83,7 +86,6 @@ The current `hurd-modules` was built from the debian packages `hurd 20070606-2` [[!GNU_Savannah_task 5468]], [[!GNU_Savannah_task 6584]]. ---- # `pv-grub` @@ -91,3 +93,15 @@ From Xen 4.0 on you'll be able to run the GNU Hurd directly using `pv-grub`, without the need to [prepare a special bootstrap image](http://youpibouh.thefreecat.org/hurd-xen/build_hurd-modules) (like an initrd). + + +# Host-side Writeback Caching + +Optimization possible as it is with [[QEMU|hurd/running/qemu/discussion]], +*Host-side Writeback Caching*? + +IRC, freenode, #hurd, 2011-06-08 + + youpi: does xen provide disk caching options ? + through a blktap, probably + ok diff --git a/toolchain.mdwn b/toolchain.mdwn index b08dba95..0059317b 100644 --- a/toolchain.mdwn +++ b/toolchain.mdwn @@ -1,4 +1,4 @@ -[[!meta copyright="Copyright © 2007, 2008, 2010 Free Software Foundation, +[[!meta copyright="Copyright © 2007, 2008, 2010, 2011 Free Software Foundation, Inc."]] [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable @@ -24,4 +24,10 @@ for example, how things are done differently for GNU/Hurd than they are done for GNU/Linux. +--- + + * [[ELFOSABI_HURD]] + +--- + * [[cross-gnu]] diff --git a/toolchain/elfosabi_hurd.mdwn b/toolchain/elfosabi_hurd.mdwn new file mode 100644 index 00000000..4d60f761 --- /dev/null +++ b/toolchain/elfosabi_hurd.mdwn @@ -0,0 +1,83 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="ELFOSABI_HURD"]] + +[[!tag open_issue_binutils open_issue_glibc]] + + +# [[!debbug 630180]] + + +# [[open_issues/binutils]] commit 51b2f560ad035dffad3371093f8e5c80608d196c + +Usage of `ELFOSABI_LINUX`/`STB_GNU_UNIQUE`. Has also been wrong before already +with respect to `STT_GNU_IFUNC`? + + +# IRC + +IRC, freenode, #debian-hurd, 2011-06-11 + + youpi: not that there would be any hope in that, but id you try + asking doko about the gcc miscompiling (wrong elf format) issue? + I didn't + I'm still investigating + maybe it's a binutils change actually + youpi: hm, are you sure it could be binutils? after all, even + some .o files are produced with format gnu/linux, so there's no binutils + involved up to that point of thecompilation yet? + as is + "as", I mean + i see + since it's so unclear, I really prefer to investigate before + bothering doko + youpi: maybe i could be wrong, + in binutils, bfd/elf.c, around lines 9580 + the faulty thing seems to be gnu_unique_object in the source .s + file produced by g++ + that's what that comment (which changed wrt binutils from eg + march) says + seems to concur with my comment above :) + http://paste.debian.net/119542/ ‘¡û extract of diff + ok, that really seems the culprit + starting reportbug + who's the fault then? + binutils + it shouldn't hardcode LINUX + g++ emitting those symbols, or binutil considering them "linux"? + it's a GNU thing, not a Linux thing + ah ok + it's the same dynamic linker actually + youpi: http://sourceware.org/bugzilla/show_bug.cgi?id=10549 + see the reporter :) + heh + youpi: see also gas/config/obj-elf.c:1725 + (another change related to that bug, it seems) + +IRC, freenode, #hurd, 2011-06-15 + + also I still get an "ELF file OS ABI invalid" error with binutils + 2.21.52.20110606-2+hurd.1, is that expected? + tschwinge: oops, the OS ABI invalid is actually due to the file + being marked GNU/Hurd + I guess the linker is simply not aware that it should accept + GNU/Hurd + youpi: So we got to work on glibc'S ld.so to teach it aboput + the Hurd OS ABI? (Or probably simply make that equivalent to the Linux + one?) + probably simply an equivalent + ELFOSABI_HURD is missing from elf/elf.h, for a start... + linux' glibc has tests in lsdodefs.h + the VALID_ELF_OSABI macro + it's thus apparently a matter of providing an ldsodefs.h file with + VALID_ELF_HEADER, VALID_ELF_OSABI and VALID_ELF_ABIVERSION definitions + (and include_next the generic one) + I've prepared a patch for ldsodefs.h, I'll test it diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn index 6aeef9fe..8e1fb097 100644 --- a/user/jkoenig/java.mdwn +++ b/user/jkoenig/java.mdwn @@ -140,7 +140,7 @@ by building on [[pochu]]'s `file_exec_file_name()` I have succeeded in building a Hotspot-enabled `libjvm.so`, although the current toolchain issues -(*Hurd OS ABI*) +([[toolchain/ELFOSABI_HURD]]) have so far prevented me from testing it. @@ -151,7 +151,8 @@ have so far prevented me from testing it. * Assuming it is, continue with getting `$ORIGIN` working. - * [!] [[Samuel|samuelthibault]]/[[tschwinge]]/[[jkoenig]]: *Hurd OS ABI*. + * [!] [[Samuel|samuelthibault]]/[[tschwinge]]/[[jkoenig]]: + [[toolchain/ELFOSABI_HURD]]. * They seem to have a rather heavy-weight process for such projects: confer , -- cgit v1.2.3 From 00b16b8a6d29e31a87c6ef89671f6f87e3be3b04 Mon Sep 17 00:00:00 2001 From: Jeremie Koenig Date: Wed, 22 Jun 2011 20:51:34 +0200 Subject: update java status --- user/jkoenig/java.mdwn | 14 ++++++++++++++ 1 file changed, 14 insertions(+) (limited to 'user/jkoenig') diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn index 8e1fb097..fad68f58 100644 --- a/user/jkoenig/java.mdwn +++ b/user/jkoenig/java.mdwn @@ -83,9 +83,23 @@ and new ones will be added on top of it. * Test patches: in progress, [[jkoenig]], Svante. More volunteers welcome, of course. + > Current status: there's an issue with gdb, + > namely signals lose their "untracedness" when they go + > through the global sigstate's pending mask, + > so gdb spins intercepting a signal and trying to deliver it. + > My current + > [patch](http://github.com/jeremie-koenig/glibc/commit/3ecb990e9d08d5f75adc40b738b35a1802cc0943) + > makes the system unstable. + > --[[jkoenig]] 2011-06-22 + * If [[jkoenig]] thinks it's mature enough: should ask [[Samuel|samuelthibault]] to test these patches on the buildds. + > There's a risk that a dependency on my patched libc + > might be pulled in while building packages + > (in particular hurd) + > --[[jkoenig]] 2011-06-22 + * Get patches reviewed (Roland?), and integrated into official sources: [!] [[tschwinge]]. -- cgit v1.2.3 From ef05ca2c0d33045566dc6c3d104a3c35a804e74e Mon Sep 17 00:00:00 2001 From: Jeremie Koenig Date: Wed, 29 Jun 2011 19:23:17 +0200 Subject: java: status update for 2011-06-29 --- user/jkoenig/java.mdwn | 46 ++++++++++++++++++++++++++++++++++------------ 1 file changed, 34 insertions(+), 12 deletions(-) (limited to 'user/jkoenig') diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn index fad68f58..b289c0ed 100644 --- a/user/jkoenig/java.mdwn +++ b/user/jkoenig/java.mdwn @@ -34,12 +34,9 @@ Modified Debian packages are available in this repository: ### Glibc signal code improvements -I have submitted -[preliminary patches](http://lists.gnu.org/archive/html/bug-hurd/2011-05/msg00182.html) -for global signal dispositions, -which I'm currently testing. -I have since fixed a few things and implemented `SA_SIGINFO` -(which is required by OpenJDK.) +2011-06-29: +Patches were submitted to `libc-alpha` +which implement global signal dispositions and `SA_SIGINFO`. My latest code is available on [github](http://github.com/jeremie-koenig/glibc/commits/master-beware-rebase), and modified Debian packages @@ -49,9 +46,10 @@ One question is how the new symbols introduced by my patches should be handled. Weak symbols turned out to be impractical, so I'm currently considering using a Debian-specific -symbol version in the interim period (`GLIBC_2.13_DEBIAN_7` so far). +symbol version in the interim period (`GLIBC_2.13_DEBIAN_8` so far). The ultimate symbol version to be used will depend on -the time at which the patches get integrated upstream, +the time at which the patches get integrated upstream +(most likely `GLIBC_2.15`), at which point we will alias the interim version to the new one in debian packages. @@ -72,10 +70,11 @@ see below. #### Plans -I will submit revised series for review later this week, -as well as matching Debian patches. -I expect only the last patch (implement global dispositions) will change, -and new ones will be added on top of it. +The patches are pending review and inclusion upstream. +As soon as we reach an agreement wrt. the new interfaces +(in particular wrt. the value of `SA_SIGINFO`), +the patches will be applied to the Debian libc packages +for broader testing. ##### Open Items @@ -92,6 +91,9 @@ and new ones will be added on top of it. > makes the system unstable. > --[[jkoenig]] 2011-06-22 + >> Correction: the patch is fine, problem solved. + >> --[[jkoenig]] 2011-06-29 + * If [[jkoenig]] thinks it's mature enough: should ask [[Samuel|samuelthibault]] to test these patches on the buildds. @@ -103,6 +105,8 @@ and new ones will be added on top of it. * Get patches reviewed (Roland?), and integrated into official sources: [!] [[tschwinge]]. + > In progress. --[[jkoenig]] 2011-06-29 + * Documentations bits (from [[proposal]] and elsewhere) should probably be moved either into the appropriate glibc or Hurd documentation files/reference manuals, or to [[glibc/signal]]. @@ -114,6 +118,9 @@ and new ones will be added on top of it. * Which numeric values to use for `SA_SIGINFO` (and `SA_NOCLDWAIT`)? + > Staying in sync with BSD seems the most logical approach, + > so I have defined it to 0x40. --[[jkoenig]] 2011-06-29 + ### Port OpenJDK @@ -157,6 +164,15 @@ although the current toolchain issues ([[toolchain/ELFOSABI_HURD]]) have so far prevented me from testing it. +> It turns out the build fails later on in `hotspot/agent` +> because Hurd lack a `libthread_db.so`. +> Also, a Shark version builds, but the result does not work so far. +> +> In other news, Damien Raude-Morvan is +> [working on a kFreeBSD version](http://lists.debian.org/debian-java/2011/06/msg00124.html), +> so I intend to merge my current patches with his. +> +> --[[jkoenig]] 2011-06-29 ##### Open Items @@ -172,6 +188,12 @@ have so far prevented me from testing it. , for example. Do we need this, too? + > Probably not. + > My current approach (and Damien's wrt. the kFreeBSD patches) + > is to add preprocessor directives in the Linux code + > to make it more portable. + > --[[jkoenig]] 2011-06-29 + * Eclipse OK for testing -- but I'd very much hope that it *just works* as soon as we -- cgit v1.2.3 From f290172eeee710a3395d5c708c251f46667e839f Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Wed, 29 Jun 2011 22:43:46 +0200 Subject: user/jkoenig/java: Update after today's meeting. --- user/jkoenig/java.mdwn | 55 +++++++++++++++++++++++++++++++++++++------------- 1 file changed, 41 insertions(+), 14 deletions(-) (limited to 'user/jkoenig') diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn index b289c0ed..868dec2b 100644 --- a/user/jkoenig/java.mdwn +++ b/user/jkoenig/java.mdwn @@ -23,6 +23,14 @@ For details, see my original [[proposal]]. ## Current status +Feeling slightly behind schedule; but project is very ambitious, which has been +known from the beginning, and there is great progress, so there is no problem. +--[[tschwinge]], 2011-06-29. + +[[tschwinge]] will be on vacations in China starting July 26th, will have +Internet access intermittently, but not regularely. We'll have to figure out +some scheme. + ### Apt repository @@ -82,17 +90,11 @@ for broader testing. * Test patches: in progress, [[jkoenig]], Svante. More volunteers welcome, of course. - > Current status: there's an issue with gdb, + > There's an issue with gdb, > namely signals lose their "untracedness" when they go > through the global sigstate's pending mask, > so gdb spins intercepting a signal and trying to deliver it. - > My current - > [patch](http://github.com/jeremie-koenig/glibc/commit/3ecb990e9d08d5f75adc40b738b35a1802cc0943) - > makes the system unstable. - > --[[jkoenig]] 2011-06-22 - - >> Correction: the patch is fine, problem solved. - >> --[[jkoenig]] 2011-06-29 + > [Patch](http://github.com/jeremie-koenig/glibc/commit/3ecb990e9d08d5f75adc40b738b35a1802cc0943). * If [[jkoenig]] thinks it's mature enough: should ask [[Samuel|samuelthibault]] to test these patches on the buildds. @@ -102,24 +104,43 @@ for broader testing. > (in particular hurd) > --[[jkoenig]] 2011-06-22 + * Waiting on ABI finalization ([!] Roland). + + * Which numeric values to use for `SA_SIGINFO` (and `SA_NOCLDWAIT`)? + + > Staying in sync with BSD seems the most logical approach, + > so I have defined it to 0x40. --[[jkoenig]] 2011-06-29 + * Get patches reviewed (Roland?), and integrated into official sources: [!] [[tschwinge]]. > In progress. --[[jkoenig]] 2011-06-29 - * Documentations bits (from [[proposal]] and elsewhere) should probably be + * Documentations bits (from here, the initial [[proposal]], and elsewhere) + should probably be moved either into the appropriate glibc or Hurd documentation files/reference manuals, or to [[glibc/signal]]. - * `SA_SIGINFO` patche is based on [[Samuel|samuelthibault]]'s earlier work. + * `SA_SIGINFO` patch is based on [[Samuel|samuelthibault]]'s earlier work. Thus, have him review the new patch? - * Perhaps have a look at `SA_NOCLDWAIT`. + * `SA_SIGINFO` patch has a few TODOs w.r.t. protocol changes for missing + information, and for FPU state. Providing even incomplete information is + an improvement on the current status. The question is, whether + applications rely on this information in any hard way if `SA_SIGINFO` is + available? + + * We could possibly rename certain fields in `struct siginfo`, say + `si_pid_not_implemented`, to ensure compilation failures for programs + which use them. Or perhaps a linker warning is possible. - * Which numeric values to use for `SA_SIGINFO` (and `SA_NOCLDWAIT`)? + * The FPU state is not included in the `ucontext_t` passed to the signal + handler. On the other hand, `ucontext_t` is actually being somewhat + deprecated: the functions to restore it are no longer in POSIX. + `thread_get_state`() should return this information, in case we decide + to fill the gap, and there might be existing glibc wrappers, too. - > Staying in sync with BSD seems the most logical approach, - > so I have defined it to 0x40. --[[jkoenig]] 2011-06-29 + * Perhaps have a look at `SA_NOCLDWAIT`. ### Port OpenJDK @@ -184,6 +205,12 @@ have so far prevented me from testing it. * [!] [[Samuel|samuelthibault]]/[[tschwinge]]/[[jkoenig]]: [[toolchain/ELFOSABI_HURD]]. + * 2011-06-29: No progress. + + * `libthread_db.so` issue. Likely, the Serviceability Agent is used by jdb + and the like only, so for now the goal should be to lose some functionality + by removing/avoiding this dependency. + * They seem to have a rather heavy-weight process for such projects: confer , for example. Do we need this, too? -- cgit v1.2.3 From ba568e7ceef45fb80c588b81710f93dd3d6bfd25 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sun, 3 Jul 2011 23:58:36 +0200 Subject: open_issues/binutils: Update. --- contributing/web_pages/news/moth_next.mdwn | 2 + open_issues/binutils.mdwn | 48 +++++++---------- open_issues/binutils/log_build.diff | 74 ++++---------------------- open_issues/binutils/log_install.diff | 4 +- open_issues/binutils/sum_hurd | 17 +++--- open_issues/binutils/sum_linux | 17 +++--- toolchain.mdwn | 2 +- toolchain/elfosabi_gnu.mdwn | 32 ++++++++++++ toolchain/elfosabi_hurd.mdwn | 83 ------------------------------ user/jkoenig/java.mdwn | 7 +-- 10 files changed, 87 insertions(+), 199 deletions(-) create mode 100644 toolchain/elfosabi_gnu.mdwn delete mode 100644 toolchain/elfosabi_hurd.mdwn (limited to 'user/jkoenig') diff --git a/contributing/web_pages/news/moth_next.mdwn b/contributing/web_pages/news/moth_next.mdwn index 34c7e2f8..728804dc 100644 --- a/contributing/web_pages/news/moth_next.mdwn +++ b/contributing/web_pages/news/moth_next.mdwn @@ -41,4 +41,6 @@ And … + * [[toolchain/ELFOSABI_GNU]] + """]] diff --git a/open_issues/binutils.mdwn b/open_issues/binutils.mdwn index 5123a0d3..e654c53c 100644 --- a/open_issues/binutils.mdwn +++ b/open_issues/binutils.mdwn @@ -30,16 +30,8 @@ though, as explained below. # Configuration -Last reviewed up to the [[Git mirror's 38b56eda7ea117def3d5075110c0221fa1989878 -(2011-06-18) sources|source_repositories/binutils]]. - - * Open Issues - - * 51b2f560ad035dffad3371093f8e5c80608d196c: [[toolchain/ELFOSABI_HURD]]. - - * 0df7bac224004772a380db3e3b29933c16411940: restricts tests to - `*-*-linux*`. [Patch - posted](http://sourceware.org/ml/binutils/2011-06/msg00217.html). +Last reviewed up to the [[Git mirror's 57a1a9e822a763aeeb3ea66fd493aa58888c76c4 +(2011-07-03) sources|source_repositories/binutils]]. * Globally @@ -116,7 +108,7 @@ Last reviewed up to the [[Git mirror's 38b56eda7ea117def3d5075110c0221fa1989878 # Build Here's a log of a binutils build run; this is from our [[Git repository's -ab9f9e3e30d95fe653957ce5f554a01dcd453370 (2011-06-19) +6d4a384ae978b7a423f2cd51b396d366b9000df2 (2011-07-03) sources|source_repositories/binutils]], run on kepler.SCHWINGE and coulomb.SCHWINGE. @@ -175,28 +167,28 @@ This needs roughly 5 min on kepler.SCHWINGE, and 15 min on coulomb.SCHWINGE. Comparing the results files, [[sum_linux]] to [[sum_hurd]]: $ diff -u -F ^Running open_issues/binutils/sum_linux open_issues/binutils/sum_hurd - --- open_issues/binutils/sum_linux 2011-06-19 18:37:02.000000000 +0200 - +++ open_issues/binutils/sum_hurd 2011-06-19 20:40:21.000000000 +0200 + --- open_issues/binutils/sum_linux 2011-07-03 23:54:19.000000000 +0200 + +++ open_issues/binutils/sum_hurd 2011-07-03 23:54:25.000000000 +0200 @@ -1,5 +1,5 @@ - -Test Run By thomas on Sun Jun 19 18:26:46 2011 + -Test Run By thomas on Sun Jul 3 18:33:54 2011 -Native configuration is i686-pc-linux-gnu - +Test Run By thomas on Sun Jun 19 20:16:01 2011 + +Test Run By thomas on Sun Jul 3 19:11:49 2011 +Native configuration is i686-unknown-gnu0.3 === binutils tests === - @@ -117,8 +117,8 @@ Running [...]/hurd/master/binutils/tests + @@ -119,8 +119,8 @@ Running [...]/hurd/master/binutils/tests - # of expected passes 86 + # of expected passes 88 # of unsupported tests 2 - -Test Run By thomas on Sun Jun 19 18:27:17 2011 + -Test Run By thomas on Sun Jul 3 18:34:31 2011 -Native configuration is i686-pc-linux-gnu - +Test Run By thomas on Sun Jun 19 20:17:58 2011 + +Test Run By thomas on Sun Jul 3 19:14:02 2011 +Native configuration is i686-unknown-gnu0.3 === ld tests === - @@ -322,10 +322,10 @@ Running [...]/hurd/master/ld/testsuite/l + @@ -324,10 +324,10 @@ Running [...]/hurd/master/ld/testsuite/l PASS: PIE init array PASS: PIE fini array PASS: PIE init array mixed @@ -211,7 +203,7 @@ Comparing the results files, [[sum_linux]] to [[sum_hurd]]: Running [...]/hurd/master/ld/testsuite/ld-elf/exclude.exp ... PASS: ld link shared library PASS: ld export symbols from archive - @@ -579,8 +579,8 @@ Running [...]/hurd/master/ld/testsuite/l + @@ -581,8 +581,8 @@ Running [...]/hurd/master/ld/testsuite/l PASS: ELF DSO weak func last DSO PASS: ELF weak func first PASS: ELF weak func last @@ -222,7 +214,7 @@ Comparing the results files, [[sum_linux]] to [[sum_hurd]]: PASS: ELF DSO weak data first PASS: ELF DSO weak data last PASS: ELF DSO weak data first DSO - @@ -591,10 +591,10 @@ Running [...]/hurd/master/ld/testsuite/l + @@ -593,10 +593,10 @@ Running [...]/hurd/master/ld/testsuite/l PASS: ELF weak data last PASS: ELF weak data first common PASS: ELF weak data last common @@ -237,20 +229,20 @@ Comparing the results files, [[sum_linux]] to [[sum_hurd]]: PASS: ELF DSO small bar (size) PASS: ELF DSO foo with small bar (size) PASS: ELF DSO big bar (size) - @@ -921,13 +921,13 @@ Running [...]/hurd/master/ld/testsuite/l + @@ -924,13 +924,13 @@ Running [...]/hurd/master/ld/testsuite/l === ld Summary === - -# of expected passes 659 + -# of expected passes 660 -# of expected failures 8 - +# of expected passes 649 + +# of expected passes 650 +# of expected failures 18 # of untested testcases 6 - [...]/hurd/master.build/ld/ld-new 2.21.52.20110618 + [...]/hurd/master.build/ld/ld-new 2.21.52.20110703 - -Test Run By thomas on Sun Jun 19 18:26:56 2011 + -Test Run By thomas on Sun Jul 3 18:34:03 2011 -Native configuration is i686-pc-linux-gnu - +Test Run By thomas on Sun Jun 19 20:16:27 2011 + +Test Run By thomas on Sun Jul 3 19:12:21 2011 +Native configuration is i686-unknown-gnu0.3 === gas tests === diff --git a/open_issues/binutils/log_build.diff b/open_issues/binutils/log_build.diff index dbb9e2f6..0350d32a 100644 --- a/open_issues/binutils/log_build.diff +++ b/open_issues/binutils/log_build.diff @@ -1,5 +1,5 @@ ---- /dev/fd/63 2011-06-19 18:21:05.226518816 +0200 -+++ /dev/fd/62 2011-06-19 18:21:05.226518816 +0200 +--- /dev/fd/63 2011-07-03 19:28:40.826894462 +0200 ++++ /dev/fd/62 2011-07-03 19:28:40.830894333 +0200 @@ -276,12 +276,12 @@ checking for sys/sysinfo.h... yes checking for machine/hal_sysinfo.h... no @@ -50,12 +50,7 @@ checking whether the shell understands some XSI constructs... yes checking whether the shell understands "+="... no checking for ld option to reload object files... -r -@@ -473,18 +473,18 @@ - checking if gcc-4.6 supports -fno-rtti -fno-exceptions... no - checking for gcc-4.6 option to produce PIC... -fPIC -DPIC - checking if gcc-4.6 PIC flag -fPIC -DPIC works... yes --checking if gcc-4.6 static flag -static works... yes -+checking if gcc-4.6 static flag -static works... no +@@ -477,7 +477,7 @@ checking if gcc-4.6 supports -c -o file.o... yes checking if gcc-4.6 supports -c -o file.o... (cached) yes checking whether the gcc-4.6 linker (ld) supports shared libraries... yes @@ -64,14 +59,6 @@ checking how to hardcode library paths into programs... immediate checking for shl_load... no checking for shl_load in -ldld... no - checking for dlopen... no - checking for dlopen in -ldl... yes - checking whether a program can dlopen itself... yes --checking whether a statically linked program can dlopen itself... no -+checking whether a statically linked program can dlopen itself... yes - checking whether stripping libraries is possible... yes - checking if libtool supports shared libraries... yes - checking whether to build shared libraries... no @@ -567,26 +567,26 @@ checking sys/procfs.h usability... yes checking sys/procfs.h presence... yes @@ -108,15 +95,6 @@ checking for win32_pstatus_t in sys/procfs.h... no checking linker --as-needed support... yes checking for cos in -lm... yes -@@ -601,7 +601,7 @@ - checking for unistd.h... (cached) yes - checking for getpagesize... (cached) yes - checking for working mmap... yes --checking for madvise... yes -+checking for madvise... no - checking for mprotect... yes - configure: updating cache ./config.cache - configure: creating ./config.status @@ -1219,36 +1219,15 @@ /bin/dash ./libtool --tag=CC --mode=compile gcc-4.6 -DHAVE_CONFIG_H -I. -I../../master/bfd -I. -I../../master/bfd -I../../master/bfd/../include -DHAVE_bfd_elf32_i386_vec -DHAVE_bfd_elf32_little_generic_vec -DHAVE_bfd_elf32_big_generic_vec -DBINDIR='"[...]/hurd/master.build.install/bin"' -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT dwarf1.lo -MD -MP -MF .deps/dwarf1.Tpo -c -o dwarf1.lo ../../master/bfd/dwarf1.c libtool: compile: gcc-4.6 -DHAVE_CONFIG_H -I. -I../../master/bfd -I. -I../../master/bfd -I../../master/bfd/../include -DHAVE_bfd_elf32_i386_vec -DHAVE_bfd_elf32_little_generic_vec -DHAVE_bfd_elf32_big_generic_vec -DBINDIR=\"[...]/hurd/master.build.install/bin\" -W -Wall -Wstrict-prototypes -Wmissing-prototypes -Wshadow -Werror -g -O2 -MT dwarf1.lo -MD -MP -MF .deps/dwarf1.Tpo -c ../../master/bfd/dwarf1.c -o dwarf1.o @@ -173,12 +151,7 @@ checking whether the shell understands some XSI constructs... yes checking whether the shell understands "+="... no checking for ld option to reload object files... -r -@@ -1341,11 +1320,11 @@ - checking if gcc-4.6 supports -fno-rtti -fno-exceptions... no - checking for gcc-4.6 option to produce PIC... -fPIC -DPIC - checking if gcc-4.6 PIC flag -fPIC -DPIC works... yes --checking if gcc-4.6 static flag -static works... yes -+checking if gcc-4.6 static flag -static works... no +@@ -1345,7 +1324,7 @@ checking if gcc-4.6 supports -c -o file.o... yes checking if gcc-4.6 supports -c -o file.o... (cached) yes checking whether the gcc-4.6 linker (ld) supports shared libraries... yes @@ -196,13 +169,7 @@ checking whether the shell understands some XSI constructs... yes checking whether the shell understands "+="... no checking for ld option to reload object files... -r -@@ -1516,12 +1495,12 @@ - checking if gcc-4.6 supports -fno-rtti -fno-exceptions... no - checking for gcc-4.6 option to produce PIC... -fPIC -DPIC - checking if gcc-4.6 PIC flag -fPIC -DPIC works... yes --checking if gcc-4.6 static flag -static works... yes -+checking if gcc-4.6 static flag -static works... no - checking if gcc-4.6 supports -c -o file.o... yes +@@ -1521,7 +1500,7 @@ checking if gcc-4.6 supports -c -o file.o... (cached) yes checking whether the gcc-4.6 linker (ld) supports shared libraries... yes checking whether -lc should be explicitly linked in... no @@ -220,13 +187,7 @@ checking whether the shell understands some XSI constructs... yes checking whether the shell understands "+="... no checking for ld option to reload object files... -r -@@ -1983,12 +1962,12 @@ - checking if gcc-4.6 supports -fno-rtti -fno-exceptions... no - checking for gcc-4.6 option to produce PIC... -fPIC -DPIC - checking if gcc-4.6 PIC flag -fPIC -DPIC works... yes --checking if gcc-4.6 static flag -static works... yes -+checking if gcc-4.6 static flag -static works... no - checking if gcc-4.6 supports -c -o file.o... yes +@@ -1988,7 +1967,7 @@ checking if gcc-4.6 supports -c -o file.o... (cached) yes checking whether the gcc-4.6 linker (ld) supports shared libraries... yes checking whether -lc should be explicitly linked in... no @@ -244,13 +205,7 @@ checking whether the shell understands some XSI constructs... yes checking whether the shell understands "+="... no checking for ld option to reload object files... -r -@@ -2237,12 +2216,12 @@ - checking if gcc-4.6 supports -fno-rtti -fno-exceptions... no - checking for gcc-4.6 option to produce PIC... -fPIC -DPIC - checking if gcc-4.6 PIC flag -fPIC -DPIC works... yes --checking if gcc-4.6 static flag -static works... yes -+checking if gcc-4.6 static flag -static works... no - checking if gcc-4.6 supports -c -o file.o... yes +@@ -2242,7 +2221,7 @@ checking if gcc-4.6 supports -c -o file.o... (cached) yes checking whether the gcc-4.6 linker (ld) supports shared libraries... yes checking whether -lc should be explicitly linked in... no @@ -268,13 +223,7 @@ checking whether the shell understands some XSI constructs... yes checking whether the shell understands "+="... no checking for ld option to reload object files... -r -@@ -2477,12 +2456,12 @@ - checking if gcc-4.6 supports -fno-rtti -fno-exceptions... no - checking for gcc-4.6 option to produce PIC... -fPIC -DPIC - checking if gcc-4.6 PIC flag -fPIC -DPIC works... yes --checking if gcc-4.6 static flag -static works... yes -+checking if gcc-4.6 static flag -static works... no - checking if gcc-4.6 supports -c -o file.o... yes +@@ -2482,7 +2461,7 @@ checking if gcc-4.6 supports -c -o file.o... (cached) yes checking whether the gcc-4.6 linker (ld) supports shared libraries... yes checking whether -lc should be explicitly linked in... no @@ -283,12 +232,7 @@ checking how to hardcode library paths into programs... immediate checking whether stripping libraries is possible... yes checking if libtool supports shared libraries... yes -@@ -2494,11 +2473,11 @@ - checking whether the g++-4.6 linker (ld) supports shared libraries... yes - checking for g++-4.6 option to produce PIC... -fPIC -DPIC - checking if g++-4.6 PIC flag -fPIC -DPIC works... yes --checking if g++-4.6 static flag -static works... yes -+checking if g++-4.6 static flag -static works... no +@@ -2498,7 +2477,7 @@ checking if g++-4.6 supports -c -o file.o... yes checking if g++-4.6 supports -c -o file.o... (cached) yes checking whether the g++-4.6 linker (ld) supports shared libraries... yes diff --git a/open_issues/binutils/log_install.diff b/open_issues/binutils/log_install.diff index f808a160..83e01c55 100644 --- a/open_issues/binutils/log_install.diff +++ b/open_issues/binutils/log_install.diff @@ -1,5 +1,5 @@ ---- /dev/fd/63 2011-06-19 18:21:13.210325860 +0200 -+++ /dev/fd/62 2011-06-19 18:21:13.210325860 +0200 +--- /dev/fd/63 2011-07-03 19:28:49.538615011 +0200 ++++ /dev/fd/62 2011-07-03 19:28:49.538615011 +0200 @@ -68,7 +68,6 @@ libtool: install: /usr/bin/install -c .libs/libbfd.a [...]/hurd/master.build.install/lib/libbfd.a libtool: install: chmod 644 [...]/hurd/master.build.install/lib/libbfd.a diff --git a/open_issues/binutils/sum_hurd b/open_issues/binutils/sum_hurd index b7368648..42f4d331 100644 --- a/open_issues/binutils/sum_hurd +++ b/open_issues/binutils/sum_hurd @@ -1,4 +1,4 @@ -Test Run By thomas on Sun Jun 19 20:16:01 2011 +Test Run By thomas on Sun Jul 3 19:11:49 2011 Native configuration is i686-unknown-gnu0.3 === binutils tests === @@ -50,10 +50,12 @@ PASS: objcopy --adjust-start PASS: objcopy --adjust-vma PASS: objcopy --adjust-section-vma + PASS: objcopy --adjust-section-vma = +PASS: strip preserving OS/ABI PASS: strip PASS: strip with saving a symbol PASS: simple objcopy of executable PASS: run objcopy of executable +PASS: run stripped executable preserving OS/ABI PASS: run stripped executable PASS: run stripped executable with saving a symbol PASS: keep only debug data @@ -115,9 +117,9 @@ Running [...]/hurd/master/binutils/testsuite/binutils-all/x86-64/x86-64.exp ... === binutils Summary === -# of expected passes 86 +# of expected passes 88 # of unsupported tests 2 -Test Run By thomas on Sun Jun 19 20:17:58 2011 +Test Run By thomas on Sun Jul 3 19:14:02 2011 Native configuration is i686-unknown-gnu0.3 === ld tests === @@ -653,6 +655,7 @@ PASS: ld-i386/nogot1 PASS: ld-i386/nogot2 PASS: ld-i386/discarded1 PASS: PR ld/12718 +PASS: PR ld/12921 PASS: undefined symbol with compressed debug sections PASS: PR ld/12627 Running [...]/hurd/master/ld/testsuite/ld-ia64/ia64.exp ... @@ -921,12 +924,12 @@ Running [...]/hurd/master/ld/testsuite/ld-xtensa/xtensa.exp ... === ld Summary === -# of expected passes 649 +# of expected passes 650 # of expected failures 18 # of untested testcases 6 -[...]/hurd/master.build/ld/ld-new 2.21.52.20110618 +[...]/hurd/master.build/ld/ld-new 2.21.52.20110703 -Test Run By thomas on Sun Jun 19 20:16:27 2011 +Test Run By thomas on Sun Jul 3 19:12:21 2011 Native configuration is i686-unknown-gnu0.3 === gas tests === @@ -1394,5 +1397,5 @@ Running [...]/hurd/master/gas/testsuite/gas/z8k/z8k.exp ... === gas Summary === # of expected passes 342 -../as-new 2.21.52.20110618 +../as-new 2.21.52.20110703 diff --git a/open_issues/binutils/sum_linux b/open_issues/binutils/sum_linux index f1a383b8..4e37f06c 100644 --- a/open_issues/binutils/sum_linux +++ b/open_issues/binutils/sum_linux @@ -1,4 +1,4 @@ -Test Run By thomas on Sun Jun 19 18:26:46 2011 +Test Run By thomas on Sun Jul 3 18:33:54 2011 Native configuration is i686-pc-linux-gnu === binutils tests === @@ -50,10 +50,12 @@ PASS: objcopy --adjust-start PASS: objcopy --adjust-vma PASS: objcopy --adjust-section-vma + PASS: objcopy --adjust-section-vma = +PASS: strip preserving OS/ABI PASS: strip PASS: strip with saving a symbol PASS: simple objcopy of executable PASS: run objcopy of executable +PASS: run stripped executable preserving OS/ABI PASS: run stripped executable PASS: run stripped executable with saving a symbol PASS: keep only debug data @@ -115,9 +117,9 @@ Running [...]/hurd/master/binutils/testsuite/binutils-all/x86-64/x86-64.exp ... === binutils Summary === -# of expected passes 86 +# of expected passes 88 # of unsupported tests 2 -Test Run By thomas on Sun Jun 19 18:27:17 2011 +Test Run By thomas on Sun Jul 3 18:34:31 2011 Native configuration is i686-pc-linux-gnu === ld tests === @@ -653,6 +655,7 @@ PASS: ld-i386/nogot1 PASS: ld-i386/nogot2 PASS: ld-i386/discarded1 PASS: PR ld/12718 +PASS: PR ld/12921 PASS: undefined symbol with compressed debug sections PASS: PR ld/12627 Running [...]/hurd/master/ld/testsuite/ld-ia64/ia64.exp ... @@ -921,12 +924,12 @@ Running [...]/hurd/master/ld/testsuite/ld-xtensa/xtensa.exp ... === ld Summary === -# of expected passes 659 +# of expected passes 660 # of expected failures 8 # of untested testcases 6 -[...]/hurd/master.build/ld/ld-new 2.21.52.20110618 +[...]/hurd/master.build/ld/ld-new 2.21.52.20110703 -Test Run By thomas on Sun Jun 19 18:26:56 2011 +Test Run By thomas on Sun Jul 3 18:34:03 2011 Native configuration is i686-pc-linux-gnu === gas tests === @@ -1394,5 +1397,5 @@ Running [...]/hurd/master/gas/testsuite/gas/z8k/z8k.exp ... === gas Summary === # of expected passes 342 -../as-new 2.21.52.20110618 +../as-new 2.21.52.20110703 diff --git a/toolchain.mdwn b/toolchain.mdwn index 0059317b..26144cc3 100644 --- a/toolchain.mdwn +++ b/toolchain.mdwn @@ -26,7 +26,7 @@ for GNU/Linux. --- - * [[ELFOSABI_HURD]] + * [[ELFOSABI_GNU]] --- diff --git a/toolchain/elfosabi_gnu.mdwn b/toolchain/elfosabi_gnu.mdwn new file mode 100644 index 00000000..76711e3b --- /dev/null +++ b/toolchain/elfosabi_gnu.mdwn @@ -0,0 +1,32 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!meta title="ELFOSABI_GNU"]] + +GNU/Hurd uses the `ELFOSABI_GNU` value for operating system/ABI identification. +This is shared with GNU/Linux. + + +# History + + * [[!debbug 630180]] + * [sourceware bug + 12913](http://sourceware.org/bugzilla/show_bug.cgi?id=12913) + * [libc-alpha + thread](http://sourceware.org/ml/libc-alpha/2011-06/threads.html#00087) + ([continues](http://sourceware.org/ml/libc-alpha/2011-07/threads.html#00031)) + * [bug-hurd + thread](http://lists.gnu.org/archive/html/bug-hurd/2011-06/threads.html#00060) + ([continues](http://lists.gnu.org/archive/html/bug-hurd/2011-07/threads.html#00020)) + * [generic-abi + thread](http://groups.google.com/group/generic-abi/browse_frm/thread/194697b94a189063) + * [binutils + thread](http://sourceware.org/ml/binutils/2011-06/threads.html#00218) + ([continues](http://sourceware.org/ml/binutils/2011-07/threads.html#00033)) diff --git a/toolchain/elfosabi_hurd.mdwn b/toolchain/elfosabi_hurd.mdwn deleted file mode 100644 index 4d60f761..00000000 --- a/toolchain/elfosabi_hurd.mdwn +++ /dev/null @@ -1,83 +0,0 @@ -[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] - -[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable -id="license" text="Permission is granted to copy, distribute and/or modify this -document under the terms of the GNU Free Documentation License, Version 1.2 or -any later version published by the Free Software Foundation; with no Invariant -Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license -is included in the section entitled [[GNU Free Documentation -License|/fdl]]."]]"""]] - -[[!meta title="ELFOSABI_HURD"]] - -[[!tag open_issue_binutils open_issue_glibc]] - - -# [[!debbug 630180]] - - -# [[open_issues/binutils]] commit 51b2f560ad035dffad3371093f8e5c80608d196c - -Usage of `ELFOSABI_LINUX`/`STB_GNU_UNIQUE`. Has also been wrong before already -with respect to `STT_GNU_IFUNC`? - - -# IRC - -IRC, freenode, #debian-hurd, 2011-06-11 - - youpi: not that there would be any hope in that, but id you try - asking doko about the gcc miscompiling (wrong elf format) issue? - I didn't - I'm still investigating - maybe it's a binutils change actually - youpi: hm, are you sure it could be binutils? after all, even - some .o files are produced with format gnu/linux, so there's no binutils - involved up to that point of thecompilation yet? - as is - "as", I mean - i see - since it's so unclear, I really prefer to investigate before - bothering doko - youpi: maybe i could be wrong, - in binutils, bfd/elf.c, around lines 9580 - the faulty thing seems to be gnu_unique_object in the source .s - file produced by g++ - that's what that comment (which changed wrt binutils from eg - march) says - seems to concur with my comment above :) - http://paste.debian.net/119542/ ‘¡û extract of diff - ok, that really seems the culprit - starting reportbug - who's the fault then? - binutils - it shouldn't hardcode LINUX - g++ emitting those symbols, or binutil considering them "linux"? - it's a GNU thing, not a Linux thing - ah ok - it's the same dynamic linker actually - youpi: http://sourceware.org/bugzilla/show_bug.cgi?id=10549 - see the reporter :) - heh - youpi: see also gas/config/obj-elf.c:1725 - (another change related to that bug, it seems) - -IRC, freenode, #hurd, 2011-06-15 - - also I still get an "ELF file OS ABI invalid" error with binutils - 2.21.52.20110606-2+hurd.1, is that expected? - tschwinge: oops, the OS ABI invalid is actually due to the file - being marked GNU/Hurd - I guess the linker is simply not aware that it should accept - GNU/Hurd - youpi: So we got to work on glibc'S ld.so to teach it aboput - the Hurd OS ABI? (Or probably simply make that equivalent to the Linux - one?) - probably simply an equivalent - ELFOSABI_HURD is missing from elf/elf.h, for a start... - linux' glibc has tests in lsdodefs.h - the VALID_ELF_OSABI macro - it's thus apparently a matter of providing an ldsodefs.h file with - VALID_ELF_HEADER, VALID_ELF_OSABI and VALID_ELF_ABIVERSION definitions - (and include_next the generic one) - I've prepared a patch for ldsodefs.h, I'll test it diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn index 868dec2b..2285c89f 100644 --- a/user/jkoenig/java.mdwn +++ b/user/jkoenig/java.mdwn @@ -182,7 +182,7 @@ by building on [[pochu]]'s `file_exec_file_name()` I have succeeded in building a Hotspot-enabled `libjvm.so`, although the current toolchain issues -([[toolchain/ELFOSABI_HURD]]) +([[toolchain/ELFOSABI_GNU]]; 2011-07-03: fix committed in binutils) have so far prevented me from testing it. > It turns out the build fails later on in `hotspot/agent` @@ -202,11 +202,6 @@ have so far prevented me from testing it. * Assuming it is, continue with getting `$ORIGIN` working. - * [!] [[Samuel|samuelthibault]]/[[tschwinge]]/[[jkoenig]]: - [[toolchain/ELFOSABI_HURD]]. - - * 2011-06-29: No progress. - * `libthread_db.so` issue. Likely, the Serviceability Agent is used by jdb and the like only, so for now the goal should be to lose some functionality by removing/avoiding this dependency. -- cgit v1.2.3 From 67d369a640f64a7076bbd1605b712c41879bd94f Mon Sep 17 00:00:00 2001 From: Jeremie Koenig Date: Wed, 6 Jul 2011 19:40:30 +0200 Subject: java status update --- user/jkoenig/java.mdwn | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-) (limited to 'user/jkoenig') diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn index 2285c89f..fe391582 100644 --- a/user/jkoenig/java.mdwn +++ b/user/jkoenig/java.mdwn @@ -114,7 +114,20 @@ for broader testing. * Get patches reviewed (Roland?), and integrated into official sources: [!] [[tschwinge]]. - > In progress. --[[jkoenig]] 2011-06-29 + > [[samuelthibault]] reviewed the patches and pointed out a couple of + > issues which I'm currently working on: + > + > * Slight behaviour change with respect to forgetting blocked ignored + > signals. POSIX is flexible in this regard but I guess we could retain + > them instead of the current behaviour. + > * Sigstate accessors could be made extern inline functions. + > I suggest we postpone this. + > * Incorrect changes for `msg_{get,set}_init_int(INIT_SIGMASK)` + > * Some comments which can be improved. + > + > Once these are fixed we can probably test the patches in Debian. + > + > --[[jk]] 2011-07-06 * Documentations bits (from here, the initial [[proposal]], and elsewhere) should probably be @@ -226,6 +239,7 @@ have so far prevented me from testing it. ### Java bindings for Mach +The code is at . #### Plans -- cgit v1.2.3 From f2cbce0dc26005f74aeb5ca1011f4923bc3aaaba Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 8 Jul 2011 22:35:03 +0200 Subject: user/jkoenig/java/java-access-bridge: New. --- user/jkoenig/java.mdwn | 2 + user/jkoenig/java/java-access-bridge.mdwn | 78 +++++++++++++++++++++++++++++++ 2 files changed, 80 insertions(+) create mode 100644 user/jkoenig/java/java-access-bridge.mdwn (limited to 'user/jkoenig') diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn index fe391582..7df1dd73 100644 --- a/user/jkoenig/java.mdwn +++ b/user/jkoenig/java.mdwn @@ -219,6 +219,8 @@ have so far prevented me from testing it. and the like only, so for now the goal should be to lose some functionality by removing/avoiding this dependency. + * [[java-access-bridge]] (not critical; JVM appears to work without) + * They seem to have a rather heavy-weight process for such projects: confer , for example. Do we need this, too? diff --git a/user/jkoenig/java/java-access-bridge.mdwn b/user/jkoenig/java/java-access-bridge.mdwn new file mode 100644 index 00000000..6f860709 --- /dev/null +++ b/user/jkoenig/java/java-access-bridge.mdwn @@ -0,0 +1,78 @@ +[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]] + +[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable +id="license" text="Permission is granted to copy, distribute and/or modify this +document under the terms of the GNU Free Documentation License, Version 1.2 or +any later version published by the Free Software Foundation; with no Invariant +Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license +is included in the section entitled [[GNU Free Documentation +License|/fdl]]."]]"""]] + +[[!tag open_issue_porting]] + +Debian's *openjdk-7-jre* package depends on *libaccess-bridge-java-jni* (source +package: *java-access-bridge*). + +The latter one has *openjdk-6-jdk* as a build dependency, but that can be +hacked around: + + # ln -s java-7-openjdk /usr/lib/jvm/java-6-openjdk + +Trying to build it: + + $ LD_LIBRARY_PATH=/usr/lib/jvm/java-7-openjdk/jre/lib/i386/jli dpkg-buildpackage -b -uc -d + [...] + make[3]: Entering directory `/media/erich/home/thomas/tmp/libaccess-bridge-java-jni/java-access-bridge-1.26.2/idlgen' + /usr/lib/jvm/java-6-openjdk/bin/idlj \ + -pkgPrefix Bonobo org.GNOME \ + -pkgPrefix Accessibility org.GNOME \ + -emitAll -i /usr/share/idl/bonobo-activation-2.0 -i /usr/share/idl/at-spi-1.0 -i /usr/share/idl/bonobo-2.0 \ + -fallTie /usr/share/idl/at-spi-1.0/Accessibility.idl + /usr/share/idl/at-spi-1.0/Accessibility_Collection.idl (line 66): WARNING: Identifier `object' collides with a keyword; use an escaped identifier to ensure future compatibility. + boolean isAncestorOf (in Accessible object); + ^ + /usr/share/idl/at-spi-1.0/Accessibility_Component.idl (line 83): WARNING: Identifier `Component' collides with a keyword; use an escaped identifier to ensure future compatibility. + interface Component : Bonobo::Unknown { + ^ + Exception in thread "main" java.lang.AssertionError: Platform not recognized + at sun.nio.fs.DefaultFileSystemProvider.create(DefaultFileSystemProvider.java:71) + at java.nio.file.FileSystems$DefaultFileSystemHolder.getDefaultProvider(FileSystems.java:108) + at java.nio.file.FileSystems$DefaultFileSystemHolder.access$000(FileSystems.java:89) + at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:98) + at java.nio.file.FileSystems$DefaultFileSystemHolder$1.run(FileSystems.java:96) + at java.security.AccessController.doPrivileged(Native Method) + at java.nio.file.FileSystems$DefaultFileSystemHolder.defaultFileSystem(FileSystems.java:95) + at java.nio.file.FileSystems$DefaultFileSystemHolder.(FileSystems.java:90) + at java.nio.file.FileSystems.getDefault(FileSystems.java:176) + at sun.util.calendar.ZoneInfoFile$1.run(ZoneInfoFile.java:489) + at sun.util.calendar.ZoneInfoFile$1.run(ZoneInfoFile.java:480) + at java.security.AccessController.doPrivileged(Native Method) + at sun.util.calendar.ZoneInfoFile.(ZoneInfoFile.java:479) + at sun.util.calendar.ZoneInfo.getTimeZone(ZoneInfo.java:658) + at java.util.TimeZone.getTimeZone(TimeZone.java:559) + at java.util.TimeZone.setDefaultZone(TimeZone.java:656) + at java.util.TimeZone.getDefaultRef(TimeZone.java:623) + at java.util.TimeZone.getDefault(TimeZone.java:610) + at java.text.SimpleDateFormat.initializeCalendar(SimpleDateFormat.java:682) + at java.text.SimpleDateFormat.(SimpleDateFormat.java:619) + at java.text.DateFormat.get(DateFormat.java:772) + at java.text.DateFormat.getDateTimeInstance(DateFormat.java:547) + at com.sun.tools.corba.se.idl.toJavaPortable.Util.writeProlog(Util.java:1139) + at com.sun.tools.corba.se.idl.toJavaPortable.Skeleton.writeHeading(Skeleton.java:145) + at com.sun.tools.corba.se.idl.toJavaPortable.Skeleton.generate(Skeleton.java:102) + at com.sun.tools.corba.se.idl.toJavaPortable.InterfaceGen.generateSkeleton(InterfaceGen.java:159) + at com.sun.tools.corba.se.idl.toJavaPortable.InterfaceGen.generate(InterfaceGen.java:108) + at com.sun.tools.corba.se.idl.InterfaceEntry.generate(InterfaceEntry.java:110) + at com.sun.tools.corba.se.idl.toJavaPortable.ModuleGen.generate(ModuleGen.java:75) + at com.sun.tools.corba.se.idl.ModuleEntry.generate(ModuleEntry.java:83) + at com.sun.tools.corba.se.idl.Compile.generate(Compile.java:324) + at com.sun.tools.corba.se.idl.toJavaPortable.Compile.start(Compile.java:169) + at com.sun.tools.corba.se.idl.toJavaPortable.Compile.main(Compile.java:146) + make[3]: *** [org/GNOME/Accessibility/Accessible.java] Error 1 + make[3]: Leaving directory `/media/erich/home/thomas/tmp/libaccess-bridge-java-jni/java-access-bridge-1.26.2/idlgen' + make[2]: *** [all-recursive] Error 1 + make[2]: Leaving directory `/media/erich/home/thomas/tmp/libaccess-bridge-java-jni/java-access-bridge-1.26.2/idlgen' + make[1]: *** [all-recursive] Error 1 + make[1]: Leaving directory `/media/erich/home/thomas/tmp/libaccess-bridge-java-jni/java-access-bridge-1.26.2' + make: *** [debian/stamp-makefile-build] Error 2 + dpkg-buildpackage: error: debian/rules build gave error exit status 2 -- cgit v1.2.3 From ca102c8b689d1409a22960b22f511eb5649c122a Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Tue, 12 Jul 2011 19:03:55 +0200 Subject: user/jkoenig/java : How to build. --- user/jkoenig/java.mdwn | 12 ++++++++++++ 1 file changed, 12 insertions(+) (limited to 'user/jkoenig') diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn index 7df1dd73..01dfae12 100644 --- a/user/jkoenig/java.mdwn +++ b/user/jkoenig/java.mdwn @@ -243,6 +243,18 @@ have so far prevented me from testing it. The code is at . +[[tschwinge]]'s notes for building with... + + * GCJ installed (due to the current Debian multilib confusion): + + $ tmp1=/usr/lib/gcc/i486-gnu/4.6 tmp2=/usr/lib/i386-gnu/gcc/i486-gnu/4.6 LIBRARY_PATH=$tmp2 COMPILER_PATH=$tmp1:$tmp2 C_INCLUDE_PATH=$tmp1/include make + + * OpenJDK installed (to have it find the shared library, and the jni.h header + file): + + $ jdk=/usr/lib/jvm/java-7-openjdk LD_LIBRARY_PATH=$jdk/jre/lib/i386/jli C_INCLUDE_PATH=$jdk/include make + + #### Plans (just started.) -- cgit v1.2.3 From 204f282c59a5ecfd99312cc6a686944375889d21 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 15 Jul 2011 22:47:27 +0200 Subject: user/jkoenig/java: OpenJDK/IcedTea upstream submission. --- user/jkoenig/java.mdwn | 7 +++++++ 1 file changed, 7 insertions(+) (limited to 'user/jkoenig') diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn index 01dfae12..935daa11 100644 --- a/user/jkoenig/java.mdwn +++ b/user/jkoenig/java.mdwn @@ -208,6 +208,13 @@ have so far prevented me from testing it. > > --[[jkoenig]] 2011-06-29 +##### Upstream Submission + +On 2011-07-15, *gnu_andrew* talked to us in the #hurd channel (freenode IRC), +who is a maintainer of IcedTea. He's supportive of the porting approach, and +is willing to review and integrate small patches for individual issues (rather +than some huge patchset). Send patches to . + ##### Open Items * [!] [[tschwinge]] to have a look at [[pochu]]'s `file_exec_file_name()` -- cgit v1.2.3 From 9f5e3555f8812ebe4447b7a5519988b4b61275bc Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Fri, 15 Jul 2011 22:47:52 +0200 Subject: user/jkoenig/java/discussion: IRC discussion, 2011-07-13. --- user/jkoenig/java/discussion.mdwn | 454 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 454 insertions(+) (limited to 'user/jkoenig') diff --git a/user/jkoenig/java/discussion.mdwn b/user/jkoenig/java/discussion.mdwn index 266a7bcc..f16d7678 100644 --- a/user/jkoenig/java/discussion.mdwn +++ b/user/jkoenig/java/discussion.mdwn @@ -8,6 +8,11 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license is included in the section entitled [[GNU Free Documentation License|/fdl]]."]]"""]] +[[!toc]] + + +# General + Some [[tschwinge]] comments regarding your proposal. Which is very good, if I may say so again! :-) @@ -70,3 +75,452 @@ either new ones or existing ones, as applicable. * We should probaby open up a *languages for Hurd* section on the web pages ([[!taglink open_issue_documentation]]). + + +# IRC, freenode, #hurd, 2011-07-13 + +[[!tag open_issue_documentation]] + + Yes, I guess so. Maybe start investigating mig because it may + have repercussions on what the best approach would be for some aspects of + the Mach bindings. + I still think that making MIG emit Java code is not too + difficult, once you have the required Java infrastructure (like what + you're writing at the moment). + On the other hand, if there's another approach that you'd like + to use, I'm not trying to force using MIG. + i still have a problem understanding your approach + at which level are your bindings located ? + I expect mig it will be the easiest route, but of course possibly + it won't. + jkoenig: Yeah, be give some high-level to low-level overview? + ok, so + at the very core, low-level, we have a very thin amount of JNI + code to access (proper) system calls. + by "proper" I mean things like mach_task_self, mach_msg and + mach_reply_port, which are actually system calls rather than RPCs to the + kernel. + right + at this level, we manipulate port names as integers, and the + message buffers for mach_msg are raw ByteBuffers (from the java.nio + package) + actually, so-called /direct/ ByteBuffers, which are backed by + memory allocated outside of the Java heap, rather than as a byte[] array + we can retreive the pointer from the JNI code and use the buffer + directly. + (so, good for performance and it's also portable.) + ok + i'm more interested in the higher level bindings :) + ok so, higher up. + design goal from my proposal: "the memory safety of Java should + be maintained and extended to Mach primitives such as port names and + out-of-line memory regions" + so integer port names are not "safe" in the sense that they can + be forged and misused in all kinds of way + which is why I have a layer of Java code whose job is to wrap + this kind of low-level Mach stuff into safe abstractions + and ideally the user should only use these safe abstractions. + (Not to restrict the programmer, but to help him write correct + code.) + right. + so you can't use mach RPCs directly + tschwinge, also to actually restrict them, in a Joe-E / + object-capability context, but that's not the primary concern right now + ;-) + or you force your wrappers to have these abstractions as input + braunr, well, actually at this level you still have Mach RPC + but for instance, port names are encapsulated into "MachPort" + objects which ensure they are handled correcly + As I understand it, you use these abstractions to prepare a + usual mach_msg message, and then invoke mach_msg. + ok + and message buffers are wrapped into "MachMsg" objects which both + help you write the messages into the ByteBuffer and prevent you from + doing funky stuff + and ensure the ports which you send/receive/pseudo-receive after + an error/... are deallocated as required, etc. + what's the interface to use IPC ? + Is MIG doing that, too, I think? (And antrik once found some + error there, which is still to be reviewed...) + braunr, so basically as a user you would be free to use either + one of these layers, or to use MIG-generated classes which would + construct and exchange messages for you using the second (safe) layer. + ok, let's just finish with the low level layer before going + further please + tschwinge, MIG does some type checking on the received message + and saves you the trouble of constructing/parsing them yourself, but I'm + not sure about how mach_msg errors are handled + what are the main methods of MachMsg for example ? + braunr, you may want to have a look at + http://jk.fr.eu.org/hurd-java/doc/html/classorg_1_1gnu_1_1mach_1_1MachMsg.html + right, sorry + grabbed the code at work and forgot here + and also + https://github.com/jeremie-koenig/hurd-java/blob/master/HelloMach.java + which uses it + but roughly, you'd use setRemotePort, setLocalPort, setId to + write your message's header + then use one of the putFoo() methods to add data items to the + message + ok, the mapping with the low level C interface is very clear + that's good for me + the putFoo() methods would write the appropriate type + descriptors, then the actual data. + we can go on with the MiG part if you want :) + right, + so here you may want to look at the UML class diagram from + http://www.bddebian.com/~hurd-web/user/jkoenig/java/proposal/ + so in the C case, mig generates 3 files + a header file which has the prototypes of the mig-generated + stubs, + a *User.c which has their actual implementation + and a *Server.c which handles demultiplexing the incoming + messages and helps with implementing servers. + so we would do something along these lines, more or less: + mig would generate the code for a Java interface in lieu of the + *.h file. + a generated FooUser class would implement this interface by doing + RPC + (so basically you would pass a MachPort object to the + constructor, and then you could use the resulting object to do RPC with + whatever is on the other end) + and the generated FooServer class would do the opposite, + ok + issues with threads ? + you would pass an object implementing the Foo interface to the + constructor, + i'm guessing the demux part may have to create threads, right ? + and the resulting object would handle messages by using the + object you passed. + braunr, right, so that would be more a libports kind of code, + the libports-like library, i see + to which you could pass Server objects (for instance the + FooServer above), and it would handle incoming messages. + how is message content mapped to a java interface ? + this would be determined from the .defs files and MIG would + generate the appropriate code, hopefully. + so the demux part would handle rpc integer identifiers ? + right. + but hm + also mapping .defs files to Java interfaces might prove to be + tricky. data types conversion and all + tschwinge: my mamory is rather hazy. IIRC the issue was that the + MIG-generated stubs deallocate out-of-line port arrays after the + implementation returns, before returning to the dispatcher + i'll just overlook this specific implementation detail + but we could use some annotation-based system if we need to + provide more information to generate the java code. + but the Hurd (or rather glibc) RPC handling also automatically + deallocates everything if an error occurs + so I changed the MIG code to deallocate only when no error occurs + jkoenig: ok, we'll talk about that when there is more progress and + you have a better view of the problem + at that time I was pretty sure that this is a correctly working + solution, but it always seemed questionable conceptually... however, I + wasn't able to come up with a better one, and nobody else commented on it + antrik: shouldn't the hurd be changed not to deallocate something + it didn't allocate in the first place ? + braunr: no, the server has to deallocate stuff before returning to + the client. the request message is destroyed before returning the reply. + jkoenig, braunr: That's what I had in mind where MIG might be a + bit awkward. Then we can indeed either add annotations to the .defs + files, or reproduce them in some other format. That's some work, but + it's mostly a one-time work. + After all, the RPC interface is a binary one, and there may be + more than one API for creating these messages, etc. + jkoenig: actually, at least in the Hurd, server-side and + client-side headers are separate -- so MIG actually creates four files + tschwinge, wrt to annotations I was more thinking about Java + ones, such as: @MIGDefsFile("mach/task.defs") @MIGCType("task_t") public + interface Task { } + antrik, oh, ok, it makes sense. + jkoenig: anything else ? + braunr, nothing that I can think of + ok + tschwinge: I think it would be a *very* bad idea to introduce + redundancy regarding RPC definitions + thanks for the tour :) + (the _request.defs/_reply.defs mess is bad enough...) + did I speak about the "Unsafe" pseudo-exception? that's + interesting :-) + jkoenig: Also, virtual memory abstractions? + jkoenig: you didn't + antrik: Well, then we could create some other super-format. + But that's just a detail IMO. + ok, so wrt virtual memory, a page we received can be wrapped with + some JNI help into a (direct) ByteBuffer object. + deallocating sent pages will be tricky, though. + antrik: To put it this way: for me the .defs files are just one + way of expressing the RPC interfaces' contracts. (At the same time, they + happen to be the actual reference for these, too. But the specification + itself could just as well be a textual one.) + on approach I've been thinking about would be to "wrap" the + ByteBuffer object into an object which has the sole reference to it, so + that when it's deallocated the reference can be replaced with "null", and + further attempts to access the buffer would throw exceptions. + sounds reasonable + but that's still in flux in my head, we may end up needing our + own implementation of ByteBuffer-like objects. + The problem being that there is no mechanism to ``revoke'' an + object once a reference to it has been shared. + right. + A wrapper is one possibility indeed. + tschwinge: they are called interface *definitions* for a reason + :-) + This is a very similar problem as with capabilities when there + is no revoke operation for these, too. + antrik: Yes, because they define MIG's input. :-P + Isn't that what is called a membrane in the capability world? + I do not say that we have to consider the format of the .defs to + be set in stone; but I do insist on using a canonical machine-parsable + source for all language bindings + attenuation + tschwinge, you mean the revokable proxy contruct ? (It's the same + principle indeed) + A common design pattern in object-capability systems: given + one reference of an object, create another reference for a proxy object + with certain security restrictions, such as only permitting read-only + access or allowing revocation. The proxy object performs security checks + on messages that it receives and passes on any that are allowed. Deep + attenuation refers to the case where the same attenuation is applied + transitively to any + objects obtained via the original attenuated object, + typically by use of a "membrane". + http://en.wikipedia.org/wiki/Object-capability_model + Yes. + Good. I understood something. ;-) + antrik: OKAY! :-P + jkoenig: And hopefully the JVM will optimize away all the + additional indirection... :-D + jkoenig: Is there anything more to say about the VM layer? + tschwinge, "hopefully", yes :-) + Like, the data that I'm sharing -- is it untyped, isn't it? + tschwinge, you mean that within the received/sent pages ? + Yes. + But that'S how it is, indeed. + well actually the type descriptor should indicate what they + contain. + I cannot trust anything I receive from externally. + it's most often used for MACH_MSG_TYPE_CHAR items I guess, and it + will be type checked when retreive + Yeah, and that then just *is* arbitrary data, like a block read + from a disk file. + you would have something like: ByteBuffer + MachMsg.getBuffer(MachMsg.Type expected), and MachMsg would check the + type descriptor against that which you specified + Or a packet transmitted over the network. + OK, yes. + jkoenig: in theory ints should be used quite often too. the whole + purpose of the type descriptors is to allow byte order swapping when + messages are passed between hosts with different architecture... + tschwinge, right, except for out-of-line port arrays, which need + to be handled differently obviously. + (which is totally irrelevat for our purposes -- especially since + the actual network IPC code doesn't exist anymore ;-) ) + antrik, oh, interesting + Yes, that was one original idea. + actually my litmus test for what the bindings should be, is you + should be able to implement such a proxy in Java :-) + antrik: And hey, you now have processors that can switch + between different modes during runtime... :-) + (although arguably that's a little bit ambitious) + tschwinge: there should be bits in page tables to indicate the + endianness to use on a page .. :) + Hehe! + jkoenig: Don't worry -- you're already known for ambitious + projects. One more can't hurt. + Also, actually the word size is not something that I've been able + to abstract so far, so I'll be hardcoding little-endian 32 bits for now. + why is that ? + some of the Hurd RPC break the idea anyways BTW + the org.vmmagic package (from Jikes RVM and JNode) could help + with that, but GCJ does not support it unfortunately (not sure about + OpenJDK) + braunr, Java does not allow us to define new unboxed types + jkoenig: does it have its own definition of the word size ? + braunr, nope. + (although, maybe, and also we could use JNI to query it) + even if virtual, i'd expect a machine to have such a defnition + braunr, maybe it has, but basically in Java nothing depends on + the word size + 'int' is 32 bits, 'long' is 64 and that's it. + oh right, i remember most types are fixed size, right ? + right. + if not all + now Jikes RVM's "org.vmmagic" provides an interface to defined + new unboxed types which can depend on the actual word size, but Jikes RVM + is its own JVM so obviously they can use and provide whatever extensions + they need :-) + (but maybe they've implemented them in OpenJDK for bootstrap + purposes, I'm not sure) + I'm missing this detail: where does the word size come into + play here? + anyway, I _could_ indiscriminately use 'long' for port names, and + sparkle the code with word size tests but that would be very clumsy + jkoenig: port names are actually ints :/ + tschwinge, the actual format of the message header and type + descriptors, for instance. + jkoenig: ok, got your point + braunr, by 'long' I mean 64-bits integers (which they are on + 64-bits machines I think?) + :) + jkoenig: port names are as large as the word size + but in C at least, they're int, not long + it doesn't change many things, but you get lots of warnings if you + try with a long :) + What is the reason that port names are an + architecture-dependent word size's width, and not simply 32 bit? + "4 billions of port names should be enough for everyone" :-) + tschwinge: an optimization is to use them as pointers in the + kernel + tschwinge: the machine's native word size is what it can process + most efficiently, and what should be used for most normal + operations... it makes sense to define stuff as int, except for network + communication + jkoenig: Well, yeah, but if you want to communicate with a + peer, you have to agree on the maximum number anyway (not for port names, + though, which are local). + antrik: int isn't the word size everywhere + antrik: the most common type matching the word size is long, at + least on ILP32/LP64 data models + braunr: that's just because some idiots assumed int would always + be 32 bits, and consequently when 64 architectures came up the compiler + guys chickened out ;-) + without int, you wouldn't have a 32 bits type + that's not true for all architectures and/or operating systems + though AFAIK + or a 16 bits one + antrik: windows guys got even more scared, so windows 64 is LLP64 + BTW, I haven't checked, but it's quite possible that 32 bit + numbers are actually preferable even on AMD64... + jkoenig: So, back on track. :-) + jkoenig: You didn't find anything yet in Mach's VM interfaces + as well a MemoryObject, etc., that can't be used/implemented in the Java + world? + antrik: they consume less memory, but don't have much effect on + performance + tschwinge, once we have the basic system calls and the + corresponding abstractions in place, I don't think anything else + fundamentally problematic could possibly show up + braunr: if you really *need* a type of a certain bit size, you + should use stdint types. so not having a 16 or 32 bit type in the + short/int/long canon is *not* an excuse + jkoenig: That speaks for the Mach designers! + antrik: right + tschwinge, on trick is that for instance, mach_task_self would + still be unsafe even if it returned a nicely wrapped Task object, because + you could still wreck your own address space and threads with it. So we + would need the "attenuation" pattern mentionned above to provide a safe + one. + (which would disallow thinks such as the port/thread/vm calls) + jkoenig: you mentioned the unsafe pseudo exception earlier + braunr, right, so the issue is with distinguishing safe from + unsafe methods + braunr: BTW, the Windows guys actually broke a lot of stuff by + fixing long at 32 bits -- this way long doesn't match size_t and pointer + types anymore, which was an assumption that was true for pretty much any + system so far... + jkoenig: Yes. (And again hope for the JVM to optim...) + antrik: that's right :) + antrik: that's LLP64 + antrik: long long and pointers + braunr, so basically the idea is that unsafe methods are declared + as "throws Unsafe" + the effect is that if you use such a method you must either + "throw Unsafe" yourself, + or if you're building a safe abstraction on top of Unsafe + methods, you'll "catch" the "exception" in question to tell the compiler + that it's okay. + it's more or less inspired from the "semantic regimes" idea from + the org.vmmagic paper which is referenced in my original proposal, + only implementing by hijacking the exception checking machinery, + which has a behaviour similar to what we want. + ok + but hmm this seems pretty normal, what's the tricky part ? :) + braunr: The idea is that the programmer explicitly has to + acknowledge if he'S using an unsafe interface. + tschwinge: sounds pretty normal too + braunr, the trick is that you would not usually declare + exceptions which are never actually thrown (and actually since the + compiler does not know it's never thrown, I need to work around it in a + few places) + oh, ok + jkoenig: that's interesting indeed + braunr, the org.vmmagic paper provides an example which uses some + annotations called @UncheckedMemoryAccess and @AssertSafe to the same + effect (which is kind of cleaner), but it would be a headache to + implement without help from the compiler I think (as far as I can tell + the annotation processor would have to inspect the bytecode) + but hm + what's the true problem about this ? + (the paper advocates "high-level low-level programming" and is a + very interesting read I think, + http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.151.5253&rep=rep1&type=pdf, + for what it's worth) + what's wrong if you just declare your methods unsafe and don't + alter anything else ? + Yes, I read it and it is interesting. Unfortunately, it seems + I forgot most of it again... + braunr, declare? alter? + you mean just tag them with an annotation? + just stating a method "throws Unsafe" + braunr, well some compiler will output a warning because they can + tell there's no way the method is going to throw such an exception. + and then some other compiler will complain that my + @SuppressWarnings("unused") does not serve any purpose to them :-) + also, when initializing final fields, I need to work around the + fact that the compiler thinks "Unsafe" might be thrown. + see for instance MachPort.DEAD + jkoenig: ok + braunr, but I'm more than willing to accept this in exchange for + a clear, compiler-enforced materialization of the border between safe an + unsafe code. + actually another question I have is the amount of static typing I + should add to the safe version, for instance should I subclass MachPort + into MachSendRight, MachReceiveRight and so on. I don't want to depart + from the C inteface too much but it could be useful. + jkoenig: can't answer that :) + jkoenig: keep them in mind for later i think + jkoenig: What's the safety concern w.r.t. having MachPort (not) + final? + tschwinge, actually I'm partly wrong in that we only need name() + and a couple other methods to be final + jkoenig: That's what I was thinking. :-) + I though I'm missing something here. + tschwinge, the idea is that the user (ie., the adversary :-) + could extend MachPort and inject their own fake port name into messages + by overriding name() or clear() + Yeah, but if these are final, that's not possible. + right. + And that *should* be enough, I think. + Unless I'm missing something. + I don't think so. Also I hope it is, because as mentionned above + there might be some value in subclassing MachPort. + Yep. + incidentally, declaring the class or the method final will allow + the JVM to inline them I think. + It will help the JVM, yes. It can also figure that out without + final, though. (And may have to de-optimize the code again in case there + are additional classes loaded during run-time.) + jkoenig: The reference counting in MachPort. I think I'm + beginning to understand this. + oh ok + tschwinge, yes the javadoc is maybe a bit obscure so far. + but basically you don't want the port name you acquire to become + invalid before you're done using it. + But how is this different from the C world? + here my goal is to provide some guarantees if you use only safe + methods + like, you can't forge a port name and things like that + so basically it should never be possible to include an invalid + port name in a message if you use only safe methods. + Ah, I see! + Now that does make sense. + but the mechanism in itself is similar to the Hurd port cells and + user_link structures + It's again ``only'' helping the programmer. + right, no object-capability ulterior motives :-) + another assumption which the javadoc does not state yet it that + basically there should be exactly one MachPort object for each mach-level + port name reference (in the sense of mach_port_mod_refs) + Yes, I figured out that bit. -- cgit v1.2.3 From e1c65722d587a623e06f56224d0d8bf7b64059f2 Mon Sep 17 00:00:00 2001 From: Thomas Schwinge Date: Sat, 16 Jul 2011 12:46:09 +0200 Subject: user/jkoenig/java: Link to Doxygen documentation. This allows me to close one browser window. ;-) --- user/jkoenig/java.mdwn | 3 +++ 1 file changed, 3 insertions(+) (limited to 'user/jkoenig') diff --git a/user/jkoenig/java.mdwn b/user/jkoenig/java.mdwn index 935daa11..700f9c4e 100644 --- a/user/jkoenig/java.mdwn +++ b/user/jkoenig/java.mdwn @@ -261,6 +261,9 @@ The code is at . $ jdk=/usr/lib/jvm/java-7-openjdk LD_LIBRARY_PATH=$jdk/jre/lib/i386/jli C_INCLUDE_PATH=$jdk/include make +Doxygen-generated documentation is available at +; or run `make doc` yourself. + #### Plans -- cgit v1.2.3