summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorArne Babenhauserheide <arne_bab@web.de>2011-12-19 06:28:18 +0100
committerArne Babenhauserheide <arne_bab@web.de>2011-12-19 06:28:18 +0100
commit83a6603ed188d746e2871decf85939fb7975b979 (patch)
tree5f671db8fa7e3828322a4d4b1b9cdce9b4bb6ac4
parentd8b7944e910af3fdc1109846698d67738761f85a (diff)
parent6c057cff39ff782e9155c07eee44884cd9c48c9c (diff)
Merge branch 'master' of flubber:~hurd-web/hurd-web
-rw-r--r--capability.mdwn8
-rw-r--r--community/meetings.mdwn4
-rw-r--r--community/meetings/fosdem_2012.mdwn32
-rw-r--r--community/meetings/froscon_2011.mdwn15
-rw-r--r--community/meetings/ghm2011.mdwn14
-rw-r--r--contributing/web_pages/news/moth_next.mdwn52
-rw-r--r--contributing/web_pages/news/skeleton.mdwn33
-rw-r--r--donate.mdwn24
-rw-r--r--glibc.mdwn9
-rw-r--r--glibc/fork.mdwn3
-rw-r--r--hurd.mdwn3
-rw-r--r--hurd/console.mdwn85
-rw-r--r--hurd/dde/guide.mdwn2
-rw-r--r--hurd/debugging/rpctrace.mdwn37
-rw-r--r--hurd/io_path.mdwn6
-rw-r--r--hurd/libpager.mdwn16
-rw-r--r--hurd/libports.mdwn19
-rw-r--r--hurd/libthreads.mdwn28
-rw-r--r--hurd/running/gnu/universal_package_manager.mdwn1
-rw-r--r--hurd/translator.mdwn1
-rw-r--r--hurd/translator/hello.mdwn14
-rw-r--r--hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn37
-rw-r--r--hurd/virtual_file_system/discussion.mdwn39
-rw-r--r--media_appearances.mdwn5
-rw-r--r--microkernel/mach/gnumach.mdwn5
-rw-r--r--microkernel/mach/gnumach/memory_management.mdwn22
-rw-r--r--news/2011-q3.mdwn200
-rw-r--r--open_issues/anatomy_of_a_hurd_system.mdwn28
-rw-r--r--open_issues/code_analysis.mdwn5
-rw-r--r--open_issues/ext2fs_page_cache_swapping_leak.mdwn88
-rw-r--r--open_issues/fakeroot_exit_0.mdwn (renamed from open_issues/fakeroot-tcp_vs_eintr.mdwn)20
-rw-r--r--open_issues/glibc.mdwn51
-rw-r--r--open_issues/glibc/t/tls.mdwn66
-rw-r--r--open_issues/gnumach_memory_management.mdwn202
-rw-r--r--open_issues/libmachuser_libhurduser_rpc_stubs.mdwn50
-rw-r--r--open_issues/mig_portable_rpc_declarations.mdwn58
-rw-r--r--open_issues/mission_statement.mdwn12
-rw-r--r--open_issues/multithreading.mdwn2
-rw-r--r--open_issues/page_cache.mdwn73
-rw-r--r--open_issues/perl.mdwn50
-rw-r--r--open_issues/posix_fadv_volatile.mdwn16
-rw-r--r--open_issues/robustness.mdwn64
-rw-r--r--open_issues/syslog.mdwn27
-rw-r--r--open_issues/translator_stdout_stderr.mdwn32
-rw-r--r--open_issues/translators_set_up_by_untrusted_users.mdwn3
-rw-r--r--shortcuts.mdwn7
-rw-r--r--user/Maksym_Planeta.mdwn142
-rw-r--r--user/musial.mdwn10
48 files changed, 1488 insertions, 232 deletions
diff --git a/capability.mdwn b/capability.mdwn
index ddadf137..7219cdce 100644
--- a/capability.mdwn
+++ b/capability.mdwn
@@ -11,7 +11,7 @@ License|/fdl]]."]]"""]]
A capability is a protected reference. It is a reference in that
it designates an object; it is protected in that in cannot be
-forged. A capabilities both designates the object it refers to and
+forged. A capability both designates the object it refers to and
carries the authority to manipulate it.
By binding [[designation]] and [[authorization]] together, capabilities
@@ -25,7 +25,7 @@ to protect against A hijacking his authority. (This problem is
refused to the [[confused_deputy]] problem.) Also, since A likely
sent a string to identify the file to B, the identifier lacks a
[[naming_context]] and therefore may resolve to a different object
-than A intended. Be ensuring that [[designation]] and [[authorization]] are
+than A intended. By ensuring that [[designation]] and [[authorization]] are
always bound together, these problems are avoided.
Capability-based system architectures strive to meet the *principle of least
@@ -39,8 +39,8 @@ individually); attenuation design pattern, membranes
(wikipedia_object-capability_model)?
-->
-A capability mechanism is typically implemented in software my the operating
-system kernel (typically a [[microkernel]]. The computing cost (as compared to
+A capability mechanism is typically implemented in software by the operating
+system kernel (typically a [[microkernel]]). The computing cost (as compared to
a hardware implementation) is neglectable.
diff --git a/community/meetings.mdwn b/community/meetings.mdwn
index 6c15d144..4ae52a1a 100644
--- a/community/meetings.mdwn
+++ b/community/meetings.mdwn
@@ -13,15 +13,17 @@ License|/fdl]]."]]"""]]
# Upcoming
- * [[GNU Hackers Meeting, 2011, Paris|ghm2011]]
## In the Future
+ * [[FOSDEM_2012]]
* [[Self-organised]]
# Past
+ * [[FrOSCon_2011]]
+ * [[GNU Hackers Meeting, 2011, Paris|ghm2011]]
* [[FOSDEM_2011]]
* [[DebConf10]]
* [[GNU Hackers Meeting, 2010, Den Haag|ghm2010]]
diff --git a/community/meetings/fosdem_2012.mdwn b/community/meetings/fosdem_2012.mdwn
new file mode 100644
index 00000000..c8ee511b
--- /dev/null
+++ b/community/meetings/fosdem_2012.mdwn
@@ -0,0 +1,32 @@
+[[!meta copyright="Copyright © 2006, 2007, 2008, 2009, 2010, 2011 Free Software
+Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!meta title="FOSDEM 2012"]]
+
+<http://fosdem.org/2012>
+
+FOSDEM will take place on February 4th/5th at the Université Libre de
+Bruxelles.
+
+
+# Who and When
+
+[[!table class="table_style_1" data="""
+"Name","Attending","Arrival","Return","Share room with us"
+"[[Maksym Planeta]]","no"
+"Olaf Buddenhagen","most likely","","","yes"
+"[[Thomas Schwinge|tschwinge]]","will try to","","","yes"
+"""]]
+
+
+# Multiserver, Microkernel-Based Operating Systems Devroom
+
+[Announcement](http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00137.html).
diff --git a/community/meetings/froscon_2011.mdwn b/community/meetings/froscon_2011.mdwn
new file mode 100644
index 00000000..b15140d6
--- /dev/null
+++ b/community/meetings/froscon_2011.mdwn
@@ -0,0 +1,15 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!meta title="FrOSCon, 2011, Sankt Augustin, Germany"]]
+
+<http://www.froscon.de/>
+
+ * [Arch Hurd booth](http://www.froscon.de/en/exhibitors/projekte.html#c1413)
diff --git a/community/meetings/ghm2011.mdwn b/community/meetings/ghm2011.mdwn
index 7a2df8a0..8e77d500 100644
--- a/community/meetings/ghm2011.mdwn
+++ b/community/meetings/ghm2011.mdwn
@@ -11,3 +11,17 @@ License|/fdl]]."]]"""]]
[[!meta title="GNU Hackers Meeting, 2011, Paris"]]
<http://www.gnu.org/ghm/2011/paris/>
+
+ * {{$thibault_hurd}}
+
+
+[[!ymlfront data="""
+
+thibault_hurd:
+
+ "presentation by Samuel Thibault: [*GNU/Hurd, aka. Extensibility from the
+ Ground*](http://www.gnu.org/ghm/2011/paris/#outline-container-2-5)
+ ([slides](http://www.gnu.org/ghm/2011/paris/slides/samuel-thibault-hurd.pdf),
+ [video](http://audio-video.gnu.org/video/ghm2011/Samuel_Thibault-GNU_Hurd.ogv))"
+
+"""]]
diff --git a/contributing/web_pages/news/moth_next.mdwn b/contributing/web_pages/news/moth_next.mdwn
index 78db041d..029c6769 100644
--- a/contributing/web_pages/news/moth_next.mdwn
+++ b/contributing/web_pages/news/moth_next.mdwn
@@ -27,6 +27,31 @@ else="
This month [hurd hacker] [item]
+<http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00079.html> -- Bouju
+Alain submitted a patch to suport cpuinfo in the /proc interface
+
+rbraun committed the last patch to mplanetas branch of the slab allocator
+work, for integration.
+
+IRC, freenode, #hurd, 2011-11-14:
+
+Features:
+
+ (22:30:39) braunr: there shouldn't be any noticeable difference with the
+ master branch
+ (22:30:46) braunr: a bit less fragmentation
+ (22:30:55) braunr: more memory can be reclaimed by the VM system
+ (22:31:02) braunr: there are debugging features
+ (22:31:06) braunr: it's SMP ready
+ (22:31:15) braunr: and overall cleaner than the zone allocator
+ (22:31:31) braunr: although a bit slower on the free path (because of
+ what's performed to reduce fragmentation)
+ (22:32:42) braunr: but even "slower" here is completely negligible
+
+**New porter box: exodar***
+
+Add stuff here? (This is already in q3)
+
Also …
[our hackers] …
@@ -37,22 +62,23 @@ Additionally …
And …
-So if you want to [reason for contibuting to the Hurd] please [[get_in_contact|contact_us]] - and maybe grab [[our_source_repos|source_repositories]].
+So if you want to [reason for contibuting to the Hurd],
+please [[get in contact|contact_us]] -- and maybe already grab the [[source
+code|source_repositories]].
-------
+---
-The **GNU Hurd** is the GNU project's replacement for the Unix kernel.
-It is a collection of servers that run on the Mach microkernel to
-implement file systems, network protocols, file access control, and
-other features that are implemented by the Unix kernel or similar
-kernels (such as Linux).
-[[More_detailed|hurd/documentation]].
+The **GNU Hurd** is the GNU project's replacement for the Unix kernel. It is a
+collection of servers that run on the Mach microkernel to implement file
+systems, network protocols, file access control, and other features that are
+implemented by the Unix kernel or similar kernels (such as Linux). [[More
+detailed|hurd/documentation]].
-**GNU Mach** is the microkernel upon which GNU Hurd is based. It
-offers Inter Process Communication (IPC) which the Hurd uses to define
-interfaces for implementing the services an operating system needs
-from a full-featured kernel.
-[[Read_more|microkernel/mach/gnumach]]
+**GNU Mach** is the microkernel upon which a GNU Hurd system is based. It
+provides an Inter Process Communication (IPC) mechanism that the Hurd uses to
+define interfaces for implementing in a distributed multi-server fashion the
+services a traditional operating system kernel provides. [[More
+detailed|microkernel/mach/gnumach]].
<!--see [[contributing/web_pages/news/writing_the_moth]] for additional information on writing the MotH.-->
diff --git a/contributing/web_pages/news/skeleton.mdwn b/contributing/web_pages/news/skeleton.mdwn
index 78db041d..d78d1b5a 100644
--- a/contributing/web_pages/news/skeleton.mdwn
+++ b/contributing/web_pages/news/skeleton.mdwn
@@ -37,22 +37,23 @@ Additionally …
And …
-So if you want to [reason for contibuting to the Hurd] please [[get_in_contact|contact_us]] - and maybe grab [[our_source_repos|source_repositories]].
-
-------
-
-The **GNU Hurd** is the GNU project's replacement for the Unix kernel.
-It is a collection of servers that run on the Mach microkernel to
-implement file systems, network protocols, file access control, and
-other features that are implemented by the Unix kernel or similar
-kernels (such as Linux).
-[[More_detailed|hurd/documentation]].
-
-**GNU Mach** is the microkernel upon which GNU Hurd is based. It
-offers Inter Process Communication (IPC) which the Hurd uses to define
-interfaces for implementing the services an operating system needs
-from a full-featured kernel.
-[[Read_more|microkernel/mach/gnumach]]
+So if you want to [reason for contibuting to the Hurd],
+please [[get in contact|contact_us]] -- and maybe already grab the [[source
+code|source_repositories]].
+
+---
+
+The **GNU Hurd** is the GNU project's replacement for the Unix kernel. It is a
+collection of servers that run on the Mach microkernel to implement file
+systems, network protocols, file access control, and other features that are
+implemented by the Unix kernel or similar kernels (such as Linux). [[More
+detailed|hurd/documentation]].
+
+**GNU Mach** is the microkernel upon which a GNU Hurd system is based. It
+provides an Inter Process Communication (IPC) mechanism that the Hurd uses to
+define interfaces for implementing in a distributed multi-server fashion the
+services a traditional operating system kernel provides. [[More
+detailed|microkernel/mach/gnumach]].
<!--see [[contributing/web_pages/news/writing_the_moth]] for additional information on writing the MotH.-->
diff --git a/donate.mdwn b/donate.mdwn
index 938f950b..6293633e 100644
--- a/donate.mdwn
+++ b/donate.mdwn
@@ -91,30 +91,6 @@ Please don't hesitate to ask [[Thomas Schwinge|tschwinge]] if you need help.
Continue to explore the [[list of open bounties|tag/bounty]].
-# g10 Code Maintenance Points
-
-If you've got more money on hand than hacking time, you might consider buying
-some [maintenance points](http://g10code.com/products.html#maintpoints) (EUR 10 a point) to
-help the Hurd along. From the [g10 Code](http://www.g10code.com/products.html)
-site:
-
-> Hurd Maintenance Points are special: Some of our employees are well known
-> Hurd hackers in their spare time; collected points for this program will be
-> given to them in form of paid time.
-
-See also this related
-[mailing-list](http://mail.gnu.org/archive/html/help-hurd/2003-04/msg00044.html)
-thread.
-
-And for further motivation, some words of wisdom from Marcus Brinkmann:
-
-> By the way, if you are more on the speculating side, then it can't harm to
-> just buy one or two maintenance points. That means that at some time I get
-> an incentive to start the hacking, and there is a chance that when I start I
-> don't stop for a while, and just continue on my private time (as I did for
-> the last five years, if I might add that ;).
-
-
# Hurd Developer Meetings
Another possibility is to meet with the Hurd developers at a
diff --git a/glibc.mdwn b/glibc.mdwn
index 78d44df0..dad3c427 100644
--- a/glibc.mdwn
+++ b/glibc.mdwn
@@ -23,8 +23,17 @@ repository|source_repositories/glibc]].
# Specifics
+
+## Ports
+
Porting glibc to a specific architecture is non-trivial.
+The main port is x86, which is somewhat complete and is maintained. There is
+an incomplete and unmaintained port for PowerPC. There [[!message-id
+desc="have" "20111129161104.25123.qmail@sourceware.org"]] [[!message-id
+desc="been" "Pine.LNX.4.64.1111291611170.26895@digraph.polyomino.org.uk"]]
+incomplete and largely unmaintained ports for Alpha and MIPS in glibc-ports.
+
## [[Hurd-specific Port|hurd/glibc]]
diff --git a/glibc/fork.mdwn b/glibc/fork.mdwn
index 9417106d..12ca2d19 100644
--- a/glibc/fork.mdwn
+++ b/glibc/fork.mdwn
@@ -53,6 +53,9 @@ they have patches for software packages, to avoid using `fork` followed by
* Can we/why can't we use the concept of *inherited ports
array*s/`mach_ports_register` ([[!taglink open_issue_glibc]])?
+ * GNUnet `vfork` signal race issue: [[!message-id
+ "87r50ww6m4.fsf@kepler.schwinge.homeip.net"]].
+
## Related
diff --git a/hurd.mdwn b/hurd.mdwn
index 255cd3d7..11995c87 100644
--- a/hurd.mdwn
+++ b/hurd.mdwn
@@ -65,7 +65,6 @@ in the *unstable* branch of the Debian archive.
## Common Problems
-* [[Console]]
* [[Xfree86]] -- [[DebianX]] -- [[DebianXorg]]
* [[GNUstep]]
* [[XattrHurd]]: Setting translators under GNU/Linux
@@ -94,9 +93,11 @@ in the *unstable* branch of the Debian archive.
* [[libnetfs]] -- short introductory material
* [[libdiskfs]]
* [[libihash]]
+ * [[libthreads]]
* [[libpthread]]
* [[IO_Path]]
* [[Porting]]
* [[Debugging]]
* [Hurd Sourcecode Reference](http://www.htu.tugraz.at/~past/hurd/global/): Searchable and browsable index of the code.
* [[Networking]]
+* [[Console]]
diff --git a/hurd/console.mdwn b/hurd/console.mdwn
index 4f976efd..f7230011 100644
--- a/hurd/console.mdwn
+++ b/hurd/console.mdwn
@@ -1,3 +1,88 @@
+[[!meta copyright="Copyright © 2002, 2003, 2004, 2005, 2006, 2007, 2009, 2011
+Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+The Hurd console's implementation is broken into two pieces each running on
+it's own process, the console client and server.
+
+The console client is also split into modules (input driver, display driver,
+speaker, ...) but they all run in the same process.
+
+The console server puts itself as a translator on top of `/dev/vcs` folder
+presenting the following hierarchy:
+
+ + /dev/vcs
+ \
+ +- 1
+ \
+ +- console
+ +- input
+ +- display
+ +- ..
+ +- n
+
+where the numbered nodes represent virtual consoles and their contents are all
+alike.
+
+As the following graph shows, the console, input and display nodes are the
+interfaces used by the terminal server, input driver and display drivers
+respectively.
+
+ +------------------+ +-----------------+
+ | Input driver | | Terminal Server |
+ | | | |
+ | pc-kbd | | |
+ +------------------+ +-----------------+
+ | _cons_vcons_input |
+ | writes to reads |
+ | vcs/i/input vcs/i/console |
+ | +-----------------+ |
+ | | Console Server | |
+ | | /hurd/console | |
+ | input_enqueue | --------------- | input_dequeue |
+ +--------------->| Input Queue |>---------------+
+ | --------------- |
+ +--------------->| Output Buffer |>---------------+
+ | +-----------------+ |
+ | |
+ | writes reads |
+ | vcs/i/console vcs/i/display |
+ | |
+ +----------------+ +-----------------+
+ | Teminal Server | | Display driver |
+ | | | |
+ | /hurd/term | | vga |
+ +----------------+ +-----------------+
+
+The input driver takes scancodes from the in-kernel kbd queue, translates them
+into characters and writes them to the input node. Then the terminal server
+reads the console node taking the characters out of the console server.
+
+Each of theese actions is actually an RPC handled by the translator on
+`/dev/vcs`. Writes to input nodes are handled by calling `input_enqueue` to
+put the character into a queue. And reads from console nodes are handled by
+calling `input_dequeue` which takes out charecters from the queue and gives
+them to the reader.
+
+It's important to note here that both `input_enqueue` and `input_dequeue` are
+blocking operations and a blocked `input_dequeue` necessarily needs an
+`input_enqueue` call to continue.
+
+[[RPC]]s are handled by the console server with the help of [[hurd/libports]]'
+`ports_manage_multithreaded` API.
+
+
+---
+
+/!\ old content; [[!taglink open_issue_documentation]]: cleanup needed.
+
The below is a reworked version of Marcus Brinkmann's [letter to the debian-hurd list](http://lists.debian.org/debian-hurd/2002/debian-hurd-200209/msg00054.html). It describes how to setup the new console server for the Hurd. I am testing this right now, so this document is a work in progress.
-- [[Main/JoachimNilsson]] - 21 Jan 2003
diff --git a/hurd/dde/guide.mdwn b/hurd/dde/guide.mdwn
index a3c08754..6a83519c 100644
--- a/hurd/dde/guide.mdwn
+++ b/hurd/dde/guide.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2010,2011 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
diff --git a/hurd/debugging/rpctrace.mdwn b/hurd/debugging/rpctrace.mdwn
index f7136056..fd24f081 100644
--- a/hurd/debugging/rpctrace.mdwn
+++ b/hurd/debugging/rpctrace.mdwn
@@ -52,6 +52,43 @@ See `rpctrace --help` about how to use it.
<antrik> note that there is a number of known bugs in rpctrace, for which zhengda has sent patches... though I haven't reviewed all of them I think
<antrik> there are some nasty Mach operations that are really hard to proxy -- but I don't think the auth mechanism needs any of these...
+* IRC, freenode, #hurd, 2011-11-04
+
+ [[!taglink open_issue_documentation]]
+
+ <mcsim> hello. Are there any documentation about understanding output
+ of rpctrace?
+ <braunr> no
+ <braunr> you should read the source code, best doc available
+ <braunr> if you have too many numbers and almost no symbolc names,
+ you're lacking rpc definition lists
+ <braunr> check that the gnumach-common package is installed, as it
+ provides the gnumach definitions
+ <braunr> (the glibc ones are almost always available)
+ <braunr> with those two, you should be fine for the beginning
+ <mcsim> gnumach-common is installed. And what is the name for glibc
+ package for gnumach definitions.
+ <mcsim> Also I'm using libraries specified by LD_LIBRARY_PATH. Does it
+ make influence on absence of symbolic names?
+ <braunr> no
+ <braunr> rpctrace --help
+ <braunr> see the --rpc-list=FILE option
+ <braunr> the default lists are in /usr/share/msgids/, with the .msgids
+ extension
+ <braunr> $ dpkg -S msgids
+ <braunr> gnumach-common: /usr/share/msgids/gnumach.msgids
+ <braunr> hurd: /usr/share/msgids/hurd.msgids
+ <braunr> ok, glibc has none, it's the hurd
+ <braunr> for more details about the output, read the source code
+ <braunr> it shouldn't be that hard to grasp
+ <mcsim> -I /usr/share/msgids helped
+ <mcsim> thank you
+ <braunr> it shouldn't have, it's the default path
+ <mcsim> but symbolic names appeared
+ <braunr> well, that's weird :)
+ <pinotree> braunr: the output of rpctrace --help should tell the
+ default dir for msgids
+
# See Also
diff --git a/hurd/io_path.mdwn b/hurd/io_path.mdwn
index 492edffe..c47b5dca 100644
--- a/hurd/io_path.mdwn
+++ b/hurd/io_path.mdwn
@@ -46,9 +46,9 @@ Back in `_Xio_read`.
If the 2048 byte buffer is not decided to be used (out-of-line case or bigger
than 2048 bytes case; server decides to instead provide a new memory region),
-the [[`dealloc`|microkernel/mach/mig/dealloc]] flag is being set, which causes
-Mach to unmap that memory region from the server's address space, i.e., doing a
-memory *move* from the server to the client.
+the [[`dealloc`|microkernel/mach/mig/documentation/dealloc]] flag is being set,
+which causes Mach to unmap that memory region from the server's address space,
+i.e., doing a memory *move* from the server to the client.
Leave server-side RPC stub `_Xio_read`.
diff --git a/hurd/libpager.mdwn b/hurd/libpager.mdwn
index 99f28f2a..d844743d 100644
--- a/hurd/libpager.mdwn
+++ b/hurd/libpager.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2007, 2008, 2010 Free Software Foundation,
+[[!meta copyright="Copyright © 2007, 2008, 2010, 2011 Free Software Foundation,
Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
@@ -6,8 +6,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this
document under the terms of the GNU Free Documentation License, Version 1.2 or
any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
-is included in the section entitled
-[[GNU Free Documentation License|/fdl]]."]]"""]]
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
Mach's [[microkernel/mach/external_pager_mechanism]].
@@ -16,6 +16,16 @@ Mach [[microkernel/mach/IPC]]'s [[microkernel/mach/ipc/sequence_numbering]].
[GNU Hurd Reference Manual: 4.2 Pager
Library](http://www.gnu.org/software/hurd/doc/hurd_5.html#SEC32).
+
+# Writeback: Writing Out Dirty Pages
+
+
+## Related
+
+ * LWN, Jonathan Corbet, [*No-I/O dirty
+ throttling*](http://lwn.net/Articles/456904/), 2011-08-31.
+
+
# Open Issues
* [[open_issues/linux_vmsig]]
diff --git a/hurd/libports.mdwn b/hurd/libports.mdwn
index 28274338..6f2cd46d 100644
--- a/hurd/libports.mdwn
+++ b/hurd/libports.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]]
[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -18,3 +18,20 @@ provide an interface independently of the underlying [[microkernel]].
*libports* does not itself depend on *[[libthreads]]*, but the appropriate
threading hooks are used if present, that is if *[[libthreads]]* is used by
another component.
+
+
+# Message Processing
+
+## `ports_manage_multithreaded`
+
+When a message is recieved, the thread acting as receiver checks if any other
+thread is also waiting for requests. If there is none, a new thread is
+spawned. Thus, the current thread continues processing the message while the
+newly created thread starts listening for new ones. ([[!taglink
+open_issue_hurd]]: [[open_issues/multithreading]].)
+
+Also, there are configurable timeouts for [[translator]]s who want to go away
+when they are not used. ([[!taglink open_issue_hurd]]: there used to be bugs
+in this area, [[!message-id "87hev152we.fsf@becket.becket.net"]], but it may be
+fixed as of [[!message-id "20111030210045.GA4983@myhost"]],
+[[!GNU_Savannah_Git_hurd_hurd 9b5429e834cde56f73b8ff605e36afc7d9bb6e1b]].)
diff --git a/hurd/libthreads.mdwn b/hurd/libthreads.mdwn
new file mode 100644
index 00000000..8b1a97e6
--- /dev/null
+++ b/hurd/libthreads.mdwn
@@ -0,0 +1,28 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+`libthreads` a.k.a. C threads.
+
+
+# Internals
+
+
+## Threads' Death
+
+C threads death doesn't actually free the thread's stack (and maybe not the
+associated Mach ports either). That's because there's no way to free the stack
+after the thread dies (because the thread of control is gone); the stack needs
+to be freed by something else, and there's nothing convenient to do it. There
+are many ways to make it work.
+
+However, it isn't really a leak, because the unfreed resources do get used for
+the next thread. So the issue is that the shrinkage of resource consumption
+never happens, but it doesn't grow without bounds; it just stays at the maximum
+even if the current number of threads is lower.
diff --git a/hurd/running/gnu/universal_package_manager.mdwn b/hurd/running/gnu/universal_package_manager.mdwn
index 58841b02..bf1b92e0 100644
--- a/hurd/running/gnu/universal_package_manager.mdwn
+++ b/hurd/running/gnu/universal_package_manager.mdwn
@@ -155,3 +155,4 @@ To join the project just list your name below.
8. Abhradip Mukherjee
9. Ermenegildo Fiorito
10. Oltion Doda
+ 11. Russell James
diff --git a/hurd/translator.mdwn b/hurd/translator.mdwn
index bf7af3ce..3527267f 100644
--- a/hurd/translator.mdwn
+++ b/hurd/translator.mdwn
@@ -80,6 +80,7 @@ The [[concept|concepts]] of translators creates its own problems, too:
# Existing Translators
+* [[hello]]
* [[auth]]
* [[exec]]
* [[pfinet]]
diff --git a/hurd/translator/hello.mdwn b/hurd/translator/hello.mdwn
new file mode 100644
index 00000000..bd56cd76
--- /dev/null
+++ b/hurd/translator/hello.mdwn
@@ -0,0 +1,14 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+The *hello* translator is an example of a simple [[libtrivfs]]-based one-node
+[[translator]]. It is shipped as part of the [[Hurd source code
+repository|source_repositories]], and exists in a single-threaded and a
+multi-threaded variant.
diff --git a/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn
index a9317c21..5228515f 100644
--- a/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn
+++ b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn
@@ -233,3 +233,40 @@ See also: [[open_issues/resource_management_problems/pagers]].
I have never looked at it.
[[open_issues/mach-defpager_vs_defpager]].
+
+
+# IRC, freenode, #hurd, 2011-11-08
+
+ <mcsim> who else uses defpager besides tmpfs and kernel?
+ <braunr> normally, nothing directly
+ <mcsim> than why tmpfs should use defpager?
+ <braunr> it's its backend
+ <braunr> backign store rather
+ <braunr> the backing store of most file systems are partitions
+ <braunr> tmpfs has none, it uses the swap space
+ <mcsim> if we allocate memory for tmpfs using vm_allocate, will it be able
+ to use swap partition?
+ <braunr> it should
+ <braunr> vm_allocate just maps anonymous memory
+ <braunr> anonymous memory uses swap space as its backing store too
+ <braunr> but be aware that this part of the vm system is known to have
+ deficiencies
+ <braunr> which is why all mach based implementations have rewritten their
+ default pager
+ <mcsim> what kind of deficiencies?
+ <braunr> bugs
+ <braunr> and design issues, making anonymous memory fragmentation horrible
+ <antrik> mcsim: vm_allocate doesn't return a memory object; so it can't be
+ passed to clients for mmap()
+ <mcsim> antrik: I use vm_allocate in pager_read_page
+ <antrik> mcsim: well, that means that you have to actually implement a
+ pager yourself
+ <antrik> also, when the kernel asks the pager to write back some pages, it
+ expects the memory to become free. if you are "paging" to ordinary
+ anonymous memory, this doesn't happen; so I expect it to have a very bad
+ effect on system performance
+ <antrik> both can be avoided by just passing a real anonymous memory
+ object, i.e. one provided by the defpager
+ <antrik> only problem is that the current defpager implementation can't
+ really handle that...
+ <antrik> at least that's my understanding of the situation
diff --git a/hurd/virtual_file_system/discussion.mdwn b/hurd/virtual_file_system/discussion.mdwn
new file mode 100644
index 00000000..9e12d01e
--- /dev/null
+++ b/hurd/virtual_file_system/discussion.mdwn
@@ -0,0 +1,39 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_documentation]]
+
+IRC, freenode, #hurd, 2011-11-12:
+
+ <sea4ever> So hurd implements a 'transparent translator' somewhere which
+ just passes all IO calls to the posix IO I'm used to? (i.e. read, write,
+ open, close, etc.?)
+ <youpi> it's the normal way of operation
+ <youpi> glibc's read() doesn't do a system call, it always does an RPC to
+ the underlying translator
+ <youpi> be it ext2fs for /, or your foobarfs for your node
+ <sea4ever> Ok that makes sense. How does one program know which translator
+ it should refer to though?
+ <sea4ever> the read() call magically knows which process to invoke?
+ <youpi> the / translator is always known
+ <youpi> and then you ask /'s translator about /home, then /home/you, then
+ /home/you/foobar
+ <youpi> it tells you which other translator tyou have to contact
+ <youpi> that's on open
+ <sea4ever> It's a tree! Ok.
+ <youpi> the notion of fd is then simply knowing the translator
+ <sea4ever> Right. 'file descriptor' is now 'translator address descriptor'
+ maybe.
+ <youpi> it's glibc which knows about FDs, nothing else knows
+ <youpi> yes
+ <youpi> actually an RPC port, simply
+ <sea4ever> I want to try out the new RPC mechanism that mach implements
+ <youpi> err, which "new" RPC ?
+ <youpi> mach's RPCs are very old actually :)
diff --git a/media_appearances.mdwn b/media_appearances.mdwn
index b52b80bf..c005c381 100644
--- a/media_appearances.mdwn
+++ b/media_appearances.mdwn
@@ -18,6 +18,11 @@ A lot of stuff is missing here.
# 2011
+## August
+
+ * GNU Hackers Meeting in Paris: {{$community/meetings/ghm2011#thibault_hurd}}
+
+
## July
diff --git a/microkernel/mach/gnumach.mdwn b/microkernel/mach/gnumach.mdwn
index d9ff6535..edd0cfdb 100644
--- a/microkernel/mach/gnumach.mdwn
+++ b/microkernel/mach/gnumach.mdwn
@@ -9,7 +9,10 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
-GNU Mach is the microkernel that the GNU/Hurd system is based on.
+GNU Mach is the microkernel upon which a GNU Hurd system is based. It provides
+an Inter Process Communication (IPC) mechanism that the Hurd uses to define
+interfaces for implementing in a distributed multi-server fashion the services
+a traditional operating system kernel provides.
It is maintained by the Hurd developers for the GNU project and remains
compatible with [[Mach]] 3.0.
diff --git a/microkernel/mach/gnumach/memory_management.mdwn b/microkernel/mach/gnumach/memory_management.mdwn
index 43b99d83..ca2f42c4 100644
--- a/microkernel/mach/gnumach/memory_management.mdwn
+++ b/microkernel/mach/gnumach/memory_management.mdwn
@@ -80,3 +80,25 @@ IRC, freenode, #hurd, 2011-06-09
<braunr> wow
<braunr> i remember the linux support for 4G/4G split when there was enough
RAM to fill the kernel space with struct page entries
+
+
+IRC, freenode, #hurd, 2011-11-12
+
+ <youpi> well, the Hurd doesn't "artificially" limits itself to 1.5GiB
+ memory
+ <youpi> i386 has only 4GiB addressing space
+ <youpi> we currently chose 2GiB for the kernel and 2GiB for the userspace
+ <youpi> since kernel needs some mappings, that leaves only 1.5GiB usable
+ physical memory
+ <sea4ever`> Hm? 2GiB for kernel, 2GiB for userspace, 500MiB are used for
+ what?
+ <youpi> for mappings
+ <youpi> such as device iomap
+ <youpi> contiguous buffer allocation
+ <youpi> and such things
+ <sea4ever`> Ah, ok. You map things in kernel space into user space then.
+ <youpi> linux does the same without the "bigmem" support
+ <youpi> no, just in kernel space
+ <youpi> kernel space is what determines how much physical memory you can
+ address
+ <youpi> unless using the linux-said-awful "bigmem" support
diff --git a/news/2011-q3.mdwn b/news/2011-q3.mdwn
index de7b869b..c1a78319 100644
--- a/news/2011-q3.mdwn
+++ b/news/2011-q3.mdwn
@@ -8,14 +8,10 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
-<!-- Date when the news item is (to be) pulished (important for RSS feeds).
-Will be set by tschwinge when publishing.
-[[!meta date="YYYY-MM-DD HH:MM UTC"]]
--->
+[[!meta date="2011-11-17 14:15 UTC"]]
-<!-- This is just a skeleton. Use it to create a new MotH. -->
-
-A quarter of the Hurd: *Arch with DDE*, *Debian boxes*, *GHM talk*, *GNU Mach fixes* and *GSoC: Java*.
+A quarter of the Hurd: *Arch Hurd with DDE*, *Debian boxes*, *GHM talk* and
+*GSoC: Java*.
[[!if test="included()" then="""[[!toggle id=full_news
text="Details."]][[!toggleable id=full_news text="[[!paste id=full_news]]"]]"""
else="
@@ -23,106 +19,98 @@ else="
[[!cut id="full_news" text="""
-<!--basic structure of a MotH entry. Adapt, reduce and add points as needed. At the end, try to make the text flow as a unified whole.-->
-
-In the third quarter of 2011, the Arch Hurd Hackers [packaged DDE](http://www.archhurd.org/news/22/),
-so a subset of Linux 2.6 drivers can now be compiled on Arch Hurd to
-run in userspace. At the time of writing it supports network cards,
-while other driver-types still need their interfaces ported. Also they
-had
-[a booth at FrOSCon](http://www.froscon.de/en/exhibitors/projekte.html)
-and
-[released a new Arch Hurd LiveCD](http://www.archhurd.org/news/24/),
-so new users can easily test the current state of the Arch flavor of
-the Hurd.
-
-Also Richard Braun contributed new Debian and KVM-based
-[[buildd,_porterbox_and_public_box|public_hurd_boxen]], making it
-easier to test the Hurd without much setup as well as improving debian
-packaging.
-
-Samuel Thibault wrote a new
-[Bits from the Debian GNU/Hurd porters](http://lists.debian.org/debian-devel-announce/2011/07/msg00002.html)
-to keep the Debian Folks up to date with the results of our work. And
-these are quite good: Thanks to the relentless work of our porters,
-you can now use
-[70% of debian packages with the Hurd](https://buildd.debian.org/stats/graph-big.png),
-so we’re coming closer towards
-[getting Hurd into Debian as a release arch](http://wiki.debian.org/Debian_GNU/Hurd). If
-you can port debian packages and want to help the Hurd, this is the
-perfect time to get in contact and
-[port your favorite missing package](http://www.debian.org/ports/hurd/hurd-devel-debian)
-to the Hurd.
-
-A different type of status update was delivered by Samuel Thibault on
-the GNU Hacker Meeting (GHM). Since the videos and slides from the GNU
-Hacker Meeting 2011 in Paris are
-[online](http://www.gnu.org/ghm/2011/paris/), now, we hope you enjoy
-his talk on
-[GNU/Hurd, aka. Extensibility from the Ground (video)](http://audio-video.gnu.org/video/ghm2011/Samuel_Thibault-GNU_Hurd.ogv)
-([slides](http://www.gnu.org/ghm/2011/paris/slides/samuel-thibault-hurd.pdf)). He
-explains nicely how the simple concept of translators gives power to
-non-priviledged and casual users (once we get some of these :) )
-without security implications, and how Sub-Hurds and Neighbor-Hurds
-compare to Linux containers.
-
- “It’s all about freedom #0”
-
-On the technical side, Thomas Schwinge improved the technical
-documentation of the [[hurd/io_path]] in translators to make it easier
-for new developers to start hacking and Guillem Jover, Fridolin
-Pokorny and Jonathan Neuschäfer
+In the third quarter of 2011, the Arch Hurd hackers [packaged DDE (Device
+Driver Environment)](http://www.archhurd.org/news/22/), so a subset of the
+Linux 2.6 device drivers can now easily be run as user-space processes on Arch
+Hurd, replacing GNU Mach's in-kernel device drivers. (This has been possible
+before, too, but involved several [[manual steps|hurd/dde/guide]].) At the
+time of writing, our DDE port supports several network cards, while for other
+driver types we will need to add further generic infrastructure. Also, Arch
+Hurd had [a booth at
+FrOSCon](http://www.froscon.de/en/exhibitors/projekte.html#c1413) and [released
+a new Arch Hurd LiveCD](http://www.archhurd.org/news/24/), so new users can
+easily test the current state of the Arch flavor of the Hurd.
+
+Richard Braun contributed additional GNU Hurd instances: [[a *Debian buildd*, a
+*Debian porterbox*, and a *public Hurd box*|public_hurd_boxen]]. Especially
+the last one is important for *you*: after requesting an account, you can use
+it to test the Hurd without any own setup.
+
+Samuel Thibault sent a new [Bits from the Debian GNU/Hurd
+porters](http://lists.debian.org/debian-devel-announce/2011/07/msg00002.html)
+to keep the Debian folks up to date with our progres. And it is quite good:
+thanks to the relentless work of our porters, you can now use [70 % of all
+Debian packages with the Hurd](https://buildd.debian.org/stats/graph-big.png),
+so we're getting closer to [the goal of finishing a Release Canditate in time
+for Debian Wheezy](http://wiki.debian.org/Debian_GNU/Hurd). If you can, for
+example, port Debian packages and want to help the Hurd, this is the perfect
+time to get in contact and [port your favorite missing
+package](http://www.debian.org/ports/hurd/hurd-devel-debian) to the Hurd.
+
+A different kind of status update was delivered by Samuel Thibault on the [[GNU
+Hacker Meeting (GHM) in Paris|community/meetings/ghm2011]]. We hope you enjoy
+watching the video of the {{$community/meetings/ghm2011#thibault_hurd}}. He
+nicely explains how the simple yet powerful concept of a [[hurd/translator]]
+gives power to a system's less-priviledged users (that is, without `root`
+access), without any security implications, and how [[hurd/subhurd]]s and
+[[hurd/neighborhurd]]s compare to Linux containers. *It's all about [freedom
+0](http://www.gnu.org/philosophy/free-sw.html)*.
+
+On the technical side, Thomas Schwinge improved the technical documentation of
+the [[I/O path|hurd/io_path]] when translators are involved, to make it easier
+for new developers to understand how all the different system components
+interact. Amongst others, Guillem Jover, Fridolín Pokorný and Jonathan
+Neuschäfer
[sent](http://lists.gnu.org/archive/html/bug-hurd/2011-08/msg00184.html)
[many](http://lists.gnu.org/archive/html/bug-hurd/2011-08/msg00093.html)
-[patches](http://lists.gnu.org/archive/html/bug-hurd/2011-08/msg00030.html)
-for GNU Mach, improving stability, fixing memory leaks and cleaning up
-code.
-
-Additionally Maksym Planeta replaced GNU Mach’s old zone memory
-allocator with the new slab allocator from Richard Braun
-([integration commit](http://git.savannah.gnu.org/cgit/hurd/gnumach.git/commit/?id=50d073c5ef0feb1676606d0068abf626e8297cd7)),
-which should waste less memory than the zone allocator. Also it has a
-cpu cache level, so it should work faster on SMP systems, once we get
-up do date SMP CPU drivers for GNU Mach. It is now being integrated.
-
-And last but definitely not least, Jeremie Koenig finished his Google
-Summer of Code project to
-[Improve Java on Hurd](http://www.gnu.org/software/hurd/user/jkoenig/java.html). He
-[improved the Hurd signalling](http://lists.gnu.org/archive/html/bug-hurd/2011-06/msg00073.html),
-ported OpenJDK and created a
-[Java Hurd-Library](https://github.com/jeremie-koenig/hurd-java) which
-already allows writing a
-[Hello World translator in Java](https://github.com/jeremie-koenig/hurd-java/blob/master/HelloMach.java). It
-is still pretty low-level, but it paves the way for extending the core
-of the Hurd with Java, which gets the count of supported languages to
-3:
-[C(++)](http://www.gnu.org/software/hurd/hacking-guide/hhg.html#An-Example-using-trivfs),
-[[common_lisp|user/flaviocruz]] and Java.
-
-So if you want to help get the Hurd into Debian as a full release arch,
-so the power to the Hurd gives to casual users can actually get into
-the Hands of these, or dig dig deep into DDE to have more Linux
-drivers running in Userspace, please [[get_in_contact|contact_us]] -
-and maybe grab [[our_source_repos|source_repositories]].
-
-------
-
-The **GNU Hurd** is the GNU project's replacement for the Unix kernel.
-It is a collection of servers that run on the Mach microkernel to
-implement file systems, network protocols, file access control, and
-other features that are implemented by the Unix kernel or similar
-kernels (such as Linux).
-[[More_detailed|hurd/documentation]].
-
-**GNU Mach** is the microkernel upon which GNU Hurd is based. It
-offers Inter Process Communication (IPC) which the Hurd uses to define
-interfaces for implementing the services an operating system needs
-from a full-featured kernel.
-[[Read_more|microkernel/mach/gnumach]]
-
-<!--see [[contributing/web_pages/news/writing_the_moth]] for additional information on writing the MotH.-->
-
-<!-- * [[toolchain/ELFOSABI_GNU]]-->
-
+[patches](http://lists.gnu.org/archive/html/bug-hurd/2011-08/msg00030.html) for
+GNU Mach, improving stability, fixing memory leaks and generally cleaning up
+the code.
+
+Maksym Planeta finished a project he has been doing as a university task:
+replace GNU Mach's old zone memory allocator with a new [[!wikipedia
+slab_allocation desc="slab allocator"]] written by Richard Braun, who also
+mentored Maksym during the project. [This
+allocator](http://git.savannah.gnu.org/cgit/hurd/gnumach.git/commit/?h=mplaneta/libbraunr/master&id=59c9da87375ad3c8401890ecd4f7f101093f2463),
+apart from being overally cleaner than the zone allocator, is meant to waste
+less memory than the zone allocator (less fragmentation and more memory can be
+reclaimed by the VM system), there are debugging/inspection features, and it's
+SPM-ready, thus readily usable once we get up-do-date SMP support in GNU Mach.
+It is now being tested and integrated.
+
+And last but definitely not least, Jérémie Koenig finished his Google Summer of
+Code project to [[improve Java support on GNU Hurd|user/jkoenig/java]]. All in
+all, he also [improved the Hurd signalling
+code](http://lists.gnu.org/archive/html/bug-hurd/2011-06/msg00073.html), ported
+OpenJDK and began designing and creating a [library for Java bindings for Mach
+and Hurd](https://github.com/jeremie-koenig/hurd-java) which already allows
+writing a [Hello World translator in
+Java](https://github.com/jeremie-koenig/hurd-java/blob/master/HelloMach.java).
+It is still pretty low-level, but it paves the way for extending the core of
+the Hurd with Java, which is one of the benefits of the Hurd's distributed
+multi-server architecture: different components of the operating system can be
+written in different programming languages; not just
+[C](http://www.gnu.org/software/hurd/hacking-guide/hhg.html#An-Example-using-trivfs),
+but also C++, [[Common Lisp|user/flaviocruz]], and now Java -- and more to
+come.
+
+So if you want to help getting the Debian GNU/Hurd Release Candidate done, or
+want to dig deep into DDE to have more device drivers running as user-space
+processes, please [[get in contact|contact_us]] -- and maybe already grab the
+[[source code|source_repositories]].
+
+---
+
+The **GNU Hurd** is the GNU project's replacement for the Unix kernel. It is a
+collection of servers that run on the Mach microkernel to implement file
+systems, network protocols, file access control, and other features that are
+implemented by the Unix kernel or similar kernels (such as Linux). [[More
+detailed|hurd/documentation]].
+
+**GNU Mach** is the microkernel upon which a GNU Hurd system is based. It
+provides an Inter Process Communication (IPC) mechanism that the Hurd uses to
+define interfaces for implementing in a distributed multi-server fashion the
+services a traditional operating system kernel provides. [[More
+detailed|microkernel/mach/gnumach]].
"""]]
diff --git a/open_issues/anatomy_of_a_hurd_system.mdwn b/open_issues/anatomy_of_a_hurd_system.mdwn
index 46526641..13599e19 100644
--- a/open_issues/anatomy_of_a_hurd_system.mdwn
+++ b/open_issues/anatomy_of_a_hurd_system.mdwn
@@ -87,7 +87,7 @@ RPC stubs.
More stuff like [[hurd/IO_path]].
---
+---
IRC, freenode, #hurd, 2011-10-18:
@@ -96,3 +96,29 @@ IRC, freenode, #hurd, 2011-10-18:
<antrik> short version: grub loads mach, ext2, and ld.so/exec; mach starts
ext2; ext2 starts exec; ext2 execs a few other servers; ext2 execs
init. from there on, it's just standard UNIX stuff
+
+---
+
+IRC, OFTC, #debian-hurd, 2011-11-02:
+
+ <sekon_> is __dir_lookup a RPC ??
+ <sekon_> where can i find the source of __dir_lookup ??
+ <sekon_> grepping most gives out rvalue assignments
+ <sekon_> -assignments
+ <sekon_> but in hurs/fs.h it is used as a function ??
+ <pinotree> it should be the mig-generated function for that rpc
+ <sekon_> how do i know how its implemented ??
+ <sekon_> is there any way to delve deeprer into mig-generated functions
+ <tschwinge> sekon_: The MIG-generated stuff will either be found in the
+ package's build directory (if it's building it for themselves), or in the
+ glibc build directory (libhurduser, libmachuser; which are all the
+ available user RPC stubs).
+ <tschwinge> sekon_: The implementation can be found in the various Hurd
+ servers/libraries.
+ <tschwinge> sekon_: For example, [hurd]/libdiskfs/dir-lookup.c.
+ <tschwinge> sekon_: What MIG does is provide a function call interface for
+ these ``functions'', and the Mach microkernel then dispatches the
+ invocation to the corresponding server, for example a /hurd/ext2fs file
+ system (via libdiskfs).
+ <tschwinge> sekon_: This may help a bit:
+ http://www.gnu.org/software/hurd/hurd/hurd_hacking_guide.html
diff --git a/open_issues/code_analysis.mdwn b/open_issues/code_analysis.mdwn
index bb74d958..d776d81a 100644
--- a/open_issues/code_analysis.mdwn
+++ b/open_issues/code_analysis.mdwn
@@ -36,6 +36,11 @@ There is a [[!FF_project 276]][[!tag bounty]] on some of these tasks.
specifiers, and have it emit useful warnings in case these are pointing
to uninitialized data (for *in* only).
+ * [[Port Sequence Numbers|microkernel/mach/ipc/sequence_numbering]]. If
+ these are used, care must be taken to update them reliably, [[!message-id
+ "1123688017.3905.22.camel@buko.sinrega.org"]]. This could be checked by a
+ static analysis tool.
+
* [Static Source Code Analysis Tools for C](http://spinroot.com/static/)
* [[!wikipedia List_of_tools_for_static_code_analysis]]
diff --git a/open_issues/ext2fs_page_cache_swapping_leak.mdwn b/open_issues/ext2fs_page_cache_swapping_leak.mdwn
index c0d0867b..075533e7 100644
--- a/open_issues/ext2fs_page_cache_swapping_leak.mdwn
+++ b/open_issues/ext2fs_page_cache_swapping_leak.mdwn
@@ -12,7 +12,10 @@ License|/fdl]]."]]"""]]
There is a [[!FF_project 272]][[!tag bounty]] on this task.
-IRC, OFTC, #debian-hurd, 2011-03-24
+[[!toc]]
+
+
+# IRC, OFTC, #debian-hurd, 2011-03-24
<youpi> I still believe we have an ext2fs page cache swapping leak, however
<youpi> as the 1.8GiB swap was full, yet the ld process was only 1.5GiB big
@@ -24,7 +27,7 @@ IRC, OFTC, #debian-hurd, 2011-03-24
<youpi> yes
<youpi> the disk content, basicallyt :)
-IRC, freenode, #hurd, 2011-04-18
+# IRC, freenode, #hurd, 2011-04-18
<antrik> damn, a cp -a simply gobbles down swap space...
<braunr> really ?
@@ -173,3 +176,84 @@ IRC, freenode, #hurd, 2011-04-18
backing store of memory objects created from its pager
<braunr> so you can view swap as the file system for everything that isn't
an external memory object
+
+
+# IRC, freenode, #hurd, 2011-11-15
+
+ <braunr> hm, now my system got unstable
+ <braunr> swap is increasing, without any apparent reason
+ <antrik> you mean without any load?
+ <braunr> with load, yes
+ <braunr> :)
+ <antrik> well, with load is "normal"...
+ <antrik> at least for some loads
+ <braunr> i can't create memory pressure to stress reclaiming without any
+ load
+ <antrik> what load are you using?
+ <braunr> ftp mirrorring
+ <antrik> hm... never tried that; but I guess it's similar to apt-get
+ <antrik> so yes, that's "normal". I talked about it several times, and also
+ wrote to the ML
+ <braunr> antrik: ok
+ <antrik> if you find out how to fix this, you are my hero ;-)
+ <braunr> arg :)
+ <antrik> I suspect it's the infamous double swapping problem; but that's
+ just a guess
+ <braunr> looks like this
+ <antrik> BTW, if you give me the exact command, I could check if I see it
+ too
+ <braunr> i use lftp (mirror -Re) from a linux git repository
+ <braunr> through sftp
+ <braunr> (lots of small files, big content)
+ <antrik> can't you just give me the exact command? I don't feel like
+ figuring it out myself
+ <braunr> antrik: cd linux-stable; lftp sftp://hurd_addr/
+ <braunr> inside lftp: mkdir linux-stable; cd linux-stable; mirror -Re
+ <braunr> hm, half of physical memory just got freed
+ <braunr> our page cache is really weird :/
+ <braunr> (i didn't delete any file when that happened)
+ <antrik> hurd_addr?
+ <braunr> ssh server ip address
+ <braunr> or name
+ <braunr> of your hurd :)
+ <antrik> I'm confused. you are mirroring *from* the Hurd box?
+ <braunr> no, to it
+ <antrik> ah, so you login via sftp and then push to it?
+ <braunr> yes
+ <braunr> fragmentation looks very fine
+ <braunr> even for the huge pv_entry cache and its 60k+ entries
+ <braunr> (and i'm running a kernel with the cpu layer enabled)
+ <braunr> git reset/status/diff/log/grep all work correctly
+ <braunr> anyway, mcsim's branch looks quite stable to me
+ <antrik> braunr: I can't reproduce the swap leak with ftp. free memory
+ idles around 6.5 k (seems to be the threshold where paging starts), and
+ swap use is constant
+ <antrik> might be because everything swappable is already present in swap
+ from previous load I guess...
+ <antrik> err... scratch that. was connected to the wrong host, silly me
+ <antrik> indeed swap gets eaten away, as expected
+ <antrik> but only if free memory actually falls below the
+ threshold. otherwise it just oscillates around a constant value, and
+ never touches swap
+ <antrik> so this seems to confirm the double swapping theory
+ <youpi> antrik: is that "double swap" theory written somewhere?
+ <youpi> (no, a quick google didn't tell me)
+
+
+## IRC, freenode, #hurd, 2011-11-16
+
+ <antrik> youpi:
+ http://lists.gnu.org/archive/html/l4-hurd/2002-06/msg00001.html talks
+ about "double paging". probably it's also the term others used for it;
+ however, the term is generally used in a completely different meaning, so
+ I guess it's not really suitable for googling either ;-)
+ <antrik> IIRC slpz (or perhaps someone else?) proposed a solution to this,
+ but I don't remember any details
+ <youpi> ok so it's the same thing I was thinking about with swap getting
+ filled
+ <youpi> my question was: is there something to release the double swap,
+ once the ext2fs pager managed to recover?
+ <antrik> apparently not
+ <antrik> the only way to free the memory seems to be terminating the FS
+ server
+ <youpi> uh :/
diff --git a/open_issues/fakeroot-tcp_vs_eintr.mdwn b/open_issues/fakeroot_exit_0.mdwn
index 36707cd2..c6075b17 100644
--- a/open_issues/fakeroot-tcp_vs_eintr.mdwn
+++ b/open_issues/fakeroot_exit_0.mdwn
@@ -8,23 +8,8 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
-[[!toc]]
-
-
-# [[!debbug 641200]]
-
-[[!message-id "87litvz9me.fsf@kepler.schwinge.homeip.net"]]
-
+ $ fakeroot ./scripts/mkinstalldirs /media/erich/home/thomas/tmp/glibc/debian/eglibc-2.13/debian/tmp-libc/usr/include
[...]
- if test -z "$*"; then
- FAKEROOTKEY=$FAKEROOTKEY LD_LIBRARY_PATH="$PATHS" LD_PRELOAD="$LIB" ${SHELL:-/bin/sh}
- RESULT=$?
- else
- FAKEROOTKEY=$FAKEROOTKEY LD_LIBRARY_PATH="$PATHS" LD_PRELOAD="$LIB" "$@"
- RESULT=$?
- fi
- + test -z './scripts/mkinstalldirs /media/erich/home/thomas/tmp/glibc/debian/eglibc-2.13/debian/tmp-libc/usr/include'
- + FAKEROOTKEY=2040
+ LD_LIBRARY_PATH=/usr/lib/libfakeroot:/usr/lib64/libfakeroot:/usr/lib32/libfakeroot
+ LD_PRELOAD=libfakeroot-tcp.so
+ ./scripts/mkinstalldirs /media/erich/home/thomas/tmp/glibc/debian/eglibc-2.13/debian/tmp-libc/usr/include
@@ -36,8 +21,7 @@ License|/fdl]]."]]"""]]
kill -s HUP 23612
+ kill -s HUP 23612
-
-## `exit(1)`
+(The `EINTR` issue has been [[!debbug desc="fixed" 641200]].)
`connect() < 0` invokes `fail()` which invokes `exit(1)`. Not yet figured out
why the process exits 0 dispite that. `LD_PRELOAD` issue? ([[!taglink
diff --git a/open_issues/glibc.mdwn b/open_issues/glibc.mdwn
index d3f88673..e8279139 100644
--- a/open_issues/glibc.mdwn
+++ b/open_issues/glibc.mdwn
@@ -27,8 +27,17 @@ Here's what's to be done for maintaining glibc.
# Configuration
-Last reviewed up to the [[Git mirror's 187da0aedcd9d0a2fb34477bef41549681ba1273
-(2011-10-08) sources|source_repositories/glibc]].
+<!--
+
+git checkout reviewed
+git log --reverse --pretty=fuller --stat=$COLUMNS,$COLUMNS -p -C --cc ..sourceware/master
+-i
+/^commit |^---$|hurd|linux|gs:|__ASSUME
+
+-->
+
+Last reviewed up to the [[Git mirror's 9d65ea3a9b83ac3961229ba296a7caf90abce68d
+(2011-11-17) sources|source_repositories/glibc]].
* t/dup3
@@ -53,13 +62,9 @@ Last reviewed up to the [[Git mirror's 187da0aedcd9d0a2fb34477bef41549681ba1273
$ git log --reverse --pretty=fuller --stat=$COLUMNS,$COLUMNS -p -C --cc -S__ASSUME_ top-bases/t/kernel-features.h_includes..baseline
- * t/tls
-
- * Discuss d2431f633e6139a62e1575ec18830f7e81160cf0 with Samuel.
+ * [[`t/tls`|t/tls]]
- * `TLS_INIT_TP_EXPENSIVE` is unused; Hurd def. can be removed.
-
- * [[t/tls-threadvar]]
+ * [[`t/tls-threadvar`|t/tls-threadvar]]
* t/verify.h
@@ -156,7 +161,8 @@ Last reviewed up to the [[Git mirror's 187da0aedcd9d0a2fb34477bef41549681ba1273
`PTRACE_*` (for example, cbff0d9689c4d68578b6a4f0a17807232506ea27),
`RLIMIT_RTTIME`, `SEEK_DATA` (`unistd.h`), `SEEK_HOLE` (`unistd.h`)
`clock_adjtime`, `fallocate`, `fallocate64`, `getcontext` (and
- `setcontext`), `name_to_handle_at`, `open_by_handle_at`, `sendmmsg`,
+ `setcontext`), `name_to_handle_at`, `open_by_handle_at`,
+ `process_vm_readv`, `process_vm_writev`, `sendmmsg`,
`setns`, `sync_file_range`
* `syncfs`
@@ -194,6 +200,14 @@ Last reviewed up to the [[Git mirror's 187da0aedcd9d0a2fb34477bef41549681ba1273
Need changes equivalent to c55fbd1ea768f9fdef34a01377702c0d72cbc213 +
14d96785125abee5e9a49a1c3037f35a581750bd.
+ * `madvise`, `MADV_DONTNEED`
+
+ [[glibc_madvise_vs_static_linking]].
+
+ * `msync`
+
+ Then define `_POSIX_MAPPED_FILES`, `_POSIX_SYNCHRONIZED_IO`.
+
* Create `t/cleanup_kernel-features.h`.
* Add tests from Linux kernel commit messages for `t/dup3` et al.
@@ -366,12 +380,23 @@ Last reviewed up to the [[Git mirror's 187da0aedcd9d0a2fb34477bef41549681ba1273
* a7c8e6a1478de9f990b11e5e853318ccbe4330f2 (`Fix invalid conversion in
__cmsg_nxthdr`). Probably just a C++ thing and not relevant for us;
see [[message-id "87r52nk1kx.fsf@kepler.schwinge.homeip.net"]].
+ * [high] `__ctype_init`, fd5bdc0924e0cfd1688b632068c1b26f3b0c88da.
+ Probably need to mirror `init-first.c` change.
+ * [high] `__attribute__ ((__leaf__))`, `BZ #13344`,
+ aa78043a4aafe5db1a1a76d544a833b63b4c5f5c +
+ 49a43d80ec5c97cf6136b1ee2687414773b2d5aa +
+ 3871f58f065dac3917eb18220a479e9591769c8c +
+ 9beb2334930db81ceada5aa6051fe5ac0554db32 +
+ 0ffc4f3ebaace42cd545db55a2ac50b6e0cc7d89 +
+ edc5984d4d18296d7aa3d8f4ed8f7336a743170e +
+ 57769839788e2c62b68d9dfbf4b35052321278ba.
+ <http://gcc.gnu.org/onlinedocs/gcc-4.6.1/gcc/Function-Attributes.html>.
# Build
Here's a log of a glibc build run; this is from our [[Git repository's
-0abded0927c62f02399658395837917548d5e281 (2011-10-14)
+d740cf9d201dc9ecb0335b0a585828dea9cce793 (2011-10-25)
sources|source_repositories/glibc]], run on coulomb.SCHWINGE.
$ export LC_ALL=C
@@ -383,7 +408,7 @@ sources|source_repositories/glibc]], run on coulomb.SCHWINGE.
This takes up around 400 MiB and needs roughly 120 min on coulomb.SCHWINGE.
<!--
- $ (make install_root=/INVALID && touch .go-check) 2>&1 | tee log_build_ && test -f .go-check && make -k install_root=/INVALID check TIMEOUTFACTOR=100 2>&1 | tee log_check
+ $ (make install_root=/INVALID && touch .go-install) 2>&1 | tee log_build_ && test -f .go-install && (make install_root="$PWD".install install && touch .go-check) 2>&1 | tee log_install && test -f .go-check && make -k install_root=/INVALID check TIMEOUTFACTOR=100 2>&1 | tee log_check
$ find ./ -name \*.o -o -name \*.os -o -name \*.oS | while read f; do ~/tmp/gcc/git/contrib/compare-debug --preserve ../Roger_Whittaker.build-gcc-4.4-486.O/"$f" "$f"; done 2>&1 | less
$ while read f; do (readelf -a "$f" && objdump -xDrtw "$f") > N && (cd ../Roger_Whittaker.build-gcc-4.4-486.O/ && readelf -a "$f" && objdump -xDrtw "$f") > O && diff -u O N | less; done
$ find ./ -name \*.o -o -name \*.os -o -name \*.oS | while read f; do readelf -h "$f" | grep OS/ABI | (read a b && [ x"$b" != x'UNIX - System V' ] && echo "### $f: $b"); done
@@ -401,7 +426,7 @@ TODO.
TODO.
<!--
- $ make install 2>&1 | tee log_install
+ $ make install_root="$PWD".install install 2>&1 | tee log_install
[...]
This takes up around 50 MiB, and needs roughly 1 min on kepler.SCHWINGE and 3
@@ -539,6 +564,8 @@ There is quite a baseline of failures.
* `elf/tst-unique3lib.so`, `elf/tst-unique3lib2.so`, `elf/tst-unique4lib.so`
+ Only with GCC 4.4; no longer with 4.5 or 4.6:
+
/home/thomas/tmp/glibc/tschwinge/Roger_Whittaker.build-gcc-4.4-486/elf/tst-unique3lib.os:(.data.DW.ref.__gxx_personality_v0[DW.ref.__gxx_personality_v0]+0x0): undefined reference to `__gxx_personality_v0'
diff --git a/open_issues/glibc/t/tls.mdwn b/open_issues/glibc/t/tls.mdwn
new file mode 100644
index 00000000..14ef36e4
--- /dev/null
+++ b/open_issues/glibc/t/tls.mdwn
@@ -0,0 +1,66 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_glibc open_issue_libpthread]]
+
+# To Do
+
+ * Discuss d2431f633e6139a62e1575ec18830f7e81160cf0 with Samuel.
+
+ * `TLS_INIT_TP_EXPENSIVE` is unused; Hurd def. can be removed.
+
+
+# Documentation
+
+[[!taglink open_issue_documentation]]
+
+ * IRC, freenode, #hurd, 2011-11-26
+
+ <tschwinge> In glibc multiarch support (strcasecmp for i686 SSE3, etc.)
+ there is access to memory via gs: -- this will need to be changed for
+ us, right?
+ <youpi> depends on the access
+ <tschwinge> * `optimized strcasecmp and strncasecmp for x86-32`
+ (multiarch),
+ <tschwinge> 76e3966e9efc3808a9e7ad09121c5dfc1211c20b +
+ <tschwinge> 6abf346582ba678f4850a88b4a5950593841df1d +
+ <tschwinge> 5583a0862cf94f71cbcde91c4043a20af65facca. `gs`
+ access.
+ <youpi> + movl __libc_tsd_LOCALE@GOTNTPOFF(%ebx), %eax
+ <youpi> that's handled by the linker fine
+ <youpi> it's only the things held in the tcb_t structure which can pose
+ problem
+ <tschwinge> tcbhead_t?
+ <tschwinge> I'm looking at this.
+ <tschwinge> So, at gs:0, there is the TCB.
+ <tschwinge> And we have the same layout as NPTL/Linux, just that we
+ don't have as much data there as they have.
+ <tschwinge> We're missing multiple_threads, sysinfo, sttack_guard,
+ pointer_guard, gscope_flag, private_futex, __private_tm[5].
+ <tschwinge> So, if one of these is referenced (be it my name or by
+ numeric offset), this is invalid for us.
+ <tschwinge> Anything else should work equivalently.
+ <youpi> yes
+ <youpi> usually the only numeric offset being used is 0
+ <youpi> so it would simply not build
+ <tschwinge> And the other offsers are generated via tcb-offsets.sym.
+ <tschwinge> glibc's elf/stackguard-macros.h is wrong for us (but not
+ used anywhere apart from elf/tst-stackguard1.c, I think).
+ <tschwinge> __thread __locale_t __libc_tsd_LOCALE = &_nl_global_locale;
+ -- this means that a __libc_tsd_LOCALE values will be in the TLS
+ segment, and this is what is being accessed from the assembler code
+ with %gs:__libc_tsd_LOCALE@NTPOFF, and the linker will resolve this.
+ <youpi> yes
+ <youpi> see in the nm output, the libc_tsd symbols
+ <youpi> these provide the offsets
+ <tschwinge> youpi: Thank you, I'm now understanding this part of TLS
+ much better.
+ <youpi> have you had a look at the tls.pdf from Uli ?
+ <youpi> all the gory details are there :)
diff --git a/open_issues/gnumach_memory_management.mdwn b/open_issues/gnumach_memory_management.mdwn
index 9a4418c1..c9c3e64f 100644
--- a/open_issues/gnumach_memory_management.mdwn
+++ b/open_issues/gnumach_memory_management.mdwn
@@ -1810,3 +1810,205 @@ There is a [[!FF_project 266]][[!tag bounty]] on this task.
<braunr> etenil: but mcsim's work is, for one, useful because the allocator
code is much clearer, adds some debugging support, and is smp-ready
+
+
+# IRC, freenode, #hurd, 2011-11-14
+
+ <braunr> i've just realized that replacing the zone allocator removes most
+ (if not all) static limit on allocated objects
+ <braunr> as we have nothing similar to rlimits, this means kernel resources
+ are actually exhaustible
+ <braunr> and i'm not sure every allocation is cleanly handled in case of
+ memory shortage
+ <braunr> youpi: antrik: tschwinge: is this acceptable anyway ?
+ <braunr> (although IMO, it's also a good thing to get rid of those limits
+ that made the kernel panic for no valid reason)
+ <youpi> there are actually not many static limits on allocated objects
+ <youpi> only a few have one
+ <braunr> those defined in kern/mach_param.h
+ <youpi> most of them are not actually enforced
+ <braunr> ah ?
+ <braunr> they are used at zinit() time
+ <braunr> i thought they were
+ <youpi> yes, but most zones are actually fine with overcoming the max
+ <braunr> ok
+ <youpi> see zone->max_size += (zone->max_size >> 1);
+ <youpi> you need both !EXHAUSTIBLE and FIXED
+ <braunr> ok
+ <pinotree> making having rlimits enforced would be nice...
+ <pinotree> s/making//
+ <braunr> pinotree: the kernel wouldn't handle many standard rlimits anyway
+
+ <braunr> i've just committed my final patch on mcsim's branch, which will
+ serve as the starting point for integration
+ <braunr> which means code in this branch won't change (or only last minute
+ changes)
+ <braunr> you're invited to test it
+ <braunr> there shouldn't be any noticeable difference with the master
+ branch
+ <braunr> a bit less fragmentation
+ <braunr> more memory can be reclaimed by the VM system
+ <braunr> there are debugging features
+ <braunr> it's SMP ready
+ <braunr> and overall cleaner than the zone allocator
+ <braunr> although a bit slower on the free path (because of what's
+ performed to reduce fragmentation)
+ <braunr> but even "slower" here is completely negligible
+
+
+# IRC, freenode, #hurd, 2011-11-15
+
+ <mcsim> I enabled cpu_pool layer and kentry cache exhausted at "apt-get
+ source gnumach && (cd gnumach-* && dpkg-buildpackage)"
+ <mcsim> I mean kernel with your last commit
+ <mcsim> braunr: I'll make patch how I've done it in a few minutes, ok? It
+ will be more specific.
+ <braunr> mcsim: did you just remove the #if NCPUS > 1 directives ?
+ <mcsim> no. I replaced macro NCPUS > 1 with SLAB_LAYER, which equals NCPUS
+ > 1, than I redefined macro SLAB_LAYER
+ <braunr> ah, you want to make the layer optional, even on UP machines
+ <braunr> mcsim: can you give me the commands you used to trigger the
+ problem ?
+ <mcsim> apt-get source gnumach && (cd gnumach-* && dpkg-buildpackage)
+ <braunr> mcsim: how much ram & swap ?
+ <braunr> let's see if it can handle a quite large aptitude upgrade
+ <mcsim> how can I check swap size?
+ <braunr> free
+ <braunr> cat /proc/meminfo
+ <braunr> top
+ <braunr> whatever
+ <mcsim> total used free shared buffers
+ cached
+ <mcsim> Mem: 786368 332296 454072 0 0
+ 0
+ <mcsim> -/+ buffers/cache: 332296 454072
+ <mcsim> Swap: 1533948 0 1533948
+ <braunr> ok, i got the problem too
+ <mcsim> braunr: do you run hurd in qemu?
+ <braunr> yes
+ <braunr> i guess the cpu layer increases fragmentation a bit
+ <braunr> which means more map entries are needed
+ <braunr> hm, something's not right
+ <braunr> there are only 26 kernel map entries when i get the panic
+ <braunr> i wonder why the cache gets that stressed
+ <braunr> hm, reproducing the kentry exhaustion problem takes quite some
+ time
+ <mcsim> braunr: what do you mean?
+ <braunr> sometimes, dpkg-buildpackage finishes without triggering the
+ problem
+ <mcsim> the problem is in apt-get source gnumach
+ <braunr> i guess the problem happens because of drains/fills, which
+ allocate/free much more object than actually preallocated at boot time
+ <braunr> ah ?
+ <braunr> ok
+ <braunr> i've never had it at that point, only later
+ <braunr> i'm unable to trigger it currently, eh
+ <mcsim> do you use *-dbg kernel?
+ <braunr> yes
+ <braunr> well, i use the compiled kernel, with the slab allocator, built
+ with the in kernel debugger
+ <mcsim> when you run apt-get source gnumach, you run it in clean directory?
+ Or there are already present downloaded archives?
+ <braunr> completely empty
+ <braunr> ah just got it
+ <braunr> ok the limit is reached, as expected
+ <braunr> i'll just bump it
+ <braunr> the cpu layer drains/fills allocate several objects at once (64 if
+ the size is small enough)
+ <braunr> the limit of 256 (actually 252 since the slab descriptor is
+ embedded in its slab) is then easily reached
+ <antrik> mcsim: most direct way to check swap usage is vmstat
+ <braunr> damn, i can't live without slabtop and the amount of
+ active/inactive cache memory any more
+ <braunr> hm, weird, we have active/inactive memory in procfs, but not
+ buffers/cached memory
+ <braunr> we could set buffers to 0 and everything as cached memory, since
+ we're currently unable to communicate the purpose of cached memory
+ (whether it's used by disk servers or file system servers)
+ <braunr> mcsim: looks like there are about 240 kernel map entries (i forgot
+ about the ones used in kernel submaps)
+ <braunr> so yes, addin the cpu layer is what makes the kernel reach the
+ limit more easily
+ <mcsim> braunr: so just increasing limit will solve the problem?
+ <braunr> mcsim: yes
+ <braunr> slab reclaiming looks very stable
+ <braunr> and unfrequent
+ <braunr> (which is surprising)
+ <pinotree> braunr: "unfrequent"?
+ <braunr> pinotree: there isn't much memory pressure
+ <braunr> slab_collect() gets called once a minute on my hurd
+ <braunr> or is it infrequent ?
+ <braunr> :)
+ <pinotree> i have no idea :)
+ <braunr> infrequent, yes
+
+
+# IRC, freenode, #hurd, 2011-11-16
+
+ <braunr> for those who want to play with the slab branch of gnumach, the
+ slabinfo tool is available at http://git.sceen.net/rbraun/slabinfo.git/
+ <braunr> for those merely interested in numbers, here is the output of
+ slabinfo, for a hurd running in kvm with 512 MiB of RAM, an unused swap,
+ and a short usage history (gnumach debian packages built, aptitude
+ upgrade for a dozen of packages, a few git commands)
+ <braunr> http://www.sceen.net/~rbraun/slabinfo.out
+ <antrik> braunr: numbers for a long usage history would be much more
+ interesting :-)
+
+
+## IRC, freenode, #hurd, 2011-11-17
+
+ <braunr> antrik: they'll come :)
+ <etenil> is something going on on darnassus? it's mighty slow
+ <braunr> yes
+ <braunr> i've rebooted it to run a modified kernel (with the slab
+ allocator) and i'm building stuff on it to stress it
+ <braunr> (i don't have any other available machine with that amount of
+ available physical memory)
+ <etenil> ok
+ <antrik> braunr: probably would be actually more interesting to test under
+ memory pressure...
+ <antrik> guess that doesn't make much of a difference for the kernel object
+ allocator though
+ <braunr> antrik: if ram is larger, there can be more objects stored in
+ kernel space, then, by building something large such as eglibc, memory
+ pressure is created, causing caches to be reaped
+ <braunr> our page cache is useless because of vm_object_cached_max
+ <braunr> it's a stupid arbitrary limit masking the inability of the vm to
+ handle pressure correctly
+ <braunr> if removing it, the kernel freezes soon after ram is filled
+ <braunr> antrik: it may help trigger the "double swap" issue you mentioned
+ <antrik> what may help trigger it?
+ <braunr> not checking this limit
+ <antrik> hm... indeed I wonder whether the freezes I see might have the
+ same cause
+
+
+## IRC, freenode, #hurd, 2011-11-19
+
+ <braunr> http://www.sceen.net/~rbraun/slabinfo.out <= state of the slab
+ allocator after building the debian libc packages and removing all files
+ once done
+ <braunr> it's mostly the same as on any other machine, because of the
+ various arbitrary limits in mach (most importantly, the max number of
+ objects in the page cache)
+ <braunr> fragmentation is still quite low
+ <antrik> braunr: actually fragmentation seems to be lower than on the other
+ run...
+ <braunr> antrik: what makes you think that ?
+ <antrik> the numbers of currently unused objects seem to be in a similar
+ range IIRC, but more of them are reclaimable I think
+ <antrik> maybe I'm misremembering the other numbers
+ <braunr> there had been more reclaims on the other run
+
+
+# IRC, freenode, #hurd, 2011-11-25
+
+ <braunr> mcsim: i've just updated the slab branch, please review my last
+ commit when you have time
+ <mcsim> braunr: Do you mean compilation/tests?
+ <braunr> no, just a quick glance at the code, see if it matches what you
+ intended with your original patch
+ <mcsim> braunr: everything is ok
+ <braunr> good
+ <braunr> i think the branch is ready for integration
diff --git a/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn b/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn
index 93055b77..80fc9fcd 100644
--- a/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn
+++ b/open_issues/libmachuser_libhurduser_rpc_stubs.mdwn
@@ -54,3 +54,53 @@ License|/fdl]]."]]"""]]
compatibility with eventual 3rd party users is not broken
<pinotree> but those using them, other than hurd itself, won't compile
anymore, so you fix them progressively
+
+
+# IRC, freenode, #hurd, 2011-11-16
+
+ <braunr> is the mach_debug interface packaged in debian ?
+ <antrik> what mach_debug interface?
+ <braunr> include/include/mach_debug/mach_debug.defs in gnumach
+ <braunr> include/mach_debug/mach_debug.defs in gnumach
+ <antrik> what exactly is supposed to be packaged there?
+ <braunr> i'm talking about the host_*_info client code
+ <antrik> braunr: you mean MIG-generated stubs?
+ <braunr> antrik: yes
+ <braunr> i wrote a tiny slabinfo tool, and rpctrace doesn't show the
+ host_slab_info call
+ <braunr> do you happen to know why ?
+ <antrik> braunr: doesn't show it at all, or just doesn't translate?
+ <braunr> antrik: doesn't at all, the msgids file contains what's needed to
+ translate
+ <braunr> btw, i was able to build the libc0.3 packages with a kernel using
+ the slab allocator today, while monitoring it with the aforementioned
+ slabinfo tool, everything went smoothly
+ <antrik> great :-)
+ <braunr> i'll probably add a /proc/slabinfo entry some day
+ <braunr> and considering the current state of our beloved kernel, i'm
+ wondering why host_*_info rpcs are considered debugging calls
+ <braunr> imo, they should always be included by default
+ <braunr> and part of the standard mach interface
+ <braunr> (if the rpc is missing, an error is simply returned)
+ <antrik> I guess that's been inherited from original Mach
+ <antrik> so you think the stubs should be provided by libmachuser?
+ <braunr> i'm not sure
+ <antrik> actually, it's a bit arguable. if interfaces are not needed by
+ libc itself, it isn't really necessary to build them as part of the libc
+ build...
+ <braunr> i don't know the complete list of potential places for such calls
+ <antrik> OTOH, as any updates will happen in sync with other Mach updates,
+ it makes sense to keep them in one place, to reduce transition pain
+ <braunr> and i didn't want to imply they should be part of libc
+ <braunr> on the contrary, libmachuser seems right
+ <antrik> libmachuser is part of libc
+ <braunr> ah
+ <braunr> :)
+ <braunr> why so ?
+ <antrik> well, for one, libc needs the Mach (and Hurd) stubs itself
+ <antrik> also, it's traditionally the role of libc to provide the call
+ wrappers for syscalls... so it makes some sense
+ <braunr> sure, but why doesn't it depend on an external libmachuser instead
+ of embedding it ?
+ <braunr> right
+ <antrik> now that's a good question... no idea TBH :-)
diff --git a/open_issues/mig_portable_rpc_declarations.mdwn b/open_issues/mig_portable_rpc_declarations.mdwn
new file mode 100644
index 00000000..084d7454
--- /dev/null
+++ b/open_issues/mig_portable_rpc_declarations.mdwn
@@ -0,0 +1,58 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_mig]]
+
+
+# IRC, freenode, #hurd, 2011-11-14
+
+ <braunr> also, what's the best way to deal with types such as
+ <braunr> type cache_info_t = struct[23] of integer_t;
+ <braunr> whereas cache_info_t contains longs, which are obviously not
+ integer-wide on 64-bits processors
+ <braunr> ?
+ <youpi> you mean, to port mach to 64bit?
+ <braunr> no, to make the RPC declaration portable
+ <braunr> just in case :)
+ <youpi> refine integer_t into something more precise
+ <youpi> such as size_t, off_t, etc.
+ <braunr> i can't use a single line then
+ <braunr> struct cache_info contains ints, vm_size_t, longs
+ <braunr> should i just use the maximum size it can get ?
+ <braunr> or declare two sizes depending on the word size ?
+ <youpi> well, I'd say three
+ <braunr> youpi: three ?
+ <youpi> the ints, the vm_size_ts, and the longs
+ <braunr> youpi: i don't get it
+ <braunr> youpi: how would i write it in mig language ?
+ <youpi> I don't know the mig language
+ <braunr> me neither :)
+ <youpi> but I'd say don't lie
+ <braunr> i just see struct[23] of smething
+ <braunr> the original zone_info struct includes both integer_t and
+ vm_size_t, and declares it as
+ <braunr> type zone_info_t = struct[9] of integer_t;
+ <braunr> in its mig defs file
+ <braunr> i don't have a good example to reuse
+ <youpi> which is lying
+ <braunr> yes
+ <braunr> which is why i was wondering if mach architects themselves
+ actually solved that problem :)
+ <braunr> "There is no way to specify the fields of a
+ <braunr> C structure to MIG. The size and type-desc are just used to
+ give the size of
+ <braunr> the structure.
+ <braunr> "
+ <braunr> well, this sucks :/
+ <braunr> well, i'll do what the rest of the code seems to do, and let it
+ rot until a viable solution is available
+ <antrik> braunr: we discussed the problem of expressing structs with MIG in
+ the libburn thread
+ <antrik> (which I still need to follow up on... [sigh])
diff --git a/open_issues/mission_statement.mdwn b/open_issues/mission_statement.mdwn
index 212d65e7..d136e3a8 100644
--- a/open_issues/mission_statement.mdwn
+++ b/open_issues/mission_statement.mdwn
@@ -10,7 +10,10 @@ License|/fdl]]."]]"""]]
[[!tag open_issue_documentation]]
-IRC, freenode, #hurd, 2011-10-12:
+[[!toc]]
+
+
+# IRC, freenode, #hurd, 2011-10-12
<ArneBab> we have a mission statement: http://hurd.gnu.org
<Gorodish> yes
@@ -37,3 +40,10 @@ IRC, freenode, #hurd, 2011-10-12:
ceases to amaze me
<Gorodish> I agree that the informational, factual, results oriented
documentation is the primary objective of documenting
+
+
+# IRC, freenode, #hurd, 2011-11-25
+
+ <antrik> heh, nice: http://telepathy.freedesktop.org/wiki/Rationale
+ <antrik> most of this could be read as a rationale for the Hurd just as
+ well ;-)
diff --git a/open_issues/multithreading.mdwn b/open_issues/multithreading.mdwn
index 1fc2c318..0f6b9f19 100644
--- a/open_issues/multithreading.mdwn
+++ b/open_issues/multithreading.mdwn
@@ -24,7 +24,7 @@ Hurd servers / VFS libraries are multithreaded.
# Design
-Roughly using one thread per
+See [[hurd/libports]]: roughly using one thread per
incoming request. This is not the best approach: it doesn't really make sense
to scale the number of worker threads with the number of incoming requests, but
instead they should be scaled according to the backends' characteristics.
diff --git a/open_issues/page_cache.mdwn b/open_issues/page_cache.mdwn
new file mode 100644
index 00000000..062fb8d6
--- /dev/null
+++ b/open_issues/page_cache.mdwn
@@ -0,0 +1,73 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_gnumach]]
+
+IRC, freenode, #hurd, 2011-11-28:
+
+ <braunr> youpi: would you find it reasonable to completely disable the page
+ cache in gnumach ?
+ <braunr> i'm wondering if it wouldn't help make the system more stable
+ under memory pressure
+ <youpi> assuming cache=writeback in gnumach?
+ <youpi> because disabling the page cache will horribly hit performance
+ <braunr> no, it doesn't have anything to do with the host
+ <braunr> i'm not so sure
+ <braunr> while observing the slab allocator, i noticed our page cache is
+ not used that often
+ <youpi> eeh?
+ <youpi> apart from the damn 4000 limitation, I've seen it used
+ <youpi> (and I don't why it wouldn't be used)
+ <youpi> (e.g. for all parts of libc)
+ <youpi> ah, no, libc would be kept open by ext2fs
+ <braunr> taht's precisely because of the 4k limit
+ <youpi> but e.g. .o file emitted during make
+ <braunr> well, no
+ <youpi> well, see the summary I had posted some time ago, the 4k limit
+ makes it completely randomized
+ <youpi> and thus you lose locality
+ <braunr> yes
+ <youpi> but dropping the limit would just fix it
+ <braunr> that's my point
+ <youpi> which I had tried to do, and there were issues, you mentioned why
+ <youpi> and (as usual), I haven't had anyu time to have a look at the issue
+ again
+ <braunr> i'm just trying to figure out the pros and cons for having teh
+ current page cache implementation
+ <braunr> but are you saying you tried with a strict limit of 0 ?
+ <youpi> non, I'm saying I tried with no limit
+ <youpi> but then memory fills up
+ <braunr> yes
+ <youpi> so trying to garbage collect
+ <braunr> i tried that too, the system became unstable very quickly
+ <youpi> but refs don't falldown to 0, you said
+ <braunr> did i ?
+ <youpi> or maybe somebody else
+ <youpi> see the list archives
+ <braunr> that's possible
+ <braunr> i'd imagine someone like sergio lopez
+ <youpi> possibly
+ <youpi> somebody that knows memory stuff way better than me in any case
+ <braunr> youpi: i'm just wondering how much we'd loose by disabling the
+ page cache, and if we actually gain more stability (and ofc, if it's
+ worth it)
+ <youpi> no idea, measures will tell
+ <youpi> fixing the page cache shouldn't be too hard I believe, however
+ <youpi> you just need to know what you are doing, which I don't
+ <youpi> I do believe the cache is still at least a bit useful
+ <youpi> even if dumb because of randomness
+ <youpi> e.g. running make lib in the glibc tree gets faster on second time
+ <youpi> because the cache wouldbe filled at least randomly with glibc tree
+ stuff
+ <braunr> yes, i agree on that
+ <youpi> braunr: btw, the current stability is fine for the buildds
+ <youpi> restarting them every few days is ok
+ <youpi> so I'd rather keep the performance :)
+ <braunr> ok
diff --git a/open_issues/perl.mdwn b/open_issues/perl.mdwn
index 45680328..48343e3e 100644
--- a/open_issues/perl.mdwn
+++ b/open_issues/perl.mdwn
@@ -36,6 +36,56 @@ First, make the language functional, have its test suite pass without errors.
[[!inline pages=community/gsoc/project_ideas/perl_python feeds=no]]
+
+## IRC, OFTC, #debian-hurd, 2011-11-08
+
+ <Dom> pinotree: so, with your three fixes applied to 5.14.2, there are
+ still 9 tests failing. They don't seem to be regressions in perl, since
+ they also fail when I build 5.14.0 (even though the buildd managed it).
+ <Dom> What do you suggest as the way forward?
+ <Dom> (incidentally I'm trying on strauss's sid chroot to see how that
+ compares)
+ <pinotree> Dom: samuel makes buildds build perl with nocheck (otherwise
+ we'd have no perl at all)
+ <pinotree> which tests still fail?
+ <Dom> ../cpan/Sys-Syslog/t/syslog.t ../cpan/Time-HiRes/t/HiRes.t
+ ../cpan/autodie/t/recv.t ../dist/IO/t/io_pipe.t ../dist/threads/t/libc.t
+ ../dist/threads/t/stack.t ../ext/Socket/t/socketpair.t io/pipe.t
+ op/sigdispatch.t
+ <Dom> buildds> I see
+ <pinotree> ah ok, those that are failing for me even with my patches
+ <Dom> I hadn't spotted that the builds were done with nocheck.
+ <Dom> (but only sometimes...)
+ <Dom> Explains a lot
+ <pinotree> syslog is kind of non-working on hurd, and syslog.t succeeds in
+ buildds (as opposted to crash the machine...) because there's no /var/log
+ in chroots
+ <pinotree> libc.t appears to succeed too in buildds
+ * Dom notices how little memory strauss has, and cancels the build, now
+ that he *knows* that running out of memory caused the crahses
+ <pinotree> iirc HiRes.t, io_pipe.t , pipe.t and sigdispatch.t fails because
+ of trobules we have with posix signals
+ <pinotree> socketpair.t is kind of curious, it seems to block on
+ socketpair()...
+ * Dom wonders if a wiki page tracking this would be worthwhile
+ <pinotree> stack.t fails because we cannot set a different size for pthread
+ stacks, yet (similar failing test cases are also in the python test
+ suite)
+ <Dom> if there are problems which aren't going to get resolved any time
+ soon, it may be worth a few SKIPs conditional on architecture, depending
+ on how serious the issue is
+ <Dom> then we'd get better visibility of other/new issues
+ <Dom> (problems which aren't bugs in perl, that is)
+ <pinotree> understandable, yes
+ <pinotree> i think nobody digged much deeper in the failing ones yet, to
+ actually classify them as due to glibc/hurd/mach
+ <pinotree> (eg unlike the pipe behaviour in sysconf.t i reported)
+
+### 2011-11-26
+
+ <pinotree> cool, my recvfrom() fix also makes the perl test recv.t pass
+
+
---
diff --git a/open_issues/posix_fadv_volatile.mdwn b/open_issues/posix_fadv_volatile.mdwn
new file mode 100644
index 00000000..6bd25e3e
--- /dev/null
+++ b/open_issues/posix_fadv_volatile.mdwn
@@ -0,0 +1,16 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!meta title="POSIX_FADV_VOLATILE"]]
+
+[[!tag open_issue_gnumach open_issue_hurd]]
+
+May be interesting to watch what happens: LWN, Jonathan Corbet,
+[`POSIX_FADV_VOLATILE`](http://lwn.net/Articles/468896/), 2011-11-22.
diff --git a/open_issues/robustness.mdwn b/open_issues/robustness.mdwn
new file mode 100644
index 00000000..d32bd509
--- /dev/null
+++ b/open_issues/robustness.mdwn
@@ -0,0 +1,64 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_documentation open_issue_hurd]]
+
+
+# IRC, freenode, #hurd, 2011-11-18
+
+ <nocturnal> I'm learning about GNU Hurd and was speculating with a friend
+ who is also a computer enthusiast. I would like to know if Hurds
+ microkernel can recover services should they crash? and if it can, does
+ that recovery code exist in multiple services or just one core kernel
+ service?
+ <braunr> nocturnal: you should read about passive translators
+ <braunr> basically, there is no dedicated service to restore crashed
+ servers
+ <etenil> Hi everyone!
+ <braunr> services can crash and be restarted, but persistence support is
+ limited, and rather per serivce
+ <braunr> actually persistence is more a side effect than a designed thing
+ <braunr> etenil: hello
+ <etenil> braunr: translators can also be spawned on an ad-hoc basis, for
+ instance when accessing a particular file, no?
+ <braunr> that's what being passive, for a translator, means
+ <etenil> ah yeah I thought so :)
+
+
+# IRC, freenode, #hurd, 2011-11-19
+
+ <chromaticwt> will hurd ever have the equivalent of a rs server?, is that
+ even possible with hurd?
+ <youpi> chromaticwt: what is an rs server ?
+ <chromaticwt> a reincarnation server
+ <youpi> ah, like minix. Well, the main ground issue is restoring existing
+ information, such as pids of processes, etc.
+ <youpi> I don't know how minix manages it
+ <antrik> chromaticwt: I have a vision of a session manager that could also
+ take care of reincarnation... but then, knowing myself, I'll probably
+ never imlement it
+ <youpi> we do get proc crashes from times to times
+ <youpi> it'd be cool to see the system heal itself :)
+ <braunr> i need a better description of reincarnation
+ <braunr> i didn't think it would make core servers like proc able to get
+ resurrected in a safe way
+ <antrik> depends on how it is implemented
+ <antrik> I don't know much about Minix, but I suspect they can recover most
+ core servers
+ <antrik> essentially, the condition is to make all precious state be
+ constantly serialised, and held by some third party, so the reincarnated
+ server could restore it
+ <braunr> should it work across reboots ?
+ <antrik> I haven't thought about the details of implementing it for each
+ core server; but proc should be doable I guess... it's not necessary for
+ the system to operate, just for various UNIX mechanisms
+ <antrik> well, I'm not aware of the Minix implementation working across
+ reboots. the one I have in mind based on a generic session management
+ infrastructure should though :-)
diff --git a/open_issues/syslog.mdwn b/open_issues/syslog.mdwn
index 5fec38b1..2e902698 100644
--- a/open_issues/syslog.mdwn
+++ b/open_issues/syslog.mdwn
@@ -43,3 +43,30 @@ IRC, freenode, #hurd, 2011-08-08
< youpi> shm should work with the latest libc
< youpi> what won't is sysv sem
< youpi> (i.e. semget)
+
+
+IRC, OFTC, #debian-hurd, 2011-11-02:
+
+ * pinotree sighs at #645790 :/
+ <tschwinge> pinotree: W.r.t. 645790 -- yeah, ``someone'' should finally
+ figure out what's going on with syslog.
+ http://lists.gnu.org/archive/html/bug-hurd/2008-07/msg00152.html
+ <tschwinge> pinotree: And this...
+ http://lists.gnu.org/archive/html/bug-hurd/2007-02/msg00042.html
+ <pinotree> tschwinge: i did that 20 invocations tests recently, and
+ basically none of them has been logged
+ <pinotree> tschwinge: when i started playing with logger more, as result i
+ had some server that started taking all the cpu, followed by other
+ servers and in the end my ssh connection were dropped and i had nothing
+ to do (not even login from console)
+ <tschwinge> pinotree: Sounds like ``fun''. Hopefully we can manage to
+ understand (and fix the underlying issue) why a simple syslog()
+ invocation can make the whole system instable.
+ <pinotree> tschwinge: to be honest, i got havoc in the system when i told
+ syslog to manually look for /dev/log (-u /dev/log), possibly alao when
+ telling to use a datagram socket (-d)
+ <pinotree> but even if a normal syslog() invocation does not cause havoc,
+ there's still the "lost messages" issue
+ <tschwinge> Yep. What I've been doing ever since, is deinstall all
+ *syslog* packages.
+ <tschwinge> This ``fixed'' all syslog() hangs.
diff --git a/open_issues/translator_stdout_stderr.mdwn b/open_issues/translator_stdout_stderr.mdwn
index 11793582..14ea1c6d 100644
--- a/open_issues/translator_stdout_stderr.mdwn
+++ b/open_issues/translator_stdout_stderr.mdwn
@@ -11,11 +11,43 @@ License|/fdl]]."]]"""]]
[[!tag open_issue_hurd]]
+There have been several discussions and proposals already, about adding a
+suitable logging mechanism to translators, for example.
+
+
Decide / implement / fix that (all?) running (passive?) translators' output
should show up on the (Mach / Hurd) console / syslog.
+
[[!taglink open_issue_documentation]]: [[!message-id
"87oepj1wql.fsf@becket.becket.net"]]
+
[[!taglink open_issue_documentation]]: Neal once had written an email on this
topic.
+
+
+IRC, freenode, #hurd, 2011-11-06
+
+ <youpi> about CLI_DEBUG, you can use #define CLI_DEBUG(fmt, ...) {
+ fprintf(stderr, fmt, ## __VA_ARGS__); fflush(stderr); }
+ <tschwinge> Isn't stderr in auto-flush mode by default?
+ <tschwinge> man setbuf: The standard error stream stderr is always
+ unbuffered by default.
+ <youpi> tschwinge: "by default" is the important thing here
+ <youpi> in the case of translators iirc stderr is buffered
+ <tschwinge> youpi: Oh!
+ <tschwinge> That sounds wrong.
+
+
+IRC, freenode, #hurd, 2011-11-23
+
+ <braunr> we'd need a special logging task for hurd servers
+ <pinotree> if syslog would work, that could be used optionally
+ <braunr> no, it relies on services provided by the hurd
+ <braunr> i'm thinking of something using merely the mach interface
+ <braunr> e.g. using mach_msg to send log messages to a special port used to
+ reference the logging service
+ <braunr> which would then store the messages in a circular buffer in ram
+ <braunr> maybe sending to syslog if the service is available
+ <braunr> the hurd system buffer if you want
diff --git a/open_issues/translators_set_up_by_untrusted_users.mdwn b/open_issues/translators_set_up_by_untrusted_users.mdwn
index 97f48bba..044d5411 100644
--- a/open_issues/translators_set_up_by_untrusted_users.mdwn
+++ b/open_issues/translators_set_up_by_untrusted_users.mdwn
@@ -282,6 +282,9 @@ and [Hardlink
Protection](https://wiki.ubuntu.com/SecurityTeam/Roadmap/KernelHardening#Hardlink_Protection)
do bear some similarity with the issue we're discussing here.
+Likewise, Kees Cook, [fs: symlink restrictions on sticky
+directories](http://lwn.net/Articles/468215/), 2011-11-18.
+
# IRC, freenode, #hurd, 2011-08-31
diff --git a/shortcuts.mdwn b/shortcuts.mdwn
index b62b2981..fd4d4dee 100644
--- a/shortcuts.mdwn
+++ b/shortcuts.mdwn
@@ -87,6 +87,13 @@ ikiwiki will include your shortcut in the standard underlay.
* [[!shortcut name=FF_project url="http://www.fossfactory.org/project/p%s" desc="FOSS Factory bounty (p%s)"]]
+## Savannah
+
+ * [[!shortcut name=GNU_Savannah_Git_hurd_hurd
+ url="http://git.savannah.gnu.org/cgit/hurd/hurd.git/commit/?id=%s"
+ desc="hurd/hurd.git commit %s"]]
+
+
## Notmuch'n'Gmane.
* [[!shortcut name=message-id url="http://thread.gmane.org/%s" desc="""`id:"%s"`"""]]
diff --git a/user/Maksym_Planeta.mdwn b/user/Maksym_Planeta.mdwn
index 6f4bda77..502f3e20 100644
--- a/user/Maksym_Planeta.mdwn
+++ b/user/Maksym_Planeta.mdwn
@@ -7,10 +7,24 @@ any later version published by the Free Software Foundation; with no Invariant
Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
-
+
[[!toc]]
#Notes on tmpfs
+## Current state
+
+7.12.11
+
+There left 4 bugs, I found:
+
+1. Passive translator doesn't work. (Haven't investigate yet)
+
+2. After sometime of inactivity tmpfs exits. (8.12.11, Fixed)
+
+3. Writing by freed address somewhere. As workaround I set kalloc_max in mach-defpager/kalloc.c to 3, so vm_allocate is always used.
+
+4. Fsx still breaks with 11 from 12 seeds, I tested the problem was in first 4 Kb. In 12th case problem was in range between 4 Kb and 8 Kb. I find this quite odd. Usual amount of operations before fsx breaks passes 100 000.
+
## mach-defpager
[[defpager|http://www.bddebian.com:8888/~hurd-web/user/Maksym_Planeta/#defpager81111]]
@@ -21,46 +35,84 @@ License|/fdl]]."]]"""]]
My vision of the problem in short: when user tries to access to the memory backed by tmpfs first time, kernel asks defpager to initialize memory object and sets it's request port to some value. But sometimes user directly accesses to defpager and supplies it own request port. So defpager should handle different request port, what causes errors.
+TODO: pager_extend always frees old pagemap and allocates new. Consider if it is necessary.
+
## Steps
-1. Find out what causes crashes in tmpfs with defpager
+### Find out what causes crashes in tmpfs with defpager
[[http://www.gnu.org/s/hurd/hurd/translator/tmpfs/notes_various.html]]
TODO: Consider deleting of parameter "port" in function mach-defpager/default_pager.c:pager_port_list_insert
since this parameter is unused
-2. Write own pager
+Probably pager_request shouldn't be stored because request may arrive from different kernels (or from kernel and translator), so this parameter doesn't have any sense.
- 6.11.11 Reading/writing for files that fit in vm_page_size works
+22.11.11 Reading/writing for any size works, [[this|http://lists.gnu.org/archive/html/bug-hurd/2011-11/msg00127.html]] works, but fsx test fails ([[see|http://www.bddebian.com:8888/~hurd-web/user/Maksym_Planeta/#fsx_fail2211]]).
- 7.11.11 Works for any size.
+24.11.11 The problem with fsx.
-3. Make links work
+Here are follow operations:
- Symlinks behavior: [[links|http://www.bddebian.com:8888/~hurd-web/user/Maksym_Planeta/#links81111]]
+1. Write some data at address 0x100 with size 0x20
+2. Truncate file to size 0x80. First written data should be lost.
+3. Write some data at address 0x200 size of 0x20. By this operation file size is increased up to 0x220.
+4. Read data at address 0x110. Fsx expects here zeros, but in fact here is data, that was written at step 1.
- 8.11.11 Symlinks work.
+When fsx tries to read data kernel calls pager with seqno_memory_object_data_request, and pager returns on step 4 zeros either with memory_object_data_provided or memory_object_data_unavailable. Before this, in default_pager_set_size memory_object_lock_request called to flush any kernel caches, that could hold data to be truncated. When I set offset to 0 and size to limit in memory_object_lock_request it appeared another error ([[see|http://www.bddebian.com:8888/~hurd-web/user/Maksym_Planeta/#fsx_fail2411]]). Both these behaviors appear to be quite strange for me. It is quite late now, so i put these notes to not forget this and went sleep. Continue tomorrow.
-4. Control of used space by tmpfs.
+5.12.11 Here is a problem with writing by address, which was freed already. It happens in function dealloc_direct in macros invalidate_block. This function is called from pager_truncate in branch when condition "if (!INDIRECT_PAGEMAP(old_size))" is true. But I didn't find why reference to freed object is kept. As workaround we can reduce kalloc_max in hurd/mach-defpager/kalloc.c to 3 to make allocator use vm_allocate always. The drawback is that allocator will allocate only multiple of vm_page_size, but this is temporary tradeoff. Till now fsx reaches operation number 14277.
- TODO: Make tmpfs use not more space than it was allowed.
+6.12.11 fsx works quite long and doesn't interrupt. I've stopped at 124784. Continued. It broke at 181091.
+
+### Write own pager
-5. Thread safety.
+ 6.11.11 Reading/writing for files that fit in vm_page_size works
+
+ 7.11.11 Works for any size.
TODO: During execution tmpfs hangs in random places. The most possible is variant is deadlocks,
because nothing was undertaken for thread safety.
-6. After sometime of inactivity tmpfs exits.
+ TODO: Make tmpfs use not more space than it was allowed.
+
+### Make links work
+
+Symlinks behavior: [[links|http://www.bddebian.com:8888/~hurd-web/user/Maksym_Planeta/#links81111]]
+
+8.11.11 Symlinks work.
+
+[[Patch by Ben Asselstine.|http://thread.gmane.org/gmane.os.hurd.bugs/11829/focus=12098]]
+
+### After sometime of inactivity tmpfs exits.
TODO: Find out why and correct this.
-#Translators vs FUSE
+> This may perhaps be the standard translator
+> shutdown-after-a-period-of-inactivity functionality? This is of course not
+> valid in the tmpfs case, as all its state (file system content) is not stored
+> on any non-volatile backend. Can this auto-shutdown functionality be
+> disabled (in [[hurd/libdiskfs]], perhaps? See `libdiskfs/init-first.c`, the
+> call to `ports_manage_port_operations_multithread`, the `global_timeout`
+> paramenter (see the [[hurd/reference_manual]]) (here: `server_timeout`).
+> Probably this should be set to `0` in tmpfs to disable timing out?
+> --[[tschwinge]]
+
+### Passive translator doesn't work
+
+> Must be some bug: `diskfs_set_translator` is implemented in `node.c`, so this
+> is supposed to work. --[[tschwinge]]
+
+#Chalanges
+
+Translators vs FUSE:
[[What can a translator do that FUSE can't?|http://lists.gnu.org/archive/html/bug-hurd/2010-07/msg00061.html]]
[[Re: Hurd translators on FUSE|http://lists.gnu.org/archive/html/l4-hurd/2009-09/msg00146.html]]
+[[Example of sane utilization of filesystem stored in RAM|http://habrahabr.ru/blogs/gdev/131043/]] (Russian). Author of this article copied some resources of game "World of Tanks" to RAM-drive and game started load much faster. Although he used Windows in this article, this could be good example of benefits, which filesystem, stored in RAM, could give.
+
#Debugging
To debug tmpfs, using libraries from "$PWD"/lib and trace rpc:
@@ -78,8 +130,20 @@ For debugging ext2fs:
#Questions
1. What are sequence numbers? What are they used for?
+
+> See [[microkernel/mach/ipc/sequence_numbering]]. --[[tschwinge]]
+
2. Is there any way to debug mach-defpager? When I set breakpoint to any function in it, pager never breaks.
+> Is that still an unresolved problem? If not, what was the problem? --[[tschwinge]]
+
+3. Is it normal that defpager panics and breaks on any wrong data?
+
+> No server must ever panic (or reach any other invalid state), whichever input
+> data it receives from untrusted sources (which may include every possible
+> source). All input data must be checked for validity before use; anything
+> else is a bug. --[[tschwinge]]
+
#Links
1. [[Cthreads manuals|http://www.cc.gatech.edu/classes/cs6432_99_winter/threads_man/]]
@@ -93,11 +157,8 @@ For debugging ext2fs:
and its target (relative to its directory) is foo/bar (which would mean /tmp/foo/foo/bar in canonical form)
(10:29:42) braunr: youpi: tschwinge: what did ludovic achieve ?
(10:30:06) tschwinge: mcsim: As Richard says, symlink targets are always relative to the directory they're contained in.
- (10:30:51) tschwinge: braunr: This Hydra/Nix (I wtill mix it all up) thing is kind of a package managing system.
- (10:31:17) tschwinge: He has written scripts for bootstrapping a Hurd toolchain.
(10:31:26) braunr: oh ok
(10:31:27) mcsim: so, if I want to create link in cd, first I need to cd there?
- (10:31:28) tschwinge: And then uses that to build a whole bootable image.
(10:31:36) mcsim: in foo*
(10:31:36) braunr: mcsim: just provide the right paths
(10:32:11) braunr: $ touch foo/bar
@@ -147,3 +208,52 @@ For debugging ext2fs:
on system performance
(15:56:54) antrik: both can be avoided by just passing a real anonymous memory object, i.e. one provided by the defpager
(15:57:07) antrik: only problem is that the current defpager implementation can't really handle that...
+
+#Pastes
+
+## fsx test on 22.11.11 <a id="fsx_fail2211"/>
+ $ ~/src/fsx/fsx -W -R -t 4096 -w 4096 -r 4096 bar
+ mapped writes DISABLED
+ truncating to largest ever: 0x32000
+ truncating to largest ever: 0x39000
+ READ BAD DATA: offset = 0x16000, size = 0xd9a0
+ OFFSET GOOD BAD RANGE
+ 0x1f000 0x0000 0x01a0 0x 2d14
+ operation# (mod 256) for the bad data may be 1
+ LOG DUMP (16 total operations):
+ 1(1 mod 256): WRITE 0x1f000 thru 0x21d28 (0x2d29 bytes) HOLE ***WWWW
+ 2(2 mod 256): WRITE 0x10000 thru 0x15bfe (0x5bff bytes)
+ 3(3 mod 256): READ 0x1d000 thru 0x21668 (0x4669 bytes)
+ 4(4 mod 256): READ 0xa000 thru 0x16a59 (0xca5a bytes)
+ 5(5 mod 256): READ 0x8000 thru 0x8f2c (0xf2d bytes)
+ 6(6 mod 256): READ 0xa000 thru 0x17fe8 (0xdfe9 bytes)
+ 7(7 mod 256): READ 0x1b000 thru 0x20f33 (0x5f34 bytes)
+ 8(8 mod 256): READ 0x15000 thru 0x1c05b (0x705c bytes)
+ 9(9 mod 256): TRUNCATE UP from 0x21d29 to 0x32000
+ 10(10 mod 256): READ 0x3000 thru 0x5431 (0x2432 bytes)
+ 11(11 mod 256): WRITE 0x29000 thru 0x34745 (0xb746 bytes) EXTEND
+ 12(12 mod 256): TRUNCATE DOWN from 0x34746 to 0x19000 ******WWWW
+ 13(13 mod 256): READ 0x14000 thru 0x186d8 (0x46d9 bytes)
+ 14(14 mod 256): TRUNCATE UP from 0x19000 to 0x39000 ******WWWW
+ 15(15 mod 256): WRITE 0x28000 thru 0x3548c (0xd48d bytes)
+ 16(16 mod 256): READ 0x16000 thru 0x2399f (0xd9a0 bytes) ***RRRR***
+ Correct content saved for comparison
+ (maybe hexdump "bar" vs "bar.fsxgood")
+
+## fsx test on 24.11.11 <a id="fsx_fail2411"/>
+
+ $ ~/src/fsx/fsx bar
+ truncating to largest ever: 0x13e76
+ READ BAD DATA: offset = 0x1f62e, size = 0x2152
+ OFFSET GOOD BAD RANGE
+ 0x1f62e 0x0206 0x0000 0x 213e
+ operation# (mod 256) for the bad data unknown, check HOLE and EXTEND ops
+ LOG DUMP (6 total operations):
+ 1(1 mod 256): TRUNCATE UP from 0x0 to 0x13e76
+ 2(2 mod 256): WRITE 0x17098 thru 0x26857 (0xf7c0 bytes) HOLE ***WWWW
+ 3(3 mod 256): READ 0xc73e thru 0x1b801 (0xf0c4 bytes)
+ 4(4 mod 256): MAPWRITE 0x32e00 thru 0x331fc (0x3fd bytes)
+ 5(5 mod 256): MAPWRITE 0x7ac1 thru 0x11029 (0x9569 bytes)
+ 6(6 mod 256): READ 0x1f62e thru 0x2177f (0x2152 bytes) ***RRRR***
+ Correct content saved for comparison
+ (maybe hexdump "bar" vs "bar.fsxgood")
diff --git a/user/musial.mdwn b/user/musial.mdwn
index 24a526be..d9aaa599 100644
--- a/user/musial.mdwn
+++ b/user/musial.mdwn
@@ -8,12 +8,8 @@ Sections, no Front-Cover Texts, and no Back-Cover Texts. A copy of the license
is included in the section entitled [[GNU Free Documentation
License|/fdl]]."]]"""]]
-Robert Musial
+Robert Musial - Cleveland, OH
-Cleveland, OH
+musial/at/gnu/dot/org
-musial [at] gnu [dot] org
-
-http://musial.sdf.org
-
-http://github.com/musial
+http://musial.dyndns.info:8080/