Merge branch 'master' of flubber:~hurd-web/hurd-web

12 files changed, 277 insertions, 8 deletions

diff --git a/hurd/console.mdwn b/hurd/console.mdwn
index 4f976efd..f7230011 100644
--- a/hurd/console.mdwn
+++ b/hurd/console.mdwn
@@ -1,3 +1,88 @@
+[[!meta copyright="Copyright © 2002, 2003, 2004, 2005, 2006, 2007, 2009, 2011
+Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+The Hurd console's implementation is broken into two pieces each running on
+it's own process, the console client and server.
+
+The console client is also split into modules (input driver, display driver,
+speaker, ...) but they all run in the same process.
+
+The console server puts itself as a translator on top of `/dev/vcs` folder
+presenting the following hierarchy:
+
+       + /dev/vcs
+        \ 
+         +- 1
+             \
+              +- console
+              +- input
+              +- display
+         +- ..
+         +- n
+
+where the numbered nodes represent virtual consoles and their contents are all
+alike.
+
+As the following graph shows, the console, input and display nodes are the
+interfaces used by the terminal server, input driver and display drivers
+respectively.
+
+       +------------------+                                 +-----------------+
+       |   Input driver   |                                 | Terminal Server |
+       |                  |                                 |                 |
+       |      pc-kbd      |                                 |                 |
+       +------------------+                                 +-----------------+
+                | _cons_vcons_input                                 |
+                | writes to                                   reads |
+                | vcs/i/input                         vcs/i/console |
+                |                +-----------------+                |
+                |                | Console Server  |                |
+                |                |  /hurd/console  |                |
+                |  input_enqueue | --------------- | input_dequeue  |
+                +--------------->|   Input Queue   |>---------------+
+                                 | --------------- |
+                +--------------->|  Output Buffer  |>---------------+
+                |                +-----------------+                |
+                |                                                   |
+                | writes                                      reads |
+                | vcs/i/console                       vcs/i/display |
+                |                                                   |
+       +----------------+                                   +-----------------+
+       | Teminal Server |                                   | Display driver  |
+       |                |                                   |                 |
+       |   /hurd/term   |                                   |      vga        |
+       +----------------+                                   +-----------------+
+
+The input driver takes scancodes from the in-kernel kbd queue, translates them
+into characters and writes them to the input node.  Then the terminal server
+reads the console node taking the characters out of the console server.
+
+Each of theese actions is actually an RPC handled by the translator on
+`/dev/vcs`.  Writes to input nodes are handled by calling `input_enqueue` to
+put the character into a queue.  And reads from console nodes are handled by
+calling `input_dequeue` which takes out charecters from the queue and gives
+them to the reader.
+
+It's important to note here that both `input_enqueue` and `input_dequeue` are
+blocking operations and a blocked `input_dequeue` necessarily needs an
+`input_enqueue` call to continue.
+
+[[RPC]]s are handled by the console server with the help of [[hurd/libports]]'
+`ports_manage_multithreaded` API.
+
+
+---
+
+/!\ old content; [[!taglink open_issue_documentation]]: cleanup needed.
+
 The below is a reworked version of Marcus Brinkmann's [letter to the debian-hurd list](http://lists.debian.org/debian-hurd/2002/debian-hurd-200209/msg00054.html). It describes how to setup the new console server for the Hurd. I am testing this right now, so this document is a work in progress.
 
 -- [[Main/JoachimNilsson]] - 21 Jan 2003
diff --git a/hurd/dde/guide.mdwn b/hurd/dde/guide.mdwn
index a3c08754..6a83519c 100644
--- a/hurd/dde/guide.mdwn
+++ b/hurd/dde/guide.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2010,2011 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]]
 
 [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
 id="license" text="Permission is granted to copy, distribute and/or modify this
diff --git a/hurd/debugging/rpctrace.mdwn b/hurd/debugging/rpctrace.mdwn
index f7136056..fd24f081 100644
--- a/hurd/debugging/rpctrace.mdwn
+++ b/hurd/debugging/rpctrace.mdwn
@@ -52,6 +52,43 @@ See `rpctrace --help` about how to use it.
         <antrik> note that there is a number of known bugs in rpctrace, for which zhengda has sent patches... though I haven't reviewed all of them I think
         <antrik> there are some nasty Mach operations that are really hard to proxy -- but I don't think the auth mechanism needs any of these...
 
+* IRC, freenode, #hurd, 2011-11-04
+
+    [[!taglink open_issue_documentation]]
+
+        <mcsim> hello. Are there any documentation about understanding output
+          of rpctrace?
+        <braunr> no
+        <braunr> you should read the source code, best doc available
+        <braunr> if you have too many numbers and almost no symbolc names,
+          you're lacking rpc definition lists
+        <braunr> check that the gnumach-common package is installed, as it
+          provides the gnumach definitions
+        <braunr> (the glibc ones are almost always available)
+        <braunr> with those two, you should be fine for the beginning
+        <mcsim> gnumach-common is installed. And what is the name for glibc
+          package for gnumach definitions.
+        <mcsim> Also I'm using libraries specified by LD_LIBRARY_PATH. Does it
+          make influence on absence of symbolic names?
+        <braunr> no
+        <braunr> rpctrace --help
+        <braunr> see the --rpc-list=FILE option
+        <braunr> the default lists are in /usr/share/msgids/, with the .msgids
+          extension
+        <braunr> $ dpkg -S msgids
+        <braunr> gnumach-common: /usr/share/msgids/gnumach.msgids
+        <braunr> hurd: /usr/share/msgids/hurd.msgids
+        <braunr> ok, glibc has none, it's the hurd
+        <braunr> for more details about the output, read the source code
+        <braunr> it shouldn't be that hard to grasp
+        <mcsim> -I /usr/share/msgids helped
+        <mcsim> thank you
+        <braunr> it shouldn't have, it's the default path
+        <mcsim> but symbolic names appeared
+        <braunr> well, that's weird :)
+        <pinotree> braunr: the output of rpctrace --help should tell the
+          default dir for msgids
+
 
 # See Also
 
diff --git a/hurd/io_path.mdwn b/hurd/io_path.mdwn
index 492edffe..c47b5dca 100644
--- a/hurd/io_path.mdwn
+++ b/hurd/io_path.mdwn
@@ -46,9 +46,9 @@ Back in `_Xio_read`.
 
 If the 2048 byte buffer is not decided to be used (out-of-line case or bigger
 than 2048 bytes case; server decides to instead provide a new memory region),
-the [[`dealloc`|microkernel/mach/mig/dealloc]] flag is being set, which causes
-Mach to unmap that memory region from the server's address space, i.e., doing a
-memory *move* from the server to the client.
+the [[`dealloc`|microkernel/mach/mig/documentation/dealloc]] flag is being set,
+which causes Mach to unmap that memory region from the server's address space,
+i.e., doing a memory *move* from the server to the client.
 
 Leave server-side RPC stub `_Xio_read`.
 
diff --git a/hurd/libpager.mdwn b/hurd/libpager.mdwn
index 99f28f2a..d844743d 100644
--- a/hurd/libpager.mdwn
+++ b/hurd/libpager.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2007, 2008, 2010 Free Software Foundation,
+[[!meta copyright="Copyright © 2007, 2008, 2010, 2011 Free Software Foundation,
 Inc."]]
 
 [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
@@ -6,8 +6,8 @@ id="license" text="Permission is granted to copy, distribute and/or modify this
 document under the terms of the GNU Free Documentation License, Version 1.2 or
 any later version published by the Free Software Foundation; with no Invariant
 Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
-is included in the section entitled
-[[GNU Free Documentation License|/fdl]]."]]"""]]
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
 
 Mach's [[microkernel/mach/external_pager_mechanism]].
 
@@ -16,6 +16,16 @@ Mach [[microkernel/mach/IPC]]'s [[microkernel/mach/ipc/sequence_numbering]].
 [GNU Hurd Reference Manual: 4.2 Pager
 Library](http://www.gnu.org/software/hurd/doc/hurd_5.html#SEC32).
 
+
+# Writeback: Writing Out Dirty Pages
+
+
+## Related
+
+  * LWN, Jonathan Corbet, [*No-I/O dirty
+    throttling*](http://lwn.net/Articles/456904/), 2011-08-31.
+
+
 # Open Issues
 
   * [[open_issues/linux_vmsig]]
diff --git a/hurd/libports.mdwn b/hurd/libports.mdwn
index 28274338..6f2cd46d 100644
--- a/hurd/libports.mdwn
+++ b/hurd/libports.mdwn
@@ -1,4 +1,4 @@
-[[!meta copyright="Copyright © 2010 Free Software Foundation, Inc."]]
+[[!meta copyright="Copyright © 2010, 2011 Free Software Foundation, Inc."]]
 
 [[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
 id="license" text="Permission is granted to copy, distribute and/or modify this
@@ -18,3 +18,20 @@ provide an interface independently of the underlying [[microkernel]].
 *libports* does not itself depend on *[[libthreads]]*, but the appropriate
 threading hooks are used if present, that is if *[[libthreads]]* is used by
 another component.
+
+
+# Message Processing
+
+## `ports_manage_multithreaded`
+
+When a message is recieved, the thread acting as receiver checks if any other
+thread is also waiting for requests.  If there is none, a new thread is
+spawned.  Thus, the current thread continues processing the message while the
+newly created thread starts listening for new ones.  ([[!taglink
+open_issue_hurd]]: [[open_issues/multithreading]].)
+
+Also, there are configurable timeouts for [[translator]]s who want to go away
+when they are not used.  ([[!taglink open_issue_hurd]]: there used to be bugs
+in this area, [[!message-id "87hev152we.fsf@becket.becket.net"]], but it may be
+fixed as of [[!message-id "20111030210045.GA4983@myhost"]],
+[[!GNU_Savannah_Git_hurd_hurd 9b5429e834cde56f73b8ff605e36afc7d9bb6e1b]].)
diff --git a/hurd/libthreads.mdwn b/hurd/libthreads.mdwn
new file mode 100644
index 00000000..8b1a97e6
--- /dev/null
+++ b/hurd/libthreads.mdwn
@@ -0,0 +1,28 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+`libthreads` a.k.a. C threads.
+
+
+# Internals
+
+
+## Threads' Death
+
+C threads death doesn't actually free the thread's stack (and maybe not the
+associated Mach ports either).  That's because there's no way to free the stack
+after the thread dies (because the thread of control is gone); the stack needs
+to be freed by something else, and there's nothing convenient to do it.  There
+are many ways to make it work.
+
+However, it isn't really a leak, because the unfreed resources do get used for
+the next thread.  So the issue is that the shrinkage of resource consumption
+never happens, but it doesn't grow without bounds; it just stays at the maximum
+even if the current number of threads is lower.
diff --git a/hurd/running/gnu/universal_package_manager.mdwn b/hurd/running/gnu/universal_package_manager.mdwn
index 58841b02..bf1b92e0 100644
--- a/hurd/running/gnu/universal_package_manager.mdwn
+++ b/hurd/running/gnu/universal_package_manager.mdwn
@@ -155,3 +155,4 @@ To join the project just list your name below.
    8. Abhradip Mukherjee
    9. Ermenegildo Fiorito
    10. Oltion Doda
+   11. Russell James
diff --git a/hurd/translator.mdwn b/hurd/translator.mdwn
index bf7af3ce..3527267f 100644
--- a/hurd/translator.mdwn
+++ b/hurd/translator.mdwn
@@ -80,6 +80,7 @@ The [[concept|concepts]] of translators creates its own problems, too:
 
 # Existing Translators
 
+* [[hello]]
 * [[auth]]
 * [[exec]]
 * [[pfinet]]
diff --git a/hurd/translator/hello.mdwn b/hurd/translator/hello.mdwn
new file mode 100644
index 00000000..bd56cd76
--- /dev/null
+++ b/hurd/translator/hello.mdwn
@@ -0,0 +1,14 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+The *hello* translator is an example of a simple [[libtrivfs]]-based one-node
+[[translator]].  It is shipped as part of the [[Hurd source code
+repository|source_repositories]], and exists in a single-threaded and a
+multi-threaded variant.
diff --git a/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn
index a9317c21..5228515f 100644
--- a/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn
+++ b/hurd/translator/tmpfs/tmpfs_vs_defpager.mdwn
@@ -233,3 +233,40 @@ See also: [[open_issues/resource_management_problems/pagers]].
       I have never looked at it.
 
 [[open_issues/mach-defpager_vs_defpager]].
+
+
+# IRC, freenode, #hurd, 2011-11-08
+
+    <mcsim> who else uses defpager besides tmpfs and kernel?
+    <braunr> normally, nothing directly
+    <mcsim> than why tmpfs should use defpager?
+    <braunr> it's its backend
+    <braunr> backign store rather
+    <braunr> the backing store of most file systems are partitions
+    <braunr> tmpfs has none, it uses the swap space
+    <mcsim> if we allocate memory for tmpfs using vm_allocate, will it be able
+      to use swap partition?
+    <braunr> it should
+    <braunr> vm_allocate just maps anonymous memory
+    <braunr> anonymous memory uses swap space as its backing store too
+    <braunr> but be aware that this part of the vm system is known to have
+      deficiencies
+    <braunr> which is why all mach based implementations have rewritten their
+      default pager
+    <mcsim> what kind of deficiencies?
+    <braunr> bugs
+    <braunr> and design issues, making anonymous memory fragmentation horrible
+    <antrik> mcsim: vm_allocate doesn't return a memory object; so it can't be
+      passed to clients for mmap()
+    <mcsim> antrik: I use vm_allocate in pager_read_page
+    <antrik> mcsim: well, that means that you have to actually implement a
+      pager yourself
+    <antrik> also, when the kernel asks the pager to write back some pages, it
+      expects the memory to become free. if you are "paging" to ordinary
+      anonymous memory, this doesn't happen; so I expect it to have a very bad
+      effect on system performance
+    <antrik> both can be avoided by just passing a real anonymous memory
+      object, i.e. one provided by the defpager
+    <antrik> only problem is that the current defpager implementation can't
+      really handle that...
+    <antrik> at least that's my understanding of the situation
diff --git a/hurd/virtual_file_system/discussion.mdwn b/hurd/virtual_file_system/discussion.mdwn
new file mode 100644
index 00000000..9e12d01e
--- /dev/null
+++ b/hurd/virtual_file_system/discussion.mdwn
@@ -0,0 +1,39 @@
+[[!meta copyright="Copyright © 2011 Free Software Foundation, Inc."]]
+
+[[!meta license="""[[!toggle id="license" text="GFDL 1.2+"]][[!toggleable
+id="license" text="Permission is granted to copy, distribute and/or modify this
+document under the terms of the GNU Free Documentation License, Version 1.2 or
+any later version published by the Free Software Foundation; with no Invariant
+Sections, no Front-Cover Texts, and no Back-Cover Texts.  A copy of the license
+is included in the section entitled [[GNU Free Documentation
+License|/fdl]]."]]"""]]
+
+[[!tag open_issue_documentation]]
+
+IRC, freenode, #hurd, 2011-11-12:
+
+    <sea4ever> So hurd implements a 'transparent translator' somewhere which
+      just passes all IO calls to the posix IO I'm used to? (i.e. read, write,
+      open, close, etc.?)
+    <youpi> it's the normal way of operation
+    <youpi> glibc's read() doesn't do a system call, it always does an RPC to
+      the underlying translator
+    <youpi> be it ext2fs for /, or your foobarfs for your node
+    <sea4ever> Ok that makes sense. How does one program know which translator
+      it should refer to though?
+    <sea4ever> the read() call magically knows which process to invoke?
+    <youpi> the / translator is always known
+    <youpi> and then you ask /'s translator about /home, then /home/you, then
+      /home/you/foobar
+    <youpi> it tells you which other translator tyou have to contact
+    <youpi> that's on open
+    <sea4ever> It's a tree! Ok.
+    <youpi> the notion of fd is then simply knowing the translator
+    <sea4ever> Right. 'file descriptor' is now 'translator address descriptor'
+      maybe.
+    <youpi> it's glibc which knows about FDs, nothing else knows
+    <youpi> yes
+    <youpi> actually an RPC port, simply
+    <sea4ever> I want to try out the new RPC mechanism that mach implements
+    <youpi> err, which "new" RPC ?
+    <youpi> mach's RPCs are very old actually :)