From 04866256970d04ba76f9c94e2cc74fed025776a9 Mon Sep 17 00:00:00 2001
From: Thomas Schwinge <thomas@schwinge.name>
Date: Thu, 1 Apr 2010 10:34:24 +0200
Subject: public_hurd_boxen/installation/snubber: Talk about
 /var/www/robots.txt.

---
 public_hurd_boxen/installation/snubber.mdwn | 18 ++++++++++++++++++
 1 file changed, 18 insertions(+)

diff --git a/public_hurd_boxen/installation/snubber.mdwn b/public_hurd_boxen/installation/snubber.mdwn
index f9542a9f..dbbade9f 100644
--- a/public_hurd_boxen/installation/snubber.mdwn
+++ b/public_hurd_boxen/installation/snubber.mdwn
@@ -30,6 +30,24 @@ License|/fdl]]."]]"""]]
     $ mkdir tmp/backup && chmod 0733 tmp/backup
 
 
+# `/var/www/robots.txt`
+
+This file used to contain:
+
+    User-agent: *
+    Disallow: /gitweb/
+    Disallow: /cgi-bin/
+
+... which I've now changed to:
+
+    User-agent: *
+    Disallow: /
+
+The goal is that robots rather index the official pages,
+<http://www.gnu.org/software/hurd/>, instead of the staging area on
+<http://www.bddebian.com:8888/~hurd-web/>.
+
+
 # Restore Backup
 
 ## `/etc/apache2/mods-enabled/`
-- 
cgit v1.2.3