[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
[or-cvs] r21059: {projects} Some cleanups, updated the docs, delete some items from the (in projects/gettor: . lib/gettor)
Author: kaner
Date: 2009-11-29 14:09:46 -0500 (Sun, 29 Nov 2009)
New Revision: 21059
Modified:
projects/gettor/GetTor.py
projects/gettor/README
projects/gettor/README.locale-spec-draft
projects/gettor/TODO
projects/gettor/lib/gettor/requests.py
projects/gettor/lib/gettor/responses.py
Log:
Some cleanups, updated the docs, delete some items from the TODO list
Modified: projects/gettor/GetTor.py
===================================================================
--- projects/gettor/GetTor.py 2009-11-29 11:00:27 UTC (rev 21058)
+++ projects/gettor/GetTor.py 2009-11-29 19:09:46 UTC (rev 21059)
@@ -73,7 +73,6 @@
except Exception, e:
log.error("Sending the reply failed.")
processFail(conf, rawMessage, replyTo, e)
- raise
return False
def processOptions(options, conf):
Modified: projects/gettor/README
===================================================================
--- projects/gettor/README 2009-11-29 11:00:27 UTC (rev 21058)
+++ projects/gettor/README 2009-11-29 19:09:46 UTC (rev 21059)
@@ -1,7 +1,7 @@
OVERVIEW
--------
-GetTor is a program for serving Tor and related files over SMTP. Users
-interact with GetTor by sending it an email.
+GetTor is a program for serving Tor and related files over SMTP. Users interact
+with GetTor by sending it email.
The following document explains how to setup GetTor for a server admin.
@@ -9,7 +9,7 @@
-------------
To deploy GetTor, you'll want a machine that invokes .forward files
-for users. You'll also want to have python and rsync installed.
+for users. You'll also want to have python (>= 2.5) and rsync installed.
There are some limits with smtp software for outgoing email sizes. You should
check your software to ensure that you can send files that are as large
@@ -61,8 +61,8 @@
2009-01-05 16:49:29,226 (9222) Syncing Tor packages done.
2009-01-05 16:49:30,155 (9222) Building packages done.
-Now you'll install the cronjob. This clears the blacklist and updates
-packages daily:
+Now you'll install the cronjob. This clears the blacklist and updates packages
+daily:
gettor@moria:~/opt/gettor$ ./GetTor.py -i
2009-01-05 17:34:53,911 (16646) Installing cron donee
@@ -74,8 +74,7 @@
Finally, you need to setup email forwarding to the GetTor bot like so:
- gettor@moria:~$ cat .forward
- "|/home/gettor/opt/gettor/GetTor.py"
+ gettor@moria:~$ echo "|/home/gettor/opt/gettor/GetTor.py" > ~/.forward
Now GetTor.py is installed, prepared and ready to serve files. Send it email!
@@ -100,21 +99,17 @@
than i18n/ (relative to GetTor.py, as provided by GetTor), you can tell this
to GetTor as follows:
- gettor@moria:~/opt/gettor$ ./GetTor.py -r -d /path/to/my/mofiles/directory
+ gettor@moria:~/opt/gettor$ ./GetTor.py -r -d /path/to/my/pofiles/directory
Note that GetTor will expect the same directory structure as provided under
-i18n/ in the GetTor source package, e.g. 'i18n/en/gettor_en.po',
-'i18n/de/gettor_de.po', and so on.
+i18n/ in the GetTor source package, e.g. 'i18n/en/gettor.po',
+'i18n/de/gettor.po', and so on.
To add a new language to GetTor, one must create a new .po file in subversion
in the proper directory. For Korean, one would create i18n/ko/ and add the
-untranslated file (called gettor_ko.po) to that directory. This file should be
+untranslated file (called gettor.po) to that directory. This file should be
translated but partial translations are allowed. Once this file has been
-created, the GetTor program does not need to be modified. It does however
-require that one adds a configuration option indicating the newly desired
-locale (the default is English):
-
- locale = ko
+created, the GetTor program does not need to be modified.
However, if a user wished to add support in responses to users, they will need
to modiy the 'requests.py' file like so:
@@ -154,7 +149,7 @@
packDir: Processed Tor packages ready to be handed out to the user will
be found here
srcEmail: The email containing the Tor package will use this as 'From:'
-locale: Choose your default mail and log locale
+locale: Choose your default mail locale
logFile: If 'file' logging is chosen, log to this file
logSubSystem: This has to be one of the following strings:
'nothing': Nothing is logged anywhere (Recommended)
@@ -214,6 +209,7 @@
Some email providers may not be happy with the files GetTor sends.
Gmail provides documentation of their incoming and outgoing file sizes. It is
currently 20MB (though we have sent around 21MB without issue):
+
http://mail.google.com/support/bin/answer.py?hl=en&answer=8770
Other providers are not as transparent and this may cause failures.
Modified: projects/gettor/README.locale-spec-draft
===================================================================
--- projects/gettor/README.locale-spec-draft 2009-11-29 11:00:27 UTC (rev 21058)
+++ projects/gettor/README.locale-spec-draft 2009-11-29 19:09:46 UTC (rev 21059)
@@ -26,21 +26,14 @@
If a user wishes, they may set the locale for all responses. The locale must be
passed by the end user as a single line. The start of the line must begin with
-'Locale: ' and be followed by the desired locale. The locale must be at the end
+'Lang: ' and be followed by the desired locale. The locale must be at the end
of the line. An example follows on the following line:
-Locale: de
+
+ Lang: de
The first valid Locale line found in the message will designate the response
locale.
-HIGH TECH LOCALE PARSING
-------------------------
-
-If a user is able to set custom mail headers, they may designate their desired
-locale by specifying it with an 'X-' header of 'X-LOCALE'. This must be
-seperated by a space and followed by the desired locale. An example follows:
-X-LOCALE: de
-
AUTOMATED LOCALE PARSING
------------------------
Modified: projects/gettor/TODO
===================================================================
--- projects/gettor/TODO 2009-11-29 11:00:27 UTC (rev 21058)
+++ projects/gettor/TODO 2009-11-29 19:09:46 UTC (rev 21059)
@@ -1,7 +1,5 @@
These are planned changes to the gettor system.
-- Investigate the usage of gettor+lang@xxxxxxxxxxxxxx email addresses for
- letting users choose a reply and tor package language
- Testing and proposals to better support users with limited email bandwidth
(multi-part download, etc.).
- Split (at least) tiger bundle into several smaller archives to avoid ~20MB
@@ -24,7 +22,6 @@
- Package names that are sent out to the user are currently hard-coded. Return
to a more dynamic approach, also: Add a nice little note about the size of
the files users may request
-- Fix rsync to follow symlinks properly. We want the data not a link to data.
- Remove 'localhost:25' to send mail and use '/usr/bin/sendmail' instead
(suggested by weasel)
- Implement test (-t switch) functionality
Modified: projects/gettor/lib/gettor/requests.py
===================================================================
--- projects/gettor/lib/gettor/requests.py 2009-11-29 11:00:27 UTC (rev 21058)
+++ projects/gettor/lib/gettor/requests.py 2009-11-29 19:09:46 UTC (rev 21059)
@@ -26,7 +26,7 @@
class requestMail:
defaultLang = "en"
- # XXX Change this or remove this
+ # XXX Move this to the config file
supportedLangs = { "en": "English",
"fa": "Farsi",
"de": "Deutsch",
@@ -62,24 +62,11 @@
self.toAddress = self.parsedMessage["to"]
log.info("User made request to %s" % self.toAddress)
# Check if we got a '+' address
- match = re.search('(?<=\+)\w+', self.toAddress)
- if match:
- # Cut back and front
- splitFrontPart = self.toAddress.split('@')
- assert len(splitFrontPart) > 0, "Splitting To: address failed"
- splitLang = splitFrontPart[0].rsplit('+')
- assert len(splitLang) > 1, "Splitting for language failed"
- self.replyLocale = splitLang[1]
- # Mark this request so that we might be able to take decisions
- # later
- self.gotPlusReq = True
- log.info("User requested language %s" % self.replyLocale)
- else:
- log.info("Not a 'plus' address")
+ self.matchPlusAddress()
# TODO XXX:
# Scrub this data
self.replytoAddress = self.parsedMessage["from"]
- assert self.replytoAddress is not None, "Replyto address is None"
+ assert self.replytoAddress is not None, "No 'from' field in mail"
# If no package name could be recognized, use 'None'
self.returnPackage = None
self.splitDelivery = False
@@ -91,64 +78,111 @@
def parseMail(self):
# Parse line by line
for line in email.Iterators.body_line_iterator(self.parsedMessage):
- # Remove quotes
+ # Skip quotes
if line.startswith(">"):
continue
# Strip HTML from line
# XXX: Actually we should rather read the whole body into a string
# and strip that. -kaner
line = self.stripTags(line)
- # XXX This is a bit clumsy, but i cant think of a better way
- # currently. A map also doesnt really help i think. -kaner
- for package in self.packages.keys():
- matchme = ".*" + package + ".*"
- match = re.match(matchme, line)
- if match:
- self.returnPackage = package
- log.info("User requested package %s" % self.returnPackage)
- break
- # If we find 'split' somewhere in the mail, we assume that the user
- # wants a split delivery
- match = re.match(".*split.*", line)
- if match:
- self.splitDelivery = True
- log.info("User requested a split delivery")
+ # Check for package name in line
+ self.matchPackage(line)
+ # Check for split delivery in line
+ self.matchSplit(line)
# Change locale only if none is set so far
if not self.gotPlusReq:
- match = re.match(".*[Ll]ang:\s+(.*)$", line)
- if match:
- self.replyLocale = match.group(1)
- log.info("User requested locale %s" % self.replyLocale)
+ self.matchLang(line)
# Check if this is a command
- match = re.match(".*[Cc]ommand:\s+(.*)$", line)
- if match:
- log.info("Command received from %s" % self.replytoAddress)
- cmd = match.group(1).split()
- length = len(cmd)
- assert length == 3, "Wrong command syntax"
- auth = cmd[0]
- # Package is parsed by the usual package parsing mechanism
- package = cmd[1]
- address = cmd[2]
- verified = gettor.utils.verifyPassword(self.config, auth)
- assert verified == True, \
- "Unauthorized attempt to command from: %s" \
- % self.replytoAddress
- self.commandaddress = address
+ self.matchCommand(line)
+ self.checkLang()
+
if self.returnPackage is None:
log.info("User didn't select any packages")
+ else:
+ # This should go sometime XXX
+ self.torSpecialPackageExpansion()
+
+ return (self.replytoAddress, self.replyLocale, self.returnPackage, \
+ self.splitDelivery, self.signature, self.commandaddress)
+
+ def matchPlusAddress(self):
+ match = re.search('(?<=\+)\w+', self.toAddress)
+ if match:
+ # Cut back and front
+ splitFrontPart = self.toAddress.split('@')
+ assert len(splitFrontPart) > 0, "Splitting To: address failed"
+ splitLang = splitFrontPart[0].rsplit('+')
+ assert len(splitLang) > 1, "Splitting for language failed"
+ self.replyLocale = splitLang[1]
+ # Mark this request so that we might be able to take decisions
+ # later
+ self.gotPlusReq = True
+ log.info("User requested language %s" % self.replyLocale)
+ else:
+ log.info("Not a 'plus' address")
+
+ def matchPackage(self, line):
+ # XXX This is a bit clumsy, but i cant think of a better way
+ # currently. A map also doesnt really help i think. -kaner
+ for package in self.packages.keys():
+ matchme = ".*" + package + ".*"
+ match = re.match(matchme, line)
+ if match:
+ self.returnPackage = package
+ log.info("User requested package %s" % self.returnPackage)
+ break
+
+ def matchSplit(self, line):
+ # If we find 'split' somewhere we assume that the user wants a split
+ # delivery
+ match = re.match(".*split.*", line)
+ if match:
+ self.splitDelivery = True
+ log.info("User requested a split delivery")
+
+ def matchLang(self, line):
+ match = re.match(".*[Ll]ang:\s+(.*)$", line)
+ if match:
+ self.replyLocale = match.group(1)
+ log.info("User requested locale %s" % self.replyLocale)
+
+ def matchCommand(self, line):
+ match = re.match(".*[Cc]ommand:\s+(.*)$", line)
+ if match:
+ log.info("Command received from %s" % self.replytoAddress)
+ cmd = match.group(1).split()
+ length = len(cmd)
+ assert length == 3, "Wrong command syntax"
+ auth = cmd[0]
+ # Package is parsed by the usual package parsing mechanism
+ package = cmd[1]
+ address = cmd[2]
+ verified = gettor.utils.verifyPassword(self.config, auth)
+ assert verified == True, \
+ "Unauthorized attempt to command from: %s" \
+ % self.replytoAddress
+ self.commandaddress = address
+
+ def checkLang(self):
# Actually use a map here later XXX
for (key, lang) in self.supportedLangs.items():
if self.replyLocale == key:
+ log.info("User requested language %s" % self.replyLocale)
break
else:
log.info("Requested language %s not supported. Falling back to %s" \
% (self.replyLocale, self.defaultLang))
self.replyLocale = self.defaultLang
- return (self.replytoAddress, self.replyLocale, self.returnPackage, \
- self.splitDelivery, self.signature, self.commandaddress)
+ def torSpecialPackageExpansion(self):
+ # If someone wants one of the localizable packages, add language
+ # suffix. This isn't nice because we're hard-coding package names here
+ # Attention: This needs to correspond to the packages in packages.py
+ if self.returnPackage == "tor-browser-bundle" \
+ or self.returnPackage == "tor-im-browser-bundle":
+ # "tor-browser-bundle" => "tor-browser-bundle_de"
+ self.returnPackage = self.returnPackage + "_" + self.replyLocale
def stripTags(self, string):
"""Simple HTML stripper"""
Modified: projects/gettor/lib/gettor/responses.py
===================================================================
--- projects/gettor/lib/gettor/responses.py 2009-11-29 11:00:27 UTC (rev 21058)
+++ projects/gettor/lib/gettor/responses.py 2009-11-29 19:09:46 UTC (rev 21059)
@@ -39,18 +39,11 @@
self.config = config
self.srcEmail = "GetTor <gettor@xxxxxxxxxxxxxx>"
self.replyTo = replyto
- assert self.replyTo is not None, "Empty replyto address."
+ assert self.replyTo is not None, "Empty reply address."
# Default lang is en
if lang is None:
lang = "en"
self.mailLang = lang
- # XXX If someone wants one of the localizable packages, add language
- # suffix
- # Attention though: This needs to correspond to the packages in
- # packages.py
- if package == "tor-browser-bundle" or package == "tor-im-browser-bundle":
- # "tor-browser-bundle" => "tor-browser-bundle_de"
- package = package + "_" + lang
self.package = package
self.splitsend = split
self.signature = signature