[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

[tor-commits] [gettor/master] Update readme



commit 744934523a64e312ce42eaaa31ef936beeae5992
Author: hiro <hiro@xxxxxxxxxxxxxx>
Date:   Thu Oct 31 15:48:05 2019 +0100

    Update readme
    Breakdown strings for translations
    Delete unused scripts
---
 README.md                              | 159 ++++-----------
 dev_scripts/blacklist.py               | 111 -----------
 dev_scripts/core_demo.py               |  19 --
 dev_scripts/create_api_mirror.py       | 305 ----------------------------
 dev_scripts/create_gh_mirrors.py       | 207 -------------------
 dev_scripts/distribution_methods.txt   |  19 --
 dev_scripts/get_mirrors.py             | 353 ---------------------------------
 dev_scripts/process_chat.py            |  22 --
 dev_scripts/process_email.py           |  41 ----
 dev_scripts/process_http.py            |  14 --
 dev_scripts/process_tweets.py          |  35 ----
 dev_scripts/report.py                  | 129 ------------
 dev_scripts/sample-htaccess-api-mirror |  21 --
 dev_scripts/stats.py                   | 123 ------------
 gettor/main.py                         |   4 +-
 gettor/services/email/sendmail.py      |  15 +-
 gettor/services/twitter/twitterdm.py   |  14 +-
 gettor/utils/options.py                |   5 +-
 share/locale/en.json                   |  17 +-
 share/locale/es.json                   |  19 +-
 share/locale/pt.json                   |  17 +-
 tests/test_email_service.py            |   2 +-
 tests/test_locales.py                  |   2 +-
 tests/test_twitter.py                  |   2 +-
 24 files changed, 102 insertions(+), 1553 deletions(-)

diff --git a/README.md b/README.md
index f7e130d..a3df81b 100644
--- a/README.md
+++ b/README.md
@@ -32,144 +32,55 @@ Here is a list of the main goals the new GetTor should accomplish:
 
 Installing GetTor
 =================
-```sh
-WORKON_HOME=${HOME}/venv
-export WORKON_HOME
-mkdir -p $WORKON_HOME
-source $(which virtualenvwrapper.sh)
-git clone https://git.torproject.org/gettor.git && cd gettor
-mkvirtualenv -a $PWD -r requirements.txt --unzip-setuptools --setuptools gettor
-```
-
-From now on, to use GetTor's virtualenv, just do ``$ workon gettor``
-(after sourcing virtualenvwrapper.sh, as before). To exit the virtualenv
-without exiting the shell, do ``$ deactivate``.
-
-```sh
-export PYTHONPATH=$PYTHONPATH:${VIRTUAL_ENV}/lib/python/site-packages
-```
-
-```sh
-$ ./scripts/create_db
-$ ./bin/gettor_service start
-```
-
-Running tests
-=================
-
-GetTor includes PyTest unit tests. To run the tests, first install some dependencies:
-
-```sh
-pip3 install -r .test.requirements.txt
-```
-
-Then you can run `pytest` against the `tests/` directory.
-
-```sh
-pytest tests/
-```
-
-
-How does the new GetTor works?
-==============================
 
-Below are some specifications and core concepts on how the new GetTor works.
+To install gettor locally please install the following packages (on debian):
 
-*Links files*: Currently links are saved in files with the '.links' extension,
-using the ConfigParser format (RFC 882). A sample link file should look like
-this:
+python3-coverage
+python3-dkim
+python3-dns
+python3-internetarchive
+python3-openssl
+python3-pytest
+python3-requests-oauthlib
+python3-service-identity
+python3-twisted
+sqlite3
 
---- BEGIN FILE ---
+The following packages are needed to run a gettor instance:
 
-  [provider]
-  name = CoolCloudProvider
+internetarchive
+jq
+rclone
 
-  [key]
-  fingerprint = AAAA BBBB CCCC DDDD EEEE FFFF GGGG HHHH IIII JJJJ
+Specifically:
+internetarchive is needed to send Tor Browser files via command line to the internet archive.
+jq is a json parser that is used to find out about the new tor browser releases.
+Both internetarchive and jq are used in: scripts/update_files
 
-  [linux]
-  en = Package (64-bit): https://cool.cloud.link64
-	ASC signature (64-bit): https://cool.cloud.link64.asc
-	Package SHA256 checksum (64-bit): superhash64,
-	Package (32-bit): https://cool.cloud.link32
-	ASC signature (32-bit): https://cool.cloud.link32.asc
-	Package SHA256 checksum (32-bit): superhash32
+The following packages are instead needed to deploy gettor via ansible:
 
-  [windows]
-  ...
+ansible
+ansible-lint
 
-  [osx]
-  ...
+Gettor ansible playbooks live at: https://gitweb.torproject.org/admin/services/gettor.git/
 
---- END FILE ---
+Finally the following package is used store Tor Browser files via git and support large files:
+git-lfs
 
-You can also check providers/dropbox.links for a better example.
-WORKON_HOME=${HOME}/.virtualenvs
-         export WORKON_HOME
-         mkdir -p $WORKON_HOME
-         source $(which virtualenvwrapper.sh)
-*Core*: the heart of GetTor. Receives requests for links for a certain OS and
-language and respond accordingly. It also presents an easy way for scripts
-to create links file.
 
-*SMTP*: Receives requests via email, process them, contact the core module if
-necessary and respond to the user in the specified language. People can send
-blank or dummy emails to it to receive a help message describing how to ask
-for links. Email forwarding is used to redirect the emails to GetTor.
 
-*XMPP*: Same as above, but via XMPP (account needed). It has been tested with
-dukgo.com, jabber.ccc.de, riseup.net. It doesn't seem to be able to interact
-with gtalk users.
+Once gettor is installed you can run it with:
 
-*Twitter*: Receive requests via Twitter direct messages, contact the core module
-if necessary and respond to the user in the specified language. Unfinished.
-
-*DB*: Store anonymous info about the people that interact with GetTor in order
-to keep count of the number of requests per person and avoid malicious users
-that try to collapse the service. It also keeps count of how many requests
-GetTor has received during its lifetime. A lot of other data was being saved
-in the original gsoc project, but it was changed to save the minimum.
-
-*Blacklist*: Provide a mechanism to avoid flood and deny interaction to
-malicious users.
-
-*Providers scripts*: every supported provider should have a script to
-automatically upload packages to 'the cloud' and create the corresponding
-links files. The script should consider the following steps:
-
- * Upload the packages.
- * Get the sha256 checksum of the files uploaded.
- * Get the PGP key fingerprint that signed the files.
- * Check for .asc file for every package uploaded.
- * Put all together in a '.link' file (using the core module).
-
-
-What is the current status of the new GetTor?
-=============================================
-
-Deployed and working.
-
-
-How can I help?
-================
-
-If you have ideas to improve GetTor and/or add new providers, please tell us!
-I'm currently the lead developer on this, so if you have any comments/doubts/
-ideas you can send me an e-mail to ilv _at_ riseup _dot_ net or ping me (ilv),
-or sukhe or mrphs at #tor-dev in the OFTC IRC network. For openning tickets you
-should use the trac[0] and select the GetTor component. Some neat ideas we
-could use are the following:
+```
+$ ./bin/gettor_service start
+```
 
- * Report bugs!
- * Create script for new providers, namely: Google Drive, Github. Check
-providers.txt
- * Create a new module for distributing links. Check distribution_methods.txt
- * Finish the Twitter module.
- * Propose code/behaviour improvements.
- * Update the specs.
+Running tests
+=================
 
+GetTor includes PyTest unit tests. To run the tests, first install some dependencies:
 
-References
-===========
 
-[0] https://trac.torproject.org/projects/tor/query?status=accepted&status=assigned&status=needs_information&status=needs_review&status=needs_revision&status=new&status=reopened&component=GetTor&col=id&col=summary&col=component&col=status&col=type&col=priority&col=milestone&order=priority
+```
+$ pytest-3 tests/
+```
diff --git a/dev_scripts/blacklist.py b/dev_scripts/blacklist.py
deleted file mode 100644
index 1aefe1a..0000000
--- a/dev_scripts/blacklist.py
+++ /dev/null
@@ -1,111 +0,0 @@
-#!/usr/bin/python
-# -*- coding: utf-8 -*-
-#
-# This file is part of GetTor, a Tor Browser distribution system.
-#
-# :authors: Israel Leiva <ilv@xxxxxxxxxx>
-#           see also AUTHORS file
-#
-# :copyright:   (c) 2008-2014, The Tor Project, Inc.
-#               (c) 2014, Israel Leiva
-#
-# :license: This is Free Software. See LICENSE for license information.
-
-import sys
-import time
-import sqlite3
-import argparse
-
-
-def main():
-    """Script for managing blacklisting of users.
-
-    See argparse usage for more details.
-
-    """
-    parser = argparse.ArgumentParser(description='Utility for GetTor'
-                                     ' blacklisting')
-    parser.add_argument('database', metavar='database.db', type=str,
-                        help='the database file')
-    parser.add_argument('-u', '--user', default=None,
-                        help='filter by user hash')
-    parser.add_argument('-s', '--service', default=None,
-                        help='filter by service')
-    parser.add_argument('-b', '--blocked', default=None,
-                        help='filter by blocked users')
-    parser.add_argument('-a', '--add', default=None, nargs=3,
-                        metavar=('USER', 'SERVICE', 'BLOCKED'),
-                        help='add user')
-    parser.add_argument('-c', '--clean', default=None, const='c', nargs='?',
-                        metavar='user hash',
-                        help='clean table (delete expired blacklistings)')
-    parser.add_argument('-r', '--requests', default=None,
-                        help='number of requests; everyone with number of'
-                        ' requests greather than this will be cleaned up')
-
-    args = parser.parse_args()
-    query = ''
-    con = sqlite3.connect(args.database)
-
-    if args.add:
-        # add new entry, useful for adding users permanently blocked
-        query = "INSERT INTO users VALUES('%s', '%s', 1, %s, %s)"\
-                % (args.add[0], args.add[1], args.add[2], time.time())
-        with con:
-            cur = con.cursor()
-            cur.execute(query)
-        print "Query execute successfully"
-    elif args.clean:
-        if args.clean == 'c':
-            if args.requests:
-                # delete by number of times
-                query = "DELETE FROM users WHERE times > %s" % args.requests
-                with con:
-                    cur = con.cursor()
-                    cur.execute(query)
-                print "Query executed successfully."
-            else:
-                sys.exit("Number of requests missing. See --help.")
-        else:
-            # delete by id
-            query = "DELETE FROM users WHERE id='%s'" % args.clean
-            with con:
-                cur = con.cursor()
-                cur.execute(query)
-            print "Query execute succcessfully."
-    else:
-        query = "SELECT * FROM users"
-        has_where = False
-        # filter
-        if args.service:
-            query = "%s %s" % (query, "WHERE service='%s'" % args.service)
-            has_where = True
-        if args.user:
-            if has_where:
-                query = "%s %s" % (query, "AND id='%s'" % args.user)
-            else:
-                query = "%s %s" % (query, "WHERE id='%s'" % args.user)
-                has_where = True
-        if args.blocked:
-            if has_where:
-                query = "%s %s" % (query, "AND blocked=%s" % args.blocked)
-                has_where = True
-            else:
-                query = "%s %s" % (query, "WHERE blocked=%s" % args.blocked)
-
-        with con:
-            cur = con.cursor()
-            cur.execute(query)
-            rows = cur.fetchall()
-            # show it nice
-            print "\nNumber of results: %s\n" % len(rows)
-            cns = [cn[0] for cn in cur.description]
-            print "%-70s %-10s %-10s %-10s %-s" % (cns[0], cns[1], cns[2],
-                                                   cns[3], cns[4])
-
-            for row in rows:
-                print "%-70s %-10s %-10s %-10s %s" % (row[0], row[1], row[2],
-                                                      row[3], row[4])
-
-if __name__ == "__main__":
-    main()
diff --git a/dev_scripts/core_demo.py b/dev_scripts/core_demo.py
deleted file mode 100644
index 88f725d..0000000
--- a/dev_scripts/core_demo.py
+++ /dev/null
@@ -1,19 +0,0 @@
-#!/usr/bin/python
-#
-# Dummy script to test GetTore's Core module
-#
-
-import gettor.core
-
-try:
-    core = gettor.core.Core()
-    links = core.get_links('dummy service', 'linux', 'en')
-    print links
-except gettor.core.ConfigError as e:
-    print "Misconfiguration: " + str(e)
-except gettor.core.UnsupportedOSError as e:
-    print "Unsupported OS: " + str(e)
-except gettor.core.UnsupportedLocaleError as e:
-    print "Unsupported Locale: " + str(e)
-except gettor.core.InternalError as e:
-    print "Internal error: " + str(e)
diff --git a/dev_scripts/create_api_mirror.py b/dev_scripts/create_api_mirror.py
deleted file mode 100644
index e80ec4a..0000000
--- a/dev_scripts/create_api_mirror.py
+++ /dev/null
@@ -1,305 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-# This file is part of GetTor.
-#
-# :authors: Israel Leiva <ilv@xxxxxxxxxxxxxx>
-#           see also AUTHORS file
-#
-# :copyright:   (c) 2008-2015, The Tor Project, Inc.
-#               (c) 2015, Israel Leiva
-#
-# :license: This is Free Software. See LICENSE for license information.
-
-import os
-import re
-import json
-import codecs
-import urllib2
-import ConfigParser
-
-from time import gmtime, strftime
-
-
-"""Script to build a static mirror of GetTor RESTful API"""
-
-# currently supported locales for Tor Browser
-LC = ['ar', 'de', 'en-US', 'es-ES', 'fa', 'fr', 'it', 'ko', 'nl', 'pl',
-      'pt-PT', 'ru', 'tr', 'vi', 'zh-CN']
-
-# https://gitweb.tpo/tor-browser-spec.git/tree/processes/VersionNumbers
-# does not say anything about operating systems, so it's possible the
-# notation might change in the future. We should always use the same three
-# strings though: linux, windows, osx.
-OS = {
-    'Linux': 'linux',
-    'Windows': 'windows',
-    'MacOS': 'osx'
-}
-
-# based on
-# https://gitweb.tpo.org/tor-browser-spec.git/tree/processes/VersionNumbers
-# except for the first one, which is based on current RecommendedTBBVersions
-RE = {
-    'os': '(.*)-(\w+)',
-    'alpha': '\d\.\d(\.\d)*a\d+',
-    'beta': '\d\.\d(\.\d)*b\d+',
-    'stable': '\d\.\d(\.\d)*'
-}
-
-# strings to build names of packages depending on OS.
-PKG = {
-    'windows': 'torbrowser-install-%s_%s.exe',
-    'linux': 'tor-browser-linux%s-%s_%s.tar.xz',
-    'osx': 'TorBrowser-%s-osx64_%s.dmg'
-}
-
-# bin and asc are used to build the download links for each version, os and lc
-URL = {
-    'version': 'https://gettor.torproject.org/api/latest.json',
-    'mirrors': 'https://gettor.torproject.org/api/mirrors.json',
-    'providers': 'https://gettor.torproject.org/api/providers.json'
-}
-
-GETTOR_URL = 'http://gettor.torproject.org'
-SERVER_URL = 'https://yourprivateserver.com'
-API_TREE = '/home/ilv/Proyectos/tor/static_api/'
-
-class ConfigError(Exception):
-    pass
-
-
-class InternalError(Exception):
-    pass
-
-
-class APIMirror(object):
-    """ Provide useful resources via RESTful API. """
-    def __init__(self, cfg=None):
-        self.url = SERVER_URL
-        self.tree = API_TREE
-
-    def _is_json(self, my_json):
-        """ Check if json generated is valid.
-
-        :param: my_json (string) data to ve verified.
-
-        :return: (bool) true if data is json-valid, false otherwise.
-
-        """
-        try:
-            json_object = json.loads(my_json)
-        except ValueError, e:
-            return False
-        return True
-    
-    def _write_json(self, path, content):
-        """
-        """
-        try:
-            with codecs.open(
-                path,
-                "w",
-                "utf-8"
-            ) as jsonfile:
-                # Make pretty json
-                json.dump(
-                    content,
-                    jsonfile,
-                    sort_keys=True,
-                    indent=4,
-                    separators=(',', ': '),
-                    encoding="utf-8",
-                )
-        except IOError as e:
-            #logging.error("Couldn't write json: %s" % str(e))
-            print "Error building %s: %s" % (path, str(e))
-        print "%s built" % path
-
-    def _get_provider_name(self, p):
-        """ Return simplified version of provider's name.
-
-        :param: p (string) provider's name.
-
-        :return: (string) provider's name in lowercase and without spaces.
-
-        """
-        p = p.replace(' ', '-')
-        return p.lower()
-
-
-    def _load_latest_version(self):
-        """ Load latest version data from GetTor API. """
-        response = urllib2.urlopen(URL['version'])
-        json_response = json.load(response)
-        self.lv = json_response
-
-    def _load_links(self):
-        """ Load links and providers data. """
-
-        response = urllib2.urlopen(URL['providers'])
-        json_response = json.load(response)
-        
-        links = {}
-        
-        for provider in json_response:
-            if provider != 'updated_at':
-                provider_url = json_response[provider]
-            
-                provider_response = urllib2.urlopen("%s.json" % provider_url)
-                provider_content = json.load(provider_response)
-                
-                json_response[provider] = provider_url.replace(
-                    GETTOR_URL,
-                    self.url
-                )
-
-                
-                links[provider] = provider_content
-
-        self.providers = json_response
-        self.links = links
-
-    def _load_mirrors(self):
-        """ Load mirrors data from GetTor API. """
-        response = urllib2.urlopen(URL['mirrors'])
-        json_response = json.load(response)
-        self.mirrors = json_response
-
-    def _load_resources(self):
-        """ Load available resources data. """
-
-        self.resources = {
-            'providers': '%s/providers' % self.url,
-            'mirrors': '%s/mirrors' % self.url,
-            'latest_version': '%s/latest' % self.url,
-            'updated_at': strftime("%Y-%m-%d %H:%M:%S", gmtime())
-        }
-
-    def load_data(self):
-        """ Load all data."""
-        self._load_links()
-        self._load_mirrors()
-        self._load_resources()
-        self._load_latest_version()
-
-    def build(self):
-        """ Build RESTful API. """
-        
-        print "Building API mirror"
-        
-        # resources
-        self._write_json(
-            os.path.join(self.tree, 'api'),
-            self.resources
-        )
-
-        api_path = os.path.join(self.tree, 'api-content')
-        if not os.path.isdir(api_path):
-            os.mkdir(api_path)
-        
-        # providers
-        self._write_json(
-            os.path.join(api_path, 'providers'),
-            self.providers
-        )
-
-        providers_path = os.path.join(api_path, 'providers-content')
-        if not os.path.isdir(providers_path):
-            os.mkdir(providers_path)
-
-        for provider in self.links:
-            if provider == 'updated_at':
-                continue
-
-            self._write_json(
-                os.path.join(providers_path, provider),
-                self.links[provider]
-            )
-
-            provider_path = os.path.join(
-                providers_path,
-                "%s-content" % provider
-            )
-
-            if not os.path.isdir(provider_path):
-                os.mkdir(provider_path)
-            
-            for osys in self.links[provider]:
-                self._write_json(
-                    os.path.join(provider_path, osys),
-                    self.links[provider][osys]
-                )
-
-                provider_os_path = os.path.join(
-                    provider_path, "%s-content" % osys
-                )            
-
-                if not os.path.isdir(provider_os_path):
-                    os.mkdir(provider_os_path)
-                
-                for lc in self.links[provider][osys]:
-                    self._write_json(
-                        os.path.join(provider_os_path, lc),
-                        self.links[provider][osys][lc]
-                    )
-
-        # latest version
-        self._write_json(
-            os.path.join(api_path, 'latest'),
-            self.lv
-        )
-        
-        lv_path = os.path.join(api_path, 'latest-content')
-        if not os.path.isdir(lv_path):
-            os.mkdir(lv_path)
-
-        for release in self.lv:
-            if release == 'updated_at':
-                continue
-
-            self._write_json(
-                os.path.join(lv_path, release),
-                self.lv[release]
-            )
-            
-            release_path = os.path.join(
-                lv_path,
-                "%s-content" % release
-            )
-
-            if not os.path.isdir(release_path):
-                os.mkdir(release_path)
-            
-            for osys in self.lv[release]['downloads']:
-                self._write_json(
-                    os.path.join(release_path, osys),
-                    self.lv[release]['downloads'][osys]
-                )
-
-                release_os_path = os.path.join(
-                    release_path,
-                    "%s-content" % osys
-                )
-
-                if not os.path.isdir(release_os_path):
-                    os.mkdir(release_os_path)
-                
-                for lc in self.lv[release]['downloads'][osys]:
-                    self._write_json(
-                        os.path.join(release_os_path, lc),
-                        self.lv[release]['downloads'][osys][lc]
-                    )
-
-        # mirrors
-        self._write_json(
-            os.path.join(api_path, 'mirrors'),
-            self.mirrors
-        )
-
-def main():
-    api = APIMirror()
-    api.load_data()
-    api.build()
-
-if __name__ == '__main__':
-    main()
diff --git a/dev_scripts/create_gh_mirrors.py b/dev_scripts/create_gh_mirrors.py
deleted file mode 100644
index e555488..0000000
--- a/dev_scripts/create_gh_mirrors.py
+++ /dev/null
@@ -1,207 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-# This file is part of GetTor
-#
-# :authors: Israel Leiva <ilv@xxxxxxxxxxxxxx>
-#           see also AUTHORS file
-#
-# :license: This is Free Software. See LICENSE for license information.
-
-"""create_gh_mirrors -- Create landing page and readme for Github."""
-
-import os
-import ConfigParser
-
-import github3
-
-import gettor.core
-
-
-def create_readme(tpl_path, md_path, tb_version, links):
-    """Create README file with links stored in github.links.
-
-    :param: tpl_path (string) path to file used as template.
-    :param: md_path (string) path to file generated.
-    :param: tb_version (string) tor browser version.
-    :param: links (object) github links.
-
-    """
-    lcs = ['FA', 'ZH', 'TR', 'EN']
-
-    md_file = open(md_path, 'w')
-    with open(tpl_path, 'r') as tpl_file:
-        content_md = tpl_file.read()
-
-        for lc in lcs:
-            win_link = links.get('windows', lc.lower())
-            win_pkg, win_sig, win_sha = [e for e in win_link.split("$") if e]
-
-            osx_link = links.get('osx', lc.lower())
-            osx_pkg, osx_sig, osx_sha = [e for e in osx_link.split("$") if e]
-
-            linux_links = links.get('linux', lc.lower())
-            linux32_link, linux64_link = linux_links.split(',')
-            linux32_pkg, linux32_sig, linux32_sha = [
-                e for e in linux32_link.split("$") if e
-            ]
-            linux64_pkg, linux64_sig, linux64_sha = [
-                e for e in linux64_link.split("$") if e
-            ]
-
-            content_md = content_md.replace(
-                "%WINDOWS_{}%".format(lc), win_pkg
-            )
-            content_md = content_md.replace(
-                "%WINDOWS_{}_SIG%".format(lc), win_sig
-            )
-
-            content_md = content_md.replace(
-                "%OSX_{}%".format(lc), osx_pkg
-            )
-            content_md = content_md.replace(
-                "%OSX_{}_SIG%".format(lc), osx_sig
-            )
-
-            content_md = content_md.replace(
-                "%LINUX32_{}%".format(lc), linux32_pkg
-            )
-            content_md = content_md.replace(
-                "%LINUX32_{}_SIG%".format(lc), linux32_sig
-            )
-            content_md = content_md.replace(
-                "%LINUX64_{}%".format(lc), linux64_pkg
-            )
-            content_md = content_md.replace(
-                "%LINUX64_{}_SIG%".format(lc), linux64_sig
-            )
-
-        content_md = content_md.replace("%TB_VERSION%", tb_version)
-        md_file.write(content_md)
-
-    print "README generated with Tor Browser %s" % tb_version
-
-
-def create_landing_html(tpl_path, html_path, tb_version, links):
-    """Create README file with links stored in github.links.
-
-    :param: tpl_path (string) path to file used as template.
-    :param: html_path (string) path to file generated.
-    :param: tb_version (string) tor browser version.
-    :param: links (object) github links.
-
-    """
-    lcs = ['FA', 'ZH', 'TR', 'EN']
-
-    html_file = open(html_path, 'w')
-    with open(tpl_path, 'r') as tpl_file:
-        content_html = tpl_file.read().replace('\n', '')
-
-        for lc in lcs:
-            win_link = links.get('windows', lc.lower())
-            win_pkg, win_sig, win_sha = [e for e in win_link.split("$") if e]
-
-            osx_link = links.get('osx', lc.lower())
-            osx_pkg, osx_sig, osx_sha = [e for e in osx_link.split("$") if e]
-
-            linux_links = links.get('linux', lc.lower())
-            linux32_link, linux64_link = linux_links.split(',')
-            linux32_pkg, linux32_sig, linux32_sha = [
-                e for e in linux32_link.split("$") if e
-            ]
-            linux64_pkg, linux64_sig, linux64_sha = [
-                e for e in linux64_link.split("$") if e
-            ]
-            
-            content_html = content_html.replace(
-                "%WINDOWS_{}%".format(lc), win_pkg
-            )
-            """
-            content_html = content_html.replace(
-                "%WINDOWS_{}_SIG%".format(lc), win_sig
-            )
-            """
-            content_html = content_html.replace(
-                "%OSX_{}%".format(lc), osx_pkg
-            )
-            """
-            content_html = content_html.replace(
-                "%OSX_{}_SIG%".format(lc), osx_sig
-            )
-            """
-            content_html = content_html.replace(
-                "%LINUX32_{}%".format(lc), linux32_pkg
-            )
-            """
-            content_html = content_html.replace(
-                "%LINUX32_{}_SIG%".format(lc), linux32_sig
-            )
-            """
-            content_html = content_html.replace(
-                "%LINUX64_{}%".format(lc), linux64_pkg
-            )
-            """
-            content_html = content_html.replace(
-                "%LINUX64_{}_SIG%".format(lc), linux64_sig
-            )
-            """
-        
-        content_html = content_html.replace(
-            "%TB_VERSION%", tb_version
-        )
-        html_file.write(content_html)
-
-    print "HTML generated with Tor Browser %s" % tb_version
-
-
-def main():
-    """Generate HTML and md files and update it in Github."""
-    github_links = 'providers/github.links'
-    tb_version_path = 'latest_torbrowser.cfg'
-    md_path = 'upload/readme_gh.md'
-    html_path = 'upload/landing_gh.html'
-    md_tpl_path = 'upload/readme_gh.tpl'
-    html_tpl_path = 'upload/landing_gh.tpl'
-    github_access_token = ''
-
-    tb_version_config = ConfigParser.ConfigParser()
-    tb_version_config.read(tb_version_path)
-    tb_version = tb_version_config.get('version', 'current')
-
-    links = ConfigParser.ConfigParser()
-    links.read(github_links)
-
-    create_landing_html(html_tpl_path, html_path, tb_version, links)
-    create_readme(md_tpl_path, md_path, tb_version, links)
-
-    landing = open(html_path, 'r')
-    content_landing = landing.read().replace('\n', '')
-
-    readme = open(md_path, 'r')
-    content_readme = readme.read()
-
-    gh = github3.login(token=github_access_token)
-    repo_landing = gh.repository('thetorproject', 'gettor')
-    repo_readme = gh.repository('thetorproject', 'gettorbrowser')
-
-    file_landing_gh = repo_landing.file_contents('index.html', 'gh-pages')
-    file_readme_gh = repo_readme.file_contents('README.md')
-
-    data_landing = {
-        'message': 'Updating landing page.',
-        'content': content_landing,
-        'branch': 'gh-pages'
-    }
-
-    data_readme = {
-        'message': 'Updating README.',
-        'content': content_readme
-    }
-
-    file_landing_gh.update(**data_landing)
-    print "Landing page updated in gettor"
-
-    file_readme_gh.update(**data_readme)
-    print "README updated in gettorbrowser"
-
-if __name__ == "__main__":
-    main()
diff --git a/dev_scripts/distribution_methods.txt b/dev_scripts/distribution_methods.txt
deleted file mode 100644
index 8bbdd4b..0000000
--- a/dev_scripts/distribution_methods.txt
+++ /dev/null
@@ -1,19 +0,0 @@
-================================== CURRENT ====================================
-
-* SMTP: this one is ready and tested.
-
-* XMPP: this one is ready and tested. We'll probably deploy SMTP first, tough.
-
-================================== IN DEVELOPMENT =============================
-
-* Twitter: This one is half done. I forked a twitter bot from wfn's gsoc work
-on BridgeDB, but I got confused with some Twitter policies. I thought that the
-bot *had* to follow someone back to send him/her DMs, but it turns out that you
-just need to be followed, not follow back (thanks to mrphs for pointing out
-this).
-
-================================== (CRAZY) IDEAS ==============================
-
-* Duckduckgo: This is just a thought, I haven't done proper research. What if
-someone could ask for gettor links like this: "!gettor"?
-
diff --git a/dev_scripts/get_mirrors.py b/dev_scripts/get_mirrors.py
deleted file mode 100644
index afa4cc9..0000000
--- a/dev_scripts/get_mirrors.py
+++ /dev/null
@@ -1,353 +0,0 @@
-# -*- coding: utf-8 -*-
-#
-# This file is part of GetTor
-#
-# :authors: Israel Leiva <ilv@xxxxxxxxxxxxxx>
-#           see also AUTHORS file
-#
-# :license: This is Free Software. See LICENSE for license information.
-
-"""get_mirrors -- Download the list of tpo's mirrors for GetTor."""
-
-import os
-import json
-import codecs
-import logging
-import argparse
-
-
-from OpenSSL import SSL
-from OpenSSL import crypto
-
-from twisted.web import client
-from twisted.python import log
-from twisted.internet import ssl
-from twisted.internet import defer
-from twisted.internet import protocol
-from twisted.internet import reactor
-from twisted.internet.error import TimeoutError
-from twisted.internet.error import DNSLookupError
-from twisted.internet.error import ConnectionRefusedError
-
-from gettor import utils
-
-# Associate each protocol with its column name in tor-mirrors.csv
-PROTOS = {
-    'https': 'httpsWebsiteMirror',
-    'http': 'httpWebsiteMirror',
-    'rsync': 'rsyncWebsiteMirror',
-    'https-dist': 'httpsDistMirror',
-    'http-dist': 'httpDistMirror',
-    'rsync-dist': 'rsyncDistMirror',
-    'ftp': 'ftpWebsiteMirror',
-    'onion': 'hiddenServiceMirror',
-}
-
-# Tor Project's website certificate
-# $ openssl s_client -showcerts -connect tpo:443 < /dev/null > tpo.pem
-CERT_TPO = '/path/to/gettor/tpo.pem'
-
-
-# Taken from get-tor-exits (BridgeDB)
-class FileWriter(protocol.Protocol):
-    """Read a downloaded file incrementally and write to file."""
-    def __init__(self, finished, file):
-        """Create a FileWriter.
-
-        .. warning:: We currently only handle the first 2MB of a file. Files
-            over 2MB will be truncated prematurely. *note*: this should be
-            enough for the mirrors file.
-
-        :param finished: A :class:`~twisted.internet.defer.Deferred` which
-            will fire when another portion of the download is complete.
-        """
-        self.finished = finished
-        self.remaining = 1024 * 1024 * 2
-        self.fh = file
-
-    def dataReceived(self, bytes):
-        """Write a portion of the download with ``bytes`` size to disk."""
-        if self.remaining:
-            display = bytes[:self.remaining]
-            self.fh.write(display)
-            self.fh.flush()
-            self.remaining -= len(display)
-
-    def connectionLost(self, reason):
-        """Called when the download is complete."""
-        logging.info('Finished receiving mirrors list: %s'
-                     % reason.getErrorMessage())
-        self.finished.callback(None)
-
-
-# Based in tor2web.utils.ssl (Tor2web)
-class HTTPSVerifyingContextFactory(ssl.ClientContextFactory):
-    def __init__(self, cn):
-        self.cn = cn
-        #
-        # From https://docs.python.org/2/library/ssl.html#ssl-security
-        #
-        # "SSL versions 2 and 3 are considered insecure and are therefore
-        # dangerous to use. If you want maximum compatibility between clients
-        # and servers, it is recommended to use PROTOCOL_SSLv23 as the protocol
-        # version and then disable SSLv2 and SSLv3 explicitly"
-        #
-        self.method = SSL.SSLv23_METHOD
-
-    def getContext(self, hostname, port):
-        """Get this connection's OpenSSL context.
-
-        We disable SSLv2 and SSLv3. We also check the certificate received
-        is the one we expect (using the "common name").
-
-        """
-        ctx = self._contextFactory(self.method)
-        ctx.set_options(SSL.OP_NO_SSLv2)
-        ctx.set_options(SSL.OP_NO_SSLv3)
-        ctx.set_verify(
-            SSL.VERIFY_PEER | SSL.VERIFY_FAIL_IF_NO_PEER_CERT,
-            self.verifyCN
-        )
-
-        return ctx
-
-    def verifyCN(self, connection, x509, errno, depth, preverifyOK):
-        # DEBUG: print "%s == %s ?" % (self.cn, x509.get_subject().commonName)
-
-        # Somehow, if I don't set this to true, the verifyCN doesn't go
-        # down in the chain, I don't know if this is OK
-        verify = True
-        if depth == 0:
-            if self.cn == x509.get_subject().commonName:
-                verify = True
-            else:
-                verify = False
-        return verify
-
-
-# Based in get-tor-exits (BridgeDB)
-def handle(failure):
-    """Handle a **failure**."""
-    if failure.type == ConnectionRefusedError:
-        logging.error("Couldn't download file; connection was refused.")
-    elif failure.type == DNSLookupError:
-        logging.error("Couldn't download file; domain resolution failed.")
-    elif failure.type == TimeoutError:
-        logging.error("Couldn't download file; connection timed out.")
-    else:
-        logging.error("Couldn't download file.")
-
-    print "Couldn't download file. Check the log."
-    os._exit(-1)
-
-
-# Taken from get-tor-exits (BridgeDB)
-def writeToFile(response, filename):
-    """Write requested content to filename."""
-    finished = defer.Deferred()
-    response.deliverBody(FileWriter(finished, filename))
-    return finished
-
-
-def is_json(my_json):
-    """Check if json generated is valid."""
-    try:
-        json_object = json.loads(my_json)
-    except ValueError, e:
-        return False
-    return True
-
-
-def add_tpo_link(url):
-    """Add the download link for Tor Browser."""
-    uri = 'projects/torbrowser.html.en#downloads'
-    if url.endswith('/'):
-        return "%s%s" % (url, uri)
-    else:
-        return "%s/%s" % (url, uri)
-
-
-def add_entry(mirrors, columns, elements):
-    """Add entry to mirrors list."""
-    entry = {}
-    count = 0
-    for e in elements:
-        e = e.replace("\n", '')
-        entry[columns[count]] = e
-        count = count + 1
-
-    mirrors.append(entry)
-
-
-def add_mirror(file, entry, proto):
-    """Add mirror to mirrors list."""
-    # if proto requested is http(s), we add link to download section
-    if PROTOS[proto] == 'http' or PROTOS[proto] == 'https':
-        uri = add_tpo_link(entry[proto])
-    else:
-        uri = entry[proto]
-    
-    file.write(
-        "%s - by %s (%s)\n" % (
-            uri,
-            entry['orgName'],
-            entry['subRegion'],
-        )
-    )
-
-
-def main():
-    """Script to get the list of tpo's mirrors from tpo and adapt it to
-    be used by GetTor.
-
-    Usage: python2.7 get_mirrors.py [-h] [--proto protocol]
-
-    By default, the protocol is 'https'. Possible values of protcol are:
-
-    http, https, rsync, ftp, onion, http-dist, https-dist, rsync-dist.
-
-    """
-    parser = argparse.ArgumentParser(
-        description="Utility to download tpo's mirrors and make it usable\
-        by GetTor."
-    )
-
-    parser.add_argument(
-        '-p', '--proto',
-        default='https',
-        help='Protocol filter. Possible values: http, https, rsync, ftp, onion\
-        http-dist, https-dist, rsync-dist. Default to https.')
-
-    args = parser.parse_args()
-    p = args.proto
-
-    gettor_path = '/path/to/gettor/'
-    csv_path = os.path.join(gettor_path, 'tor-mirrors.csv')
-    json_path = os.path.join(gettor_path, 'tor-mirrors')
-    mirrors_list = os.path.join(gettor_path, 'mirrors-list.txt')
-
-    # Load tpo certificate and extract common name, we'll later compare this
-    # with the certificate sent by tpo to check we're really taltking to it
-    try:
-        data = open(CERT_TPO).read()
-        x509 = crypto.load_certificate(crypto.FILETYPE_PEM, data)
-        cn_tpo = x509.get_subject().commonName
-    except Exception as e:
-        logging.error("Error with certificate: %s" % str(e))
-        return
-
-    # While we wait the json of mirrors to be implemented in tpo, we need
-    # to download the csv file and transform it to json
-
-    # The code below is based in get-tor-exits script from BridgeDB and
-    # the tor2web.utils.ssl module from Tor2web
-    url = 'https://www.torproject.org/include/tor-mirrors.csv'
-
-    try:
-        fh = open(csv_path, 'w')
-    except IOError as e:
-        logging.error("Could not open %s" % csv_path)
-        return
-
-    logging.info("Requesting %s..." % url)
-
-    # If certificate don't match an exception will be raised
-    # this is my first experience with twisted, maybe I'll learn to handle
-    # this better some time in the future...
-    contextFactory = HTTPSVerifyingContextFactory(cn_tpo)
-    agent = client.Agent(reactor, contextFactory)
-    d = agent.request("GET", url)
-    d.addCallback(writeToFile, fh)
-    d.addErrback(handle)
-    d.addCallbacks(log.msg, log.err)
-
-    if not reactor.running:
-        d.addCallback(lambda ignored: reactor.stop())
-        reactor.run()
-
-    logging.info("File downloaded!")
-
-    # Now transform it to json -- I couldn't find out how to use a
-    # two-character delimiter with the csv package, so I decided to handle
-    # the csv data by hand. We are doing this until #16601 gets deployed.
-    # https://trac.torproject.org/projects/tor/ticket/16601
-
-    # Code below is based in update-mirrors-json.py script from tpo
-
-    # These are the names of each column e.g. adminContact
-    columns = []
-    # List of mirrors to be built
-    mirrors = []
-
-    logging.info("Transforming csv data into json...")
-    logging.info("Getting data from csv")
-    try:
-        with codecs.open(csv_path, "rb", "utf-8") as csvfile:
-            for line in csvfile:
-                elements = line.split(", ")
-                # first entry have the names of the columns
-                if not columns:
-                    columns = elements
-                else:
-                    add_entry(mirrors, columns, elements)
-    except IOError as e:
-        logging.error("Couldn't read csv file: %s" % str(e))
-        return
-
-    logging.info("Creating json")
-    if is_json(json.dumps(mirrors)):
-        try:
-            with codecs.open(json_path, "w", "utf-8") as jsonfile:
-                # Make pretty json
-                json.dump(
-                    mirrors,
-                    jsonfile,
-                    sort_keys=True,
-                    indent=4,
-                    separators=(',', ': '),
-                    encoding="utf-8",
-                )
-        except IOError as e:
-            logging.error("Couldn't write json: %s" % str(e))
-            return
-    else:
-        logging.error("Invalid json file")
-        return
-
-    # Now make the mirrors list to be used by GetTor
-    logging.info("Reading json")
-    try:
-        mirrors_json = codecs.open(json_path, "rb", "utf-8")
-        mirrors = json.load(mirrors_json)
-    except IOError as e:
-        logging.error("Couldn't open %s" % json_path)
-        return
-
-    logging.info("Creating new list with protocol: %s" % p)
-    try:
-        list = codecs.open(mirrors_list, "w", "utf-8")
-        for entry in mirrors:
-            if args.proto is not 'all':
-                for e in entry:
-                    if e == PROTOS[p] and entry[PROTOS[p]]:
-                        add_mirror(list, entry, PROTOS[p])
-            else:
-                for k, v in PROTOS:
-                    if entry[v]:
-                        add_mirror(list, entry, v)
-        logging.info("List created: %s" % mirrors_list)
-    except IOError as e:
-        logging.error("Couldn't open %s" % mirrors_list)
-        return
-
-if __name__ == "__main__":
-    logging_format = utils.get_logging_format()
-    logging.basicConfig(
-        filename='/path/to/gettor/log/get-mirrors.log',
-        format=logging_format,
-        datefmt="%Y-%m-%d %H:%M:%S",
-        level=logging.INFO
-    )
-    logging.info("Started")
-    main()
-    logging.info("Finished")
diff --git a/dev_scripts/process_chat.py b/dev_scripts/process_chat.py
deleted file mode 100644
index 6ad7b59..0000000
--- a/dev_scripts/process_chat.py
+++ /dev/null
@@ -1,22 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-import sys
-import logging
-
-import gettor.xmpp
-
-def main():
-    try:
-        bot = gettor.xmpp.XMPP()
-        bot.start_bot()
-    except gettor.xmpp.ConfigError as e:
-        print "Configuration error: %s" % str(e)
-    except gettor.xmpp.InternalError as e:
-        print "Core module not working: %s" % str(e)
-    except Exception as e:
-        # in case something unexpected happens
-        print "Unexpected error: %s" % str(e)
-
-if __name__ == '__main__':
-    main()
diff --git a/dev_scripts/process_email.py b/dev_scripts/process_email.py
deleted file mode 100644
index 504b5dd..0000000
--- a/dev_scripts/process_email.py
+++ /dev/null
@@ -1,41 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-import sys
-import logging
-
-import gettor.smtp
-
-def main():
-    logging_level = 'INFO'
-    logging_file = '/path/to/gettor/log/process_email.log'
-    logging_format = '[%(levelname)s] %(asctime)s - %(message)s'
-    date_format = "%Y-%m-%d" # %H:%M:%S
-
-    logging.basicConfig(
-        format=logging_format,
-        datefmt=date_format,
-        filename = logging_file,
-        level = logging_level
-    )
-
-    logging.debug("New email received")
-    logging.debug("Creating new SMTP object")
-    
-    try:
-        service = gettor.smtp.SMTP('/path/to/gettor/smtp.cfg')
-        incoming = sys.stdin.read()
-        service.process_email(incoming)
-        logging.debug("Email processed sucessfully")
-    except gettor.smtp.ConfigError as e:
-        logging.error("Configuration error: %s" % str(e))        
-    except gettor.smtp.SendEmailError as e:
-        logging.error("SMTP not working: %s" % str(e))
-    except gettor.smtp.InternalError as e:
-        logging.error("Core module not working: %s" % str(e))
-    except Exception as e:
-        # in case something unexpected happens
-        logging.critical("Unexpected error: %s" % str(e))
-
-if __name__ == '__main__':
-    main()
diff --git a/dev_scripts/process_http.py b/dev_scripts/process_http.py
deleted file mode 100644
index ec20d5f..0000000
--- a/dev_scripts/process_http.py
+++ /dev/null
@@ -1,14 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-import gettor.http2
-
-
-def main():
-    api = gettor.http2.HTTP('http.cfg')
-    api.load_data()
-    # api.run()
-    api.build()
-
-if __name__ == '__main__':
-    main()
diff --git a/dev_scripts/process_tweets.py b/dev_scripts/process_tweets.py
deleted file mode 100644
index 59c2f48..0000000
--- a/dev_scripts/process_tweets.py
+++ /dev/null
@@ -1,35 +0,0 @@
-#!/usr/bin/env python
-# -*- coding: utf-8 -*-
-
-import sys
-import logging
-
-import gettor.twitter
-
-def main():
-    logging_level = 'DEBUG'
-    logging_file = '/home/ilv/Proyectos/tor/gettor/log/process_tweets.log'
-    logging_format = '[%(levelname)s] %(asctime)s - %(message)s'
-    date_format = "%Y-%m-%d" # %H:%M:%S
-
-    logging.basicConfig(
-        format=logging_format,
-        datefmt=date_format,
-        filename = logging_file,
-        level = logging_level
-    )
-
-    logging.debug("Starting bot")
-    try:
-        bot = gettor.twitter.TwitterBot()
-        bot.start()
-    except gettor.twitter.ConfigError as e:
-        logging.error("Configuration error: %s" % str(e))
-    except gettor.twitter.InternalError as e:
-        logging.error("Core module not working: %s" % str(e))
-    except Exception as e:
-        # in case something unexpected happens
-        logging.error("Unexpected error: %s" % str(e))
-
-if __name__ == '__main__':
-    main()
diff --git a/dev_scripts/report.py b/dev_scripts/report.py
deleted file mode 100644
index b75d7c8..0000000
--- a/dev_scripts/report.py
+++ /dev/null
@@ -1,129 +0,0 @@
-#!/usr/bin/env python
-
-# This file is part of GetTor, a Tor Browser distribution system.
-
-# :authors: Sukhbir Singh <sukhbir@xxxxxxxxxxxxxx>
-#           see also AUTHORS file
-#
-# :copyright:   (c) 2008-2015, The Tor Project, Inc.
-
-# To read the log files and populate the db, run the script with `--logs' once
-# every day and then clear the log files. To generate the report, run it with
-# `--report', once (or twice?) a month.
-
-# TODO: Logging? argparse?
-
-import os
-import sys
-import ConfigParser
-from datetime import datetime as dt
-
-import sqlite3
-
-CORE_CFG = "core.cfg"
-
-CHANNELS = ["smtp", "xmpp", "twitter"]
-LOG_FILES = [channel + ".log" for channel in CHANNELS]
-
-DB = "gettor2.db"
-
-OUTPUT = "report.log"
-
-REPORT = """@ GetTor Report for {}
-
-We received a total of {} requests in {}, with a peak of {} requests on {}.
-
-[*] Request
-{}
-
-[*] OS
-{}
-
-[*] Language
-{}
-
-[*] Channel
-{}
-
-"""
-
-
-class Report(object):
-    def __init__(self):
-        self.config = ConfigParser.ConfigParser()
-        self.config.read(CORE_CFG)
-
-        self.log_dir = self.config.get("log", "dir")
-        # This is defined in the sample log file.
-        self.log_format = ["date", "request", "os", "locale"]
-
-        self.conn = sqlite3.connect(DB)
-
-    def db_write(self):
-        self.cursor = self.conn.cursor()
-        for each in self.logs:
-            self.cursor.execute("""INSERT INTO requests
-                                (date, request, os, locale, channel) VALUES
-                                (:date, :request, :os, :locale, :channel)""",
-                                each)
-            self.conn.commit()
-        self.conn.close()
-
-    def get_logs(self):
-        self.logs = []
-        for each in LOG_FILES:
-            with open(os.path.join(self.log_dir, each)) as f:
-                for line in f:
-                    if line.startswith("[INFO] "):
-                        # The "7" here is for the "[INFO] " text.
-                        logs = [log.strip() for log in line[7:].split(";")]
-                        log_data = {key: value for key, value in
-                                    zip(self.log_format, logs)}
-                        # We also need the channel.
-                        log_data["channel"] = each.split(".")[0]
-                        self.logs.append(log_data)
-        self.db_write()
-        self.conn.close()
-
-    def generate_report(self):
-        self.cursor = self.conn.cursor()
-        self.requests = []
-
-        self.cursor.execute("SELECT COUNT(*) FROM requests")
-        self.requests.append(self.cursor.fetchone())
-
-        self.cursor.execute("""SELECT date, COUNT(*) FROM requests
-GROUP BY date
-ORDER BY COUNT(date)
-DESC""")
-        self.requests.append(self.cursor.fetchone())
-
-        self.columns = ["request", "os", "locale", "channel"]
-        for each in self.columns:
-            self.cursor.execute("""SELECT {0}, COUNT(*) FROM requests
-GROUP BY {0}
-ORDER BY COUNT({0})
-DESC""".format(each))
-            result = "\n".join(["{0:>16}: {1}".format(each[0], each[1])
-                                for each in self.cursor.fetchall()
-                                if not each[0] == "none"])
-            self.requests.append(result)
-
-        with open(OUTPUT, "a") as f:
-            f.write(REPORT.format(dt.now().strftime("%B %Y"),
-                                  self.requests[0][0],
-                                  dt.now().strftime("%B"),
-                                  self.requests[1][1],
-                                  dt.strptime(self.requests[1][0], "%Y-%m-%d")
-                                  .strftime("%B %-d"),
-                                  *self.requests[2:]))
-        self.conn.close()
-
-if __name__ == "__main__":
-    try:
-        if sys.argv[1] == "--logs":
-            Report().get_logs()
-        if sys.argv[1] == "--report":
-            Report().generate_report()
-    except IndexError:
-        sys.exit("You need either `--logs' or `--report.'")
diff --git a/dev_scripts/sample-htaccess-api-mirror b/dev_scripts/sample-htaccess-api-mirror
deleted file mode 100644
index 41594db..0000000
--- a/dev_scripts/sample-htaccess-api-mirror
+++ /dev/null
@@ -1,21 +0,0 @@
-Options +FollowSymLinks
-RewriteEngine On
-
-# Operating System and Locale filters for providers
-RewriteCond %{QUERY_STRING} os=([a-z]+)&lc=([a-z]+)
-RewriteRule ^api/providers/([a-z\-]+)$ api-content/providers-content/$1-content/%1-content/%2.json
-RewriteCond %{QUERY_STRING} os=([a-z]+)
-RewriteRule ^api/providers/([a-z\-]+)$ api-content/providers-content/$1-content/%1
-
-# Operating System and Locale filters for TPO downloads
-RewriteCond %{QUERY_STRING} os=([a-z]+)&lc=([a-zA-Z\-]+)
-RewriteRule ^api/latest/([a-z]+)$ api-content/latest-content/$1-content/%1-content/%2.json
-RewriteCond %{QUERY_STRING} os=([a-z]+)
-RewriteRule ^api/latest/([a-z]+)$ api-content/latest-content/$1-content/%1
-
-# General rules
-RewriteRule ^api/providers/([a-z\-]+)$ api-content/providers-content/$1 [NC]
-RewriteRule ^api/latest/([a-z]+)$ api-content/latest-content/$1 [NC]
-RewriteRule ^api/([a-z]+)$ api-content/$1 [NC]
-
-ErrorDocument 404 404-file
diff --git a/dev_scripts/stats.py b/dev_scripts/stats.py
deleted file mode 100644
index 7fca2af..0000000
--- a/dev_scripts/stats.py
+++ /dev/null
@@ -1,123 +0,0 @@
-#!/usr/bin/python
-# -*- coding: utf-8 -*-
-#
-# This file is part of GetTor, a Tor Browser distribution system.
-#
-# :authors: Israel Leiva <ilv@xxxxxxxxxx>
-#           see also AUTHORS file
-#
-# :copyright:   (c) 2008-2014, The Tor Project, Inc.
-#               (c) 2014, Israel Leiva
-#
-# :license: This is Free Software. See LICENSE for license information.
-
-
-import sqlite3
-import argparse
-
-
-def main():
-    """Script for showing stats.
-
-    See argparse usage for more details.
-
-    """
-    parser = argparse.ArgumentParser(description='Utility for GetTor stats')
-    parser.add_argument('database', metavar='database.db', type=str,
-                        help='the database file')
-    parser.add_argument('-s', '--service', default=None,
-                        help='filter by service')
-    parser.add_argument('-t', '--type', default=None,
-                        help='filter by type of request')
-    parser.add_argument('-o', '--os', default=None,
-                        help='filter by OS')
-    parser.add_argument('-l', '--lc', default=None,
-                        help='filter by locale')
-    parser.add_argument('-p', '--pt', default=None,
-                        help='filter by PT requests')
-    parser.add_argument('-y', '--year', default=None,
-                        help='filter by year')
-    parser.add_argument('-m', '--month', default=None,
-                        help='filter by month')
-    parser.add_argument('-d', '--day', default=None,
-                        help='filter by day')
-    parser.add_argument('-u', '--status', default=None,
-                        help='filter by status of the request')
-
-    args = parser.parse_args()
-    query = 'SELECT * FROM requests'
-    has_where = False
-
-    # we build the query piece by piece
-    if args.service:
-        query = "%s %s" % (query, "WHERE service = '%s'" % args.service)
-        has_where = True
-    if args.type:
-        if has_where:
-            query = "%s %s" % (query, "AND type = '%s'" % args.type)
-        else:
-            query = "%s %s" % (query, "WHERE type = '%s'" % args.type)
-            has_where = True
-    if args.os:
-        if has_where:
-            query = "%s %s" % (query, "AND os = '%s'" % args.os)
-        else:
-            query = "%s %s" % (query, "WHERE os = '%s'" % args.os)
-            has_where = True
-    if args.lc:
-        if has_where:
-            query = "%s %s" % (query, "AND lc = '%s'" % args.lc)
-        else:
-            query = "%s %s" % (query, "WHERE lc = '%s'" % args.lc)
-            has_where = True
-    if args.pt:
-        if has_where:
-            query = "%s %s" % (query, "AND pt = %s" % args.pt)
-        else:
-            query = "%s %s" % (query, "WHERE pt = %s" % args.pt)
-            has_where = True
-    if args.year:
-        if has_where:
-            query = "%s %s" % (query, "AND year = %s" % args.year)
-        else:
-            query = "%s %s" % (query, "WHERE year = %s" % args.year)
-            has_where = True
-    if args.month:
-        if has_where:
-            query = "%s %s" % (query, "AND month = %s" % args.month)
-        else:
-            query = "%s %s" % (query, "WHERE month = %s" % args.month)
-            has_where = True
-    if args.day:
-        if has_where:
-            query = "%s %s" % (query, "AND day = %s" % args.day)
-        else:
-            query = "%s %s" % (query, "WHERE day = %s" % args.day)
-            has_where = True
-    if args.status:
-        if has_where:
-            query = "%s %s" % (query, "AND status = '%s'" % args.status)
-        else:
-            query = "%s %s" % (query, "WHERE status = '%s'" % args.status)
-            has_where = True
-
-    con = sqlite3.connect(args.database)
-
-    with con:
-        cur = con.cursor()
-        cur.execute(query)
-        rows = cur.fetchall()
-        # show it nice
-        print "\nNumber of results: %s\n" % len(rows)
-        cns = [cn[0] for cn in cur.description]
-        print "%-10s %-10s %-10s %-10s %-10s %-10s %-10s %-10s"\
-              " %-15s %s"\ % (cns[0], cns[1], cns[2], cns[3], cns[4], cns[5],
-                              cns[6], cns[7], cns[8], cns[9])
-
-        for row in rows:
-            print "%-10s %-10s %-10s %-10s %-10s %-10s %-10s %-10s"\
-                  " %-15s %s" % (row[0], row[1], row[2], row[3], row[4],
-                                 row[5], row[6], row[7], row[8], row[9])
-
-if __name__ == "__main__":
-    main()
diff --git a/gettor/main.py b/gettor/main.py
index 9cd18b7..d59f66c 100644
--- a/gettor/main.py
+++ b/gettor/main.py
@@ -25,7 +25,9 @@ def run(gettor, app):
     """
     This is GetTor's main entry point and main runtime loop.
     """
-    settings = options.parse_settings()
+    config = "/home/gettor/gettor/gettor.conf.json"
+
+    settings = options.parse_settings("en", config)
 
     sendmail = Sendmail(settings)
     twitterdm = Twitterdm(settings)
diff --git a/gettor/services/email/sendmail.py b/gettor/services/email/sendmail.py
index 545fbb8..4644cb5 100644
--- a/gettor/services/email/sendmail.py
+++ b/gettor/services/email/sendmail.py
@@ -132,10 +132,14 @@ class Sendmail(object):
                         )
                     )
 
+                    body_msg = strings._("help_body_intro")
+                    body_msg += strings._("help_body_paragraph")
+                    body_msg += strings._("help_body_support")
+
                     yield self.sendmail(
                         email_addr=id,
                         subject=strings._("help_subject"),
-                        body=strings._("help_body")
+                        body=body_msg
                     )
 
                     yield self.conn.update_stats(
@@ -197,7 +201,14 @@ class Sendmail(object):
                         else:
                             link_msg = link_str
 
-                    body_msg = strings._("links_body").format(platform, link_msg, file)
+                    body_msg = strings._("links_body_platform").format(platform)
+                    body_msg += trings._("links_body_links").format(link_msg)
+                    body_msg += trings._("links_body_archive")
+                    body_msg += trings._("links_body_internet_archive")
+                    body_msg += trings._("links_body_google_drive")
+                    body_msg += trings._("links_body_internet_archive").format(file)
+                    body_msg += trings._("links_body_ending")
+
                     subject_msg = strings._("links_subject")
 
                     hid = hashlib.sha256(id.encode('utf-8'))
diff --git a/gettor/services/twitter/twitterdm.py b/gettor/services/twitter/twitterdm.py
index d78604a..629fa5f 100644
--- a/gettor/services/twitter/twitterdm.py
+++ b/gettor/services/twitter/twitterdm.py
@@ -144,9 +144,13 @@ class Twitterdm(object):
                         )
                     )
 
+                    body_msg = strings._("help_body_intro")
+                    body_msg += strings._("help_body_paragraph")
+                    body_msg += strings._("help_body_support")
+
                     yield self.twitterdm(
                         twitter_id=twitter_id,
-                        message=strings._("help_body")
+                        message=body_msg
                     )
 
                     yield self.conn.update_stats(
@@ -210,7 +214,13 @@ class Twitterdm(object):
                         else:
                             link_msg = link_str
 
-                    body_msg = strings._("links_body").format(platform, link_msg, file)
+                        body_msg = strings._("links_body_platform").format(platform)
+                        body_msg += trings._("links_body_links").format(link_msg)
+                        body_msg += trings._("links_body_archive")
+                        body_msg += trings._("links_body_internet_archive")
+                        body_msg += trings._("links_body_google_drive")
+                        body_msg += trings._("links_body_internet_archive").format(file)
+                        body_msg += trings._("links_body_ending")
 
                     hid = hashlib.sha256(twitter_id.encode('utf-8'))
                     log.info(
diff --git a/gettor/utils/options.py b/gettor/utils/options.py
index d313876..8504f42 100644
--- a/gettor/utils/options.py
+++ b/gettor/utils/options.py
@@ -24,12 +24,11 @@ def load_settings(config):
     settings.load()
     return settings
 
-def parse_settings():
+def parse_settings(locale, config):
     """
     Parse settings and loads strings in a given locale
     This function needs to be rewritten considering passing a locale and
     returing translated strings
     """
-    strings.load_strings("en")
-    config = "/home/gettor/gettor/gettor.conf.json"
+    strings.load_strings(locale)
     return load_settings(config)
diff --git a/share/locale/en.json b/share/locale/en.json
index 8ed6722..be7dd33 100644
--- a/share/locale/en.json
+++ b/share/locale/en.json
@@ -1,7 +1,15 @@
 {
-  "links_body": "You requested Tor Browser for {}.\n\nYou will need only one of the links below to download the bundle. If a link does not work for you, try the next one.\n\n{}\n\nShould you have issues with any of the links above you can access the following archives:\n\n- Internet Archive: https://archive.org/details/@gettor\n\n- Google Drive folder: https://drive.google.com/open?id=13CADQTsCwrGsIID09YQbNz2DfRMUoxUU\n\nDownload the file: {}\n\n \n--\nGetTor",
+  "links_body_platform": "You requested Tor Browser for {}.\n\n",
+  "links_body_links": "You will need only one of the links below to download the bundle. If a link does not work for you, try the next one.\n\n{}\n\n",
+  "links_body_archive": "Should you have issues with any of the links above you can access the following archives:\n\n",
+  "links_body_internet_archive": "- Internet Archive: https://archive.org/details/@gettor\n\n";,
+  "links_body_google_drive": "- Google Drive folder: https://drive.google.com/open?id=13CADQTsCwrGsIID09YQbNz2DfRMUoxUU\n\n";,
+  "links_body_internet_archive": "Download the file: {}\n\n",
+  "links_body_ending": "\n--\nGetTor",
   "links_subject": "[GetTor] Links for your request",
-  "help_body": "This is how you can request a tor browser bundle link.\n\n Send an email to: gettor@xxxxxxxxxxxxxx\n\nIn the body of the email only write: <operating system> <language>.\n\nWe only support windows, osx and linux as operating systems.\n\nAt the moment please only use en as language.\n\n",
+  "help_body_intro": "This is how you can request a tor browser bundle link.\n\n",
+  "help_body_paragraph": "Send an email to: gettor@xxxxxxxxxxxxxx\n\nIn the body of the email only write: <operating system> <language>.\n\n",
+  "help_body_support": "We only support windows, osx and linux as operating systems.\n\nAt the moment please only use en as language.\n\n",
   "help_subject": "[GetTor] Help Email",
   "help_debug": "Log application errors to stdout",
   "help_config": "Custom config file location (optional)",
@@ -9,8 +17,5 @@
   "smtp_mirrors_subject": "[GetTor] Mirrors",
   "smtp_help_subject": "[GetTor] Help",
   "smtp_unsupported_locale_subject": "[GetTor] Unsupported locale",
-  "smtp_unsupported_locale_msg": "The locale you requested '{}' is not supported.",
-  "smtp_vlinks_msg": "You requested Tor Browser for {}.\n\nYou will need only one of the links below to download the bundle. If a link does not work for you, try the next one.\n\n{}\n\nShould you have issues with any of the links above you can access the following Google Drive folder: https://drive.google.com/open?id=13CADQTsCwrGsIID09YQbNz2DfRMUoxUU\n\n Download the file: {}\n\n \n--\nGetTor",
-  "smtp_mirrors_msg": "Hi! this is the GetTor robot.\n\nThank you for your request. Attached to this email you will find\nan updated list of mirrors of Tor Project's website.",
-  "smtp_help_msg": "Hi! This is the GetTor robot. I am here to help you download the\nlatest version of Tor Browser.\n\nPlease reply to this message with one of the options below:\n\nwindows\nlinux\nosx\nmirrors\n\nI will then send you the download instructions.\n\nIf you are unsure, just send a blank reply to this message."
+  "smtp_unsupported_locale_msg": "The locale you requested '{}' is not supported."
 }
diff --git a/share/locale/es.json b/share/locale/es.json
index 4d18f61..be7dd33 100644
--- a/share/locale/es.json
+++ b/share/locale/es.json
@@ -1,16 +1,21 @@
 {
-  "links_body": "You requested Tor Browser for {}.\n\nYou will need only one of the links below to download the bundle. If a link does not work for you, try the next one.\n\n{}\n\nShould you have issues with any of the links above you can access the following archives:\n\n- Internet Archive: https://archive.org/details/@gettor\n\n- Google Drive folder: https://drive.google.com/open?id=13CADQTsCwrGsIID09YQbNz2DfRMUoxUU\n\nDownload the file: {}\n\n \n--\nGetTor",
+  "links_body_platform": "You requested Tor Browser for {}.\n\n",
+  "links_body_links": "You will need only one of the links below to download the bundle. If a link does not work for you, try the next one.\n\n{}\n\n",
+  "links_body_archive": "Should you have issues with any of the links above you can access the following archives:\n\n",
+  "links_body_internet_archive": "- Internet Archive: https://archive.org/details/@gettor\n\n";,
+  "links_body_google_drive": "- Google Drive folder: https://drive.google.com/open?id=13CADQTsCwrGsIID09YQbNz2DfRMUoxUU\n\n";,
+  "links_body_internet_archive": "Download the file: {}\n\n",
+  "links_body_ending": "\n--\nGetTor",
   "links_subject": "[GetTor] Links for your request",
-  "help_body": "This is how you can request a tor browser bundle link.\n\n Send an email to: gettor@xxxxxxxxxxxxxx\n\nIn the body of the email only write: <operating system> <language>.\n\nWe only support windows, osx and linux as operating systems.\n\nAt the moment please only use en as language.\n\n",
+  "help_body_intro": "This is how you can request a tor browser bundle link.\n\n",
+  "help_body_paragraph": "Send an email to: gettor@xxxxxxxxxxxxxx\n\nIn the body of the email only write: <operating system> <language>.\n\n",
+  "help_body_support": "We only support windows, osx and linux as operating systems.\n\nAt the moment please only use en as language.\n\n",
   "help_subject": "[GetTor] Help Email",
   "help_debug": "Log application errors to stdout",
   "help_config": "Custom config file location (optional)",
   "smtp_links_subject": "[GetTor] Links for your request",
   "smtp_mirrors_subject": "[GetTor] Mirrors",
-  "smtp_help_subject": "[GetTor] Ayuda",
+  "smtp_help_subject": "[GetTor] Help",
   "smtp_unsupported_locale_subject": "[GetTor] Unsupported locale",
-  "smtp_unsupported_locale_msg": "The locale you requested '{}' is not supported.",
-  "smtp_vlinks_msg": "You requested Tor Browser for {}.\n\nYou will need only one of the links below to download the bundle. If a link does not work for you, try the next one.\n\n{}\n\nShould you have issues with any of the links above you can access the following Google Drive folder: https://drive.google.com/open?id=13CADQTsCwrGsIID09YQbNz2DfRMUoxUU\n\n Download the file: {}\n\n \n--\nGetTor",
-  "smtp_mirrors_msg": "Hi! this is the GetTor robot.\n\nThank you for your request. Attached to this email you will find\nan updated list of mirrors of Tor Project's website.",
-  "smtp_help_msg": "Hi! This is the GetTor robot. I am here to help you download the\nlatest version of Tor Browser.\n\nPlease reply to this message with one of the options below:\n\nwindows\nlinux\nosx\nmirrors\n\nI will then send you the download instructions.\n\nIf you are unsure, just send a blank reply to this message."
+  "smtp_unsupported_locale_msg": "The locale you requested '{}' is not supported."
 }
diff --git a/share/locale/pt.json b/share/locale/pt.json
index 8ed6722..be7dd33 100644
--- a/share/locale/pt.json
+++ b/share/locale/pt.json
@@ -1,7 +1,15 @@
 {
-  "links_body": "You requested Tor Browser for {}.\n\nYou will need only one of the links below to download the bundle. If a link does not work for you, try the next one.\n\n{}\n\nShould you have issues with any of the links above you can access the following archives:\n\n- Internet Archive: https://archive.org/details/@gettor\n\n- Google Drive folder: https://drive.google.com/open?id=13CADQTsCwrGsIID09YQbNz2DfRMUoxUU\n\nDownload the file: {}\n\n \n--\nGetTor",
+  "links_body_platform": "You requested Tor Browser for {}.\n\n",
+  "links_body_links": "You will need only one of the links below to download the bundle. If a link does not work for you, try the next one.\n\n{}\n\n",
+  "links_body_archive": "Should you have issues with any of the links above you can access the following archives:\n\n",
+  "links_body_internet_archive": "- Internet Archive: https://archive.org/details/@gettor\n\n";,
+  "links_body_google_drive": "- Google Drive folder: https://drive.google.com/open?id=13CADQTsCwrGsIID09YQbNz2DfRMUoxUU\n\n";,
+  "links_body_internet_archive": "Download the file: {}\n\n",
+  "links_body_ending": "\n--\nGetTor",
   "links_subject": "[GetTor] Links for your request",
-  "help_body": "This is how you can request a tor browser bundle link.\n\n Send an email to: gettor@xxxxxxxxxxxxxx\n\nIn the body of the email only write: <operating system> <language>.\n\nWe only support windows, osx and linux as operating systems.\n\nAt the moment please only use en as language.\n\n",
+  "help_body_intro": "This is how you can request a tor browser bundle link.\n\n",
+  "help_body_paragraph": "Send an email to: gettor@xxxxxxxxxxxxxx\n\nIn the body of the email only write: <operating system> <language>.\n\n",
+  "help_body_support": "We only support windows, osx and linux as operating systems.\n\nAt the moment please only use en as language.\n\n",
   "help_subject": "[GetTor] Help Email",
   "help_debug": "Log application errors to stdout",
   "help_config": "Custom config file location (optional)",
@@ -9,8 +17,5 @@
   "smtp_mirrors_subject": "[GetTor] Mirrors",
   "smtp_help_subject": "[GetTor] Help",
   "smtp_unsupported_locale_subject": "[GetTor] Unsupported locale",
-  "smtp_unsupported_locale_msg": "The locale you requested '{}' is not supported.",
-  "smtp_vlinks_msg": "You requested Tor Browser for {}.\n\nYou will need only one of the links below to download the bundle. If a link does not work for you, try the next one.\n\n{}\n\nShould you have issues with any of the links above you can access the following Google Drive folder: https://drive.google.com/open?id=13CADQTsCwrGsIID09YQbNz2DfRMUoxUU\n\n Download the file: {}\n\n \n--\nGetTor",
-  "smtp_mirrors_msg": "Hi! this is the GetTor robot.\n\nThank you for your request. Attached to this email you will find\nan updated list of mirrors of Tor Project's website.",
-  "smtp_help_msg": "Hi! This is the GetTor robot. I am here to help you download the\nlatest version of Tor Browser.\n\nPlease reply to this message with one of the options below:\n\nwindows\nlinux\nosx\nmirrors\n\nI will then send you the download instructions.\n\nIf you are unsure, just send a blank reply to this message."
+  "smtp_unsupported_locale_msg": "The locale you requested '{}' is not supported."
 }
diff --git a/tests/test_email_service.py b/tests/test_email_service.py
index 4c8ff10..8e60f7a 100644
--- a/tests/test_email_service.py
+++ b/tests/test_email_service.py
@@ -13,7 +13,7 @@ class EmailServiceTests(unittest.TestCase):
     # Fail any tests which take longer than 15 seconds.
     timeout = 15
     def setUp(self):
-        self.settings = conftests.options.parse_settings()
+        self.settings = conftests.options.parse_settings("en","./gettor.conf.json")
         self.sm_client = conftests.sendmail.Sendmail(self.settings)
         self.locales = conftests.strings.get_locales()
 
diff --git a/tests/test_locales.py b/tests/test_locales.py
index bea7e9f..d4e52c5 100644
--- a/tests/test_locales.py
+++ b/tests/test_locales.py
@@ -11,7 +11,7 @@ class LocalesTests(unittest.TestCase):
     # Fail any tests which take longer than 15 seconds.
     timeout = 15
     def setUp(self):
-        self.settings = conftests.options.parse_settings()
+        self.settings = conftests.options.parse_settings("en","./gettor.conf.json")
         self.locales = conftests.strings.get_locales()
 
 
diff --git a/tests/test_twitter.py b/tests/test_twitter.py
index fe155cc..7458cfc 100644
--- a/tests/test_twitter.py
+++ b/tests/test_twitter.py
@@ -10,7 +10,7 @@ class TwitterTests(unittest.TestCase):
     # Fail any tests which take longer than 15 seconds.
     timeout = 15
     def setUp(self):
-        self.settings = conftests.options.parse_settings()
+        self.settings = conftests.options.parse_settings("en","./gettor.conf.json")
         self.tw_client = conftests.twitter.Twitter(self.settings)
 
 



_______________________________________________
tor-commits mailing list
tor-commits@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-commits