[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [tor-bugs] #9874 [BridgeDB]: Research/design a way to automate testing of BridgeDB's HTTPS and email distributors
#9874: Research/design a way to automate testing of BridgeDB's HTTPS and email
distributors
-------------------------+-------------------------------------------------
Reporter: isis | Owner: isis
Type: | Status: needs_review
enhancement | Milestone:
Priority: normal | Version:
Component: | Keywords: bridgedb-unittests, automation, ci,
BridgeDB | bridgedb-gsoc-application
Resolution: | Parent ID:
Actual Points: |
Points: |
-------------------------+-------------------------------------------------
Comment (by isis):
Replying to [comment:18 trygve]:
> Thank you for feedback. I've attached a patch to fix the failing test.
I'm new to git (used to cvs, svn and hg), so apologies if I've done this
wrong.
>
Nope, it's cool. `git checkout -b YOURBRANCH develop` and then linking to
a publicly available remote is generally the best way to go, mostly
because straight patch files obviously lack commit metadata. I ''think''
there is a way to use `git-hg` and `hg-git` to work on a git repo in
mercurial, but then I'm not super familiar with Mercurial.
> The ipv6 bridge has been changed to a vanilla ipv4 bridge. I've also
added a new test to check for a scramblesuit bridge. Both of these
surprised me because they had a different number of fields in each bridge
line, but I've adjusted the tests accordingly.
>
Yep! There's actually a possibly infinite (sort of) amount of `k=v`s which
can be on the end of bridge-extrainfo `transport` lines, which
[https://gitweb.torproject.org/torspec.git/blob/HEAD:/pt-spec.txt#l113
BridgeDB is supposed to put on the end of a bridge line].
> A few comments:
> * mechanize is in '.test.requirements.txt' but not 'requirements.txt'.
I'm assuming this is intentional, but it means that the https tests fail
when using 'requirements.txt'
I actually need to remove `requirements.txt`, or remove at least the part
in the `setup.py` script which reads `requirements.txt` and parses
everything into `setuptools`/`distribute` dependencies. On the production
server, we actually delete `requirements.txt` before installing, because
otherwise `setuptools`/`distribute` drops down to the embedded
`easy_install`, which we believe is quite unsafe.
> * leakspin is in '.test.requirements.txt', but its dependency 'ipaddr'
is not
When `pip` installs dependencies, it looks into the
`setuptools`/`distribute` dependencies listed in the `install_requires`
and `dependency_links` parameters to the `setuptools.setup()` call, and
also in the `MANIFEST.in` file, if one is present. So `pip` recursively
downloads dependencies. Also `ipaddr` is specified with a rather
strange/special syntax in BridgeDB's `requirements.txt`:
{{{
https://ipaddr-
py.googlecode.com/files/ipaddr-2.1.10.tar.gz#sha1=c608450b077b19773d4f1b5f1ef88b26f6650ce0#egg=ipaddr-2.1.10-py2.7
}}}
because BridgeDB requires a specific version of `ipaddr` (due to upstream
bugs, which I've pointed out to the maintainers). This version is
1. missing from PyPI (although other versions are there)
2. https://googlecode.com is what `pip` calls "an external index"
(meaning that it is not https://pypi.org). I filed several CVEs against
pip last year for unsafe behaviours while crawling indexes, and they very
responsibly fixed these issues. (In fact, it was already fixed in alpha
versions of `pip` at the time, just not the version in all the distros.)
Because we ''need'' to use "an external index" for this dependency, we
specify `https://` and the expected SHA-1 hash for the package, so that
`pip` knows how to verify it.
> * I had to change EMAIL_SMTP_PORT to 2525 in bridgedb.conf. This might
not be obvious to others.
> * I had to add "127.0.0.1" to EMAIL_DOMAINS in bridgedb.conf. This might
not be obvious to others. It may be possible to change the scripts to send
from one of the 3 allowed domains, as the SMTP server in the test script
should intercept these emails without forwarding on to the real domains.
This is possible, and I considered doing it, but I didn't really want to
change your scripts while testing them out. It requires a change to
`/etc/hosts`, however, which
[https://gitweb.torproject.org/bridgedb.git/blob/HEAD:/.travis.yml#l26 we
already do for Travis], but it's perhaps less nice to ask other people to
do this in order to run the tests.
We could change your `test_smtp.py` script to use `example.com`, set that
in `/etc/hosts` for Travis, and then have your tests skip themselves if
they aren't being run on Travis. Or...as you suggest:
> * I had to perform a few more steps to get everything to work ('leakspin
-n 100' and 'scripts/make-ssl-cert'). This was all documented in the
excellent README, but would it be useful to create a little script that
sets up the test environment automatically? You've added something that
does this for Travis CI, but I've been working from the shell, entering
commands manually.
We could move the `before_install` section in `.travis.yml` to some
script, but then we need to tell people to run a script which runs more
scripts before testing. The nice thing about having the command separate
in Travis is that you can see which one failed, versus a whole setup
script failing. I'm not really sure which one is better. Unless you have
more ideas?
It occurs to me that your tests might fail on random machines, like if
port 2525 was already in use, or they have a weird firewall configuration.
Perhaps we should only expect them to run in special environments?
We could do `export TESTING_BRIDGEDB=1` in the test environment setup
script and Travis, and then check for that in your tests, rather than
checking for `os.env.get("TRAVIS")` and only running on Travis. That way,
your tests would only run if the test environment setup script had been
run, or on CI machines, but would skip if someone tried to run all the
tests on a random machine.
Or we could just add the `sed` commands to the `README` and wish everyone
the best of luck getting this crazy program to run. :/
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/9874#comment:19>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs