[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
[or-cvs] [torflow/master] Give the masses what they want.
Author: Mike Perry <mikeperry-git@xxxxxxxxxx>
Date: Thu, 30 Sep 2010 18:57:17 -0700
Subject: Give the masses what they want.
Commit: 279fc81c630a8b6b9f00c5dba7a70f5ffee4a0c2
Tweak params at the best guess to running this thing fast, with as little
noise as possible at the expense of reliability.
Also provide a FAQ ;)
---
NetworkScanners/ExitAuthority/README.ADHD-FAQ | 35 +++++++++++++++++++++
NetworkScanners/ExitAuthority/README.ExitScanning | 4 +-
NetworkScanners/ExitAuthority/cron.sh | 8 ++++-
NetworkScanners/ExitAuthority/snakeinspector.py | 16 ++++++---
NetworkScanners/ExitAuthority/soat_config.py | 7 ++--
5 files changed, 57 insertions(+), 13 deletions(-)
create mode 100644 NetworkScanners/ExitAuthority/README.ADHD-FAQ
diff --git a/NetworkScanners/ExitAuthority/README.ADHD-FAQ b/NetworkScanners/ExitAuthority/README.ADHD-FAQ
new file mode 100644
index 0000000..839d2a9
--- /dev/null
+++ b/NetworkScanners/ExitAuthority/README.ADHD-FAQ
@@ -0,0 +1,35 @@
+ SOAT FAQ
+ Don't Panic.
+
+
+
+Q: HELP! IS IT REALLY THIS SIMPLE? ARE THIS ALL THE COMMANDS NEED?
+A: Yes. Until it breaks :)
+
+
+Q: HELP! I NEED TO START TOR FOR THIS!
+A:
+ tor -f ./data/tor/torrc &
+
+
+Q: HELP! I NEED TO FIND SNAKES!
+A:
+ ./soat.py --http --ssl >& ./data/soat.log
+
+
+Q: HELP! I WANT TO JUST TEST A FIXED URL!
+A:
+ echo 'scan_filetypes = ["html"]' >> soat_config.py
+ ./soat.py --http --target "http://moria.csail.mit.edu/index.html"; >& ./data/soat.log
+
+
+Q: HELP! I NEED TO EMAIL TEH LIST EVERY HOWAR!
+A:
+ echo -e "45 0-23 * * * $PWD/cron.sh" | crontab -
+
+
+Q: HELP! SOAT CRASHED! I NEED TO RESTART!
+A:
+ ./soat.py --resume 0 --http --ssl >& ./data/soat.log.0
+
+
diff --git a/NetworkScanners/ExitAuthority/README.ExitScanning b/NetworkScanners/ExitAuthority/README.ExitScanning
index dfb5989..9f7c477 100644
--- a/NetworkScanners/ExitAuthority/README.ExitScanning
+++ b/NetworkScanners/ExitAuthority/README.ExitScanning
@@ -251,10 +251,10 @@ soat_config, you can set mail_password = None, and you'll be
prompted to provide it when snakeinspector is run.
In this current directory is a cron.sh script that calls snakeinspector to
-email results that completed in the last 24 hours, or since the last time
+email results that completed in the last hour, or since the last time
you've run it. Add it to `crontab -e` like so:
-0 3 * * * ~/code/torflow.git/NetworkScanners/ExitAuthority/cron.sh
+0 * * * * ~/code/torflow.git/NetworkScanners/ExitAuthority/cron.sh
Alright that covers the basics. Let's get those motherfuckin snakes off
this motherfuckin Tor!
diff --git a/NetworkScanners/ExitAuthority/cron.sh b/NetworkScanners/ExitAuthority/cron.sh
index f768810..18ce1d0 100755
--- a/NetworkScanners/ExitAuthority/cron.sh
+++ b/NetworkScanners/ExitAuthority/cron.sh
@@ -2,7 +2,13 @@
SCANDIR=~/code/torflow.git/NetworkScanners/ExitAuthority/
-$SCANDIR/snakeinspector.py --confirmed --email --noreason FailureTimeout --croninterval 24 --siterate 3
+# 1. Email results to addresses in soat_config.py (--email)
+# 2. Ignore timeout errors (--noreason FailureTimeout)
+# 3. Schedule this script every hour (--croninterval 1).
+# 4. Only report from urls that fail from less than 10% of the total
+# exits tested so far. (--siterate 10)
+# 5. Only report exits that fail 100% of their tests (--exitrate 100)
+$SCANDIR/snakeinspector.py --email --noreason FailureTimeout --exitrate 100 --siterate 10 --croninterval 1
# Optionally, you can use these two lines to allow less regular cron
# scheduling:
diff --git a/NetworkScanners/ExitAuthority/snakeinspector.py b/NetworkScanners/ExitAuthority/snakeinspector.py
index d085914..fcabf78 100755
--- a/NetworkScanners/ExitAuthority/snakeinspector.py
+++ b/NetworkScanners/ExitAuthority/snakeinspector.py
@@ -219,12 +219,16 @@ def main(argv):
else:
if conf.cron_interval and r.timestamp < now-conf.cron_interval-60:
continue
- if r.site_result_rate[1] != 0 and \
- conf.siterate < (100.0*r.site_result_rate[0])/r.site_result_rate[1]:
- continue
- if r.exit_result_rate[1] != 0 and \
- conf.exitrate > (100.0*r.exit_result_rate[0])/r.exit_result_rate[1]:
- continue
+ # Only apply siterate filters if enough tests have run for them to be
+ # true. Otherwise, assume they are true (don't check them).
+ if 100.0/conf.siterate > r.site_result_rate[1]:
+ if r.site_result_rate[1] != 0 and \
+ conf.siterate < (100.0*r.site_result_rate[0])/r.site_result_rate[1]:
+ continue
+ if 100.0/conf.exitrate > r.exit_result_rate[1]:
+ if r.exit_result_rate[1] != 0 and \
+ conf.exitrate > (100.0*r.exit_result_rate[0])/r.exit_result_rate[1]:
+ continue
if (not conf.statuscode or r.status == conf.statuscode) and \
(not conf.proto or r.proto == conf.proto) and \
(not conf.resultfilter or r.__class__.__name__ == conf.resultfilter):
diff --git a/NetworkScanners/ExitAuthority/soat_config.py b/NetworkScanners/ExitAuthority/soat_config.py
index 32e12fd..3e13463 100644
--- a/NetworkScanners/ExitAuthority/soat_config.py
+++ b/NetworkScanners/ExitAuthority/soat_config.py
@@ -83,8 +83,8 @@ min_targets = 10
# How many times each node should be tested before removing it from
# a run loop (can be overriden on command line)
-num_tests_per_node = 1
-num_rescan_tests_per_node = 3
+num_tests_per_node = 5
+num_rescan_tests_per_node = 5
# Number of timeouts before we consider a node failed.
num_timeouts_per_node = 4
@@ -98,7 +98,7 @@ num_connfails_per_node = 2
# Rescan failures upon finishing the run loop.
# FIXME: This does have the downside that we do NOT prune excessively
# dynamic URLs during this loop, and so false positives may accumulate...
-rescan_at_finish = False
+rescan_at_finish = True
# Should we restart scanning from the beginning at the finish?
restart_at_finish = True
@@ -267,4 +267,3 @@ http_failed_dir = soat_dir + 'http/failed/'
http_inconclusive_dir = soat_dir + 'http/inconclusive/'
http_falsepositive_dir = soat_dir + 'http/falsepositive/'
-
--
1.7.1