[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
[or-cvs] r14663: add blossom files not captured by the initial Tor SVN snapsh (blossom/trunk)
Author: goodell
Date: 2008-05-19 00:23:06 -0400 (Mon, 19 May 2008)
New Revision: 14663
Added:
blossom/trunk/blossom-init
blossom/trunk/desiderata
blossom/trunk/edgeproxy
blossom/trunk/exit.py
blossom/trunk/exitlist
Log:
add blossom files not captured by the initial Tor SVN snapshot
Added: blossom/trunk/blossom-init
===================================================================
--- blossom/trunk/blossom-init (rev 0)
+++ blossom/trunk/blossom-init 2008-05-19 04:23:06 UTC (rev 14663)
@@ -0,0 +1,3 @@
+#!/bin/sh
+#$Id: blossom-init,v 1.3 2005-04-15 22:13:18 goodell Exp $
+/afs/eecs.harvard.edu/user/goodell/research/blossom/src/blossom-client.py &
Added: blossom/trunk/desiderata
===================================================================
--- blossom/trunk/desiderata (rev 0)
+++ blossom/trunk/desiderata 2008-05-19 04:23:06 UTC (rev 14663)
@@ -0,0 +1,66 @@
+SECTION 5B: Tor+Blossom desiderata
+
+The interests of Blossom would be best served by implementing the following
+modifications to Tor:
+
+I. CLIENTS
+
+Objectives: Ultimately, we want Blossom requests to be indistinguishable in
+format from non-Blossom .exit requests, i.e. hostname.forwarder.exit.
+
+Proposal: Blossom is a process that manipulates Tor, so it should be
+implemented as a Tor Control, extending control-spec.txt. For each request,
+Tor uses the control protocol to ask the Blossom process whether it (the
+Blossom process) wants to build or assign a particular circuit to service the
+request. Blossom chooses one of the following responses:
+
+a. (Blossom exit node, circuit cached) "use this circuit" -- provides a circuit
+ID
+
+b. (Blossom exit node, circuit not cached) "I will build one" -- provides a
+list of routers, gets a circuit ID.
+
+c. (Regular (non-Blossom) exit node) "No, do it yourself" -- provides nothing.
+
+II. ROUTERS
+
+Objectives: Blossom routers are like regular Tor routers, except that Blossom
+routers need these features as well:
+
+a. the ability to open peresistent connections,
+
+b. the ability to know whwther they should use a persistent connection to reach
+another router,
+
+c. the ability to define a set of routers to which to establish persistent
+connections, as readable from a configuration file, and
+
+d. the ability to tell a directory server that (1) it is Blossom-enabled, and
+(2) it can be reached by some set of routers to which it explicitly establishes
+persistent connections.
+
+Proposal: Address the aforementioned points as follows.
+
+a. need the ability to open a specified number of persistent connections. This
+can be accomplished by implementing a generic should_i_close_this_conn() and
+which_conns_should_i_try_to_open_even_when_i_dont_need_them().
+
+b. The Tor design already supports this, but we must be sure to establish the
+persistent connections explicitly, re-establish them when they are lost, and
+not close them unnecessarily.
+
+c. We must modify Tor to add a new configuration option, allowing either (a)
+explicit specification of the set of routers to which to establish persistent
+connections, or (b) a random choice of some nodes to which to establish
+persistent connections, chosen from the set of nodes local to the transport
+domain of the specified directory server (for example).
+
+III. DIRSERVERS
+
+Objective: Blossom directory servers may provide extra
+fields in their network-status pages. Blossom directory servers may
+communicate with Blossom clients/routers in nonstandard ways in addition to
+standard ways.
+
+Proposal: Geoff should be able to implement a directory server according to the
+Tor specification (dir-spec.txt).
Added: blossom/trunk/edgeproxy
===================================================================
--- blossom/trunk/edgeproxy (rev 0)
+++ blossom/trunk/edgeproxy 2008-05-19 04:23:06 UTC (rev 14663)
@@ -0,0 +1,467 @@
+#!/usr/bin/perl -w
+# $Id: edgeproxy,v 1.18 2006-03-13 14:05:49 goodell Exp $
+$license = <<EOF
+Copyright (c) 2005 Geoffrey Goodell.
+
+This program is free software; you can redistribute it and/or modify it
+under the terms of version 2 of the GNU General Public License as
+published by the Free Software Foundation.
+
+This program is distributed in the hope that it will be useful, but WITHOUT ANY
+WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A
+PARTICULAR PURPOSE. See the GNU General Public License for more details.
+
+You should have received a copy of the GNU General Public License along with
+this program; if not, write to the Free Software Foundation, Inc., 59 Temple
+Place - Suite 330, Boston, MA 02111-1307, USA.
+
+EOF
+;
+
+# derived from The Perl Cookbook (O'Reilly), Recipe 17.19.
+# http://www.unix.org.ua/orelly/perl/cookbook/ch17_19.htm
+# fwdport -- act as proxy forwarder for dedicated services
+
+use strict; # require declarations
+use Getopt::Long; # for option processing
+use Net::hostent; # by-name interface for host info
+use IO::Socket; # for creating server and client sockets
+use POSIX ":sys_wait_h"; # for reaping our dead children
+
+use vars qw($license);
+
+my (
+ %Children, # hash of outstanding child processes
+ $REMOTE, # whom we connect to on the outside
+ $LOCAL, # where we listen to on the inside
+ $SERVICE, # our service name or port number
+ $proxy_server, # the socket we accept() from
+ $ME, # basename of this program
+);
+
+my $DEBUG = 0; # debug level
+
+($ME = $0) =~ s,.*/,,; # retain just basename of script name
+
+check_args(); # processing switches
+start_proxy(); # launch our own server
+service_clients(); # wait for incoming
+die "NOT REACHED"; # you can't get here from there
+
+# process command line switches using the extended
+# version of the getopts library.
+sub check_args {
+ GetOptions(
+ "remote=s" => \$REMOTE,
+ "local=s" => \$LOCAL,
+ "service=s" => \$SERVICE,
+ "debug=s" => \$DEBUG,
+ ) or die <<EOUSAGE;
+ usage: $0 [ --remote host ] [ --local interface ] [ --service service ] [ --debug level ]
+EOUSAGE
+ die "Need remote" unless $REMOTE;
+ die "Need local or service" unless $LOCAL || $SERVICE;
+}
+
+sub license() {
+ print $license; exit 0;
+}
+
+sub log_info($$) {
+ my ($debuglevel, $line) = (shift, shift);
+ chomp $line;
+ print STDERR "$line\n" if $debuglevel < $DEBUG;
+}
+
+sub append_exit($$) {
+ my ($host, $router) = (shift, shift);
+ if ($host !~ /\.[A-Za-z0-9-]+\.exit(:[0-9+])?$/
+ and $host !~ /\.[cq]\.[A-Za-z0-9-.]+\.blossom(:[0-9+])?$/) {
+ if($host =~ /^(\S+)(:[0-9]+)$/) {
+ $host = $1 . ".$router" . $2;
+ } else {
+ $host .= ".$router";
+ }
+ }
+ return $host;
+}
+
+# begin our server
+sub start_proxy {
+ my @proxy_server_config = (
+ Proto => 'tcp',
+ Reuse => 1,
+ Listen => SOMAXCONN,
+ );
+ push @proxy_server_config, LocalPort => $SERVICE if $SERVICE;
+ push @proxy_server_config, LocalAddr => $LOCAL if $LOCAL;
+ $proxy_server = IO::Socket::INET->new(@proxy_server_config)
+ or die "can't create proxy server: $@";
+ log_info(1, "[proxy server on " . ($LOCAL || $SERVICE) . " initialized]");
+}
+
+sub service_clients {
+ my (
+ $local_client, # someone internal wanting out
+ $lc_info, # local client's name/port information
+ $remote_server, # the socket for escaping out
+ @rs_config, # temp array for remote socket options
+ $rs_info, # remote server's name/port information
+ $kidpid, # spawned child for each connection
+ );
+
+ $SIG{CHLD} = \&REAPER; # harvest the moribund
+
+ accepting();
+
+ # an accepted connection here means someone inside wants out
+ while (1) {
+ while ($local_client = $proxy_server->accept()) {
+ $lc_info = peerinfo($local_client);
+ set_state("servicing local $lc_info");
+ log_info(1, "[received connect from $lc_info]");
+
+ @rs_config = (
+ Proto => 'tcp',
+ PeerAddr => $REMOTE,
+ );
+ push(@rs_config, PeerPort => $SERVICE) if $SERVICE;
+
+ log_info(1, "[connecting to $REMOTE]");
+ set_state("connecting to $REMOTE"); # see below
+ $remote_server = IO::Socket::INET->new(@rs_config);
+
+ if(not defined $remote_server) {
+ next;
+ }
+
+ $rs_info = peerinfo($remote_server);
+ set_state("connected to $rs_info");
+
+ $kidpid = fork();
+ die "Cannot fork" unless defined $kidpid;
+ if ($kidpid) {
+ $Children{$kidpid} = time(); # remember his start time
+ close $remote_server; # no use to master
+ close $local_client; # likewise
+ next; # go get another client
+ }
+
+ # at this point, we are the forked child process dedicated
+ # to the incoming client. but we want a twin to make i/o
+ # easier.
+
+ close $proxy_server; # no use to slave
+
+ pipe READER, WRITER;
+
+ $kidpid = fork();
+ die "Cannot fork" unless defined $kidpid;
+
+ # now each twin sits around and ferries lines of data.
+ # see how simple the algorithm is when you can have
+ # multiple threads of control?
+
+ # this is the fork's parent, the master's child
+ if ($kidpid) {
+ close WRITER;
+
+ my $proxypath = "";
+ my $reverseproxy = undef;
+
+ my $router = <READER>;
+ if($router) {
+ chomp $router;
+ } else {
+ $router = "";
+ }
+ if($router =~ /\+(\S+)$/) {
+ $proxypath = "/proxy/";
+ $reverseproxy = $proxypath . "http://$1";;
+ $router =~ s/\+\S+$//;
+ log_info(1, "reverse proxy: $reverseproxy");
+ }
+ $router = undef if $router eq "";
+
+ set_state("$rs_info --> $lc_info");
+ select($local_client); $| = 1;
+
+ # Perform substitution for A, IMG, and LINK tags in HTML documents
+
+ my $html = undef;
+ my $length = undef;
+ my $content = "";
+ my $headers = "";
+ my $type = "";
+
+ while(<$remote_server>) {
+ log_info(1, " recv: $_");
+ if(/^Content-Type: (\S+?)(;.*)?\r$/i) {
+ $type = $1;
+ $html = $type if $type =~ /^text\/html$/;
+ $headers .= $_;
+ } elsif(/^Content-Length: (\S+)\r$/i) {
+ $length = $1;
+ $headers .= $_;
+ } elsif(/^Location: (https?:\/\/)([A-Za-z0-9.-]+)(:[0-9]+)?(\/?)(.*?)\r/i) {
+ my ($pre, $host, $port, $post, $rest) = ($1, $2, $3, $4, $5);
+
+ $port = "" if not $port;
+
+ my $before = "$pre$host$port$post";
+
+ $host = append_exit($host, $router) if $router;
+ $pre = "$proxypath$pre" if $reverseproxy;
+
+ my $after = "$pre$host$port$post";
+
+ log_info(2, "converting: Location: $before --> Location: $after");
+
+ $headers .= "Location: $after$rest\r\n";
+ } elsif($reverseproxy and /^Location: \/(.*)\r$/) {
+ my $before = "/$1";
+ my $after = "$reverseproxy$before";
+
+ log_info(2, "converting: Location: $before --> Location: $after");
+
+ $headers .= "Location: $after\n";
+ } elsif(/^\r$/) {
+ $headers .= $_;
+ last;
+ } else {
+ $headers .= $_;
+ }
+ }
+
+ if($html) {
+ log_info(0, " data: $type (recognized as HTML)");
+ my $next = "";
+
+ while(<$remote_server>) {
+ my $line = $_;
+
+ unless($router or $reverseproxy) {
+ $content .= $_;
+ next;
+ }
+
+ $line = "$next $line";
+ chomp $line;
+ $next = "";
+
+ while($line) {
+ if($line =~ /^<(a|form|frame|img|input|link)([^>]+)(action|href|src)=(\'?\"?)(https?:\/\/)([A-Za-z0-9.-]+)(:[0-9]+)?(\/|\")(.*)$/i) {
+ my ($tag, $attr, $label, $quote, $pre, $host, $port, $post, $rest)
+ = ($1, $2, $3, $4, $5, $6, $7, $8, $9);
+
+ $port = "" if not $port;
+
+ my $before = "$tag$attr$label=$quote$pre$host$port";
+
+ # normalize
+ $tag =~ y/A-Z/a-z/;
+ $host =~ y/A-Z/a-z/;
+
+ if($router) {
+ $host = append_exit($host, $router);
+ log_info(1, "<$tag tag: $host>");
+ }
+
+ my $after = "$tag$attr$label=$quote$proxypath$pre$host$port";
+
+ log_info(2, "converting: <$before --> <$after");
+
+ $content .= "<$after";
+ $line = "$post$rest";
+ } elsif($reverseproxy and $line =~ /^<(a|form|frame|img|input|link)([^>]+)(action|href|src)=(\'?\"?)\/(.*)$/i) {
+ my ($tag, $attr, $label, $quote, $rest) = ($1, $2, $3, $4, $5);
+
+ my $before = "$tag$attr$label=$quote/";
+
+ # normalize
+ $tag =~ y/A-Z/a-z/;
+
+ my $after = "$tag$attr$label=$quote$reverseproxy/";
+
+ log_info(2, "converting: <$before --> <$after");
+
+ $content .= "<$after";
+ $line = $rest;
+ } elsif($line =~ /^(<.*?>)(.*)$/) {
+ $content .= $1;
+ $line = $2;
+ } elsif($line =~ /^(<.*)$/) {
+ $next = $1;
+ $line = undef;
+ } elsif($line =~ /^(.*?)(<.*)$/) {
+ $content .= $1;
+ $line = $2;
+ } else {
+ $content .= $line;
+ $line = undef;
+ }
+ }
+ $content .= "\n";
+ }
+
+ foreach my $line (split /\n/, $headers) {
+ if($line =~ /^Content-Length: (\S+)\r$/i) {
+ $length = length $content;
+ print "Content-Length: $length\n";
+ } else {
+ print "$line\n";
+ }
+ }
+ print $content;
+
+ } elsif($length) {
+ my $data = "";
+ print $headers;
+
+ log_info(0, " data: $type [$length]");
+ read($remote_server, $data, $length) or die " error: $?";
+ print "$data\n";
+ } else {
+ print $headers;
+ if($type) {
+ log_info(0, " data: $type");
+ } else {
+ log_info(0, " data: unspecified type");
+ }
+ while(<$remote_server>) {
+ print;
+ }
+ }
+
+ kill('TERM', $kidpid); # kill my twin cause we're done
+ }
+ # this is the fork's child, the master's grandchild
+ else {
+ close READER;
+
+ set_state("$rs_info <-- $lc_info");
+ select($remote_server); $| = 1;
+
+ # Perform HTTP Host field substitution
+
+ my $post = 1;
+ while($post) {
+ my $length = 0;
+ my $reverseproxy = undef;
+
+ $post = undef;
+
+ while(<$local_client>) {
+ if(/^Content-Length: (\S+)\r$/i) {
+ $length = $1;
+ print;
+ log_info(1, " send: $_");
+ } elsif(/^Host: (\S+)\r$/i) {
+ my $router = "";
+ my $repl = $reverseproxy || $1;
+ if($repl =~ /\.[A-Za-z0-9-]+\.exit(:[0-9]+)?$/) {
+ $repl =~ s/\.([A-Za-z0-9-]+\.exit)((:[0-9]+)?)$/$2/;
+ $router = $1;
+ }
+ if($repl =~ /\.[cq]\.[A-Za-z0-9-.]+\.blossom(:[0-9]+)?$/) {
+ $repl =~ s/\.([cq]\.[A-Za-z0-9-.]+\.blossom)((:[0-9]+)?)$/$2/;
+ $router = $1;
+ }
+ log_info(0, "transmitting router: [$router]");
+ $router .= "+$reverseproxy" if $reverseproxy;
+ print WRITER "$router\n";
+ close WRITER;
+
+ $repl = "Host: $repl\r\n";
+ log_info(1, " send: $repl");
+ print $repl;
+ } elsif(/^(GET|POST) /) {
+ my $line = $_;
+ my $type = $1;
+ if($line =~ /^$type \/http:\/\/([^\/]+)\//) {
+ $reverseproxy = $1;
+ $line =~ s/^$type \//$type /;
+ }
+ print $line;
+ log_info(0, " send: $line");
+ $post = 1 if $type eq "POST";
+ } elsif(/^\r$/) {
+ print;
+ last if $post;
+ } else {
+ print;
+ log_info(1, " send: $_");
+ }
+ }
+
+ if($post) {
+ my $data = "";
+
+ read($local_client, $data, $length) or die " error: $?";
+
+ print "$data";
+ log_info(1, " post: [$length]");
+ log_info(2, " data: $data");
+ }
+ }
+
+ kill('TERM', getppid()); # kill my twin cause we're done
+ }
+ exit; # whoever's still alive bites it
+ }
+ }
+}
+
+# helper function to produce a nice string in the form HOST:PORT
+sub peerinfo {
+ my $sock = shift;
+ my $hostinfo = undef;
+ my ($peeraddr, $peerport) = ("*", "*");
+
+ if($sock->peeraddr) {
+ $hostinfo = gethostbyaddr($sock->peeraddr);
+ if($hostinfo and $hostinfo->name) {
+ $peeraddr = $hostinfo->name;
+ } elsif($hostinfo and $sock->peerhost) {
+ $peeraddr = $sock->peerhost;
+ } elsif($sock->peeraddr and length($sock->peeraddr) == 4) {
+ $peeraddr = inet_ntoa($sock->peeraddr);
+ }
+ }
+ $peerport = $sock->peerport if $sock->peerport;
+ return sprintf "%s:%s", $peeraddr, $peerport;
+}
+
+# reset our $0, which on some systems make "ps" report
+# something interesting: the string we set $0 to!
+sub set_state { $0 = "$ME [@_]" }
+
+# helper function to call set_state
+sub accepting {
+ set_state("accepting proxy for " . ($REMOTE || $SERVICE));
+}
+
+# somebody just died. keep harvesting the dead until
+# we run out of them. check how long they ran.
+sub REAPER {
+ my $child;
+ my $start;
+ while (($child = waitpid(-1,WNOHANG)) > 0) {
+ if ($start = $Children{$child}) {
+ my $runtime = time() - $start;
+ my $line = sprintf "process $child completed in %dm %ss\n",
+ $runtime / 60,
+ $runtime % 60;
+ chomp $line;
+ log_info(1, $line);
+ delete $Children{$child};
+ } else {
+ log_info(1, "process $child exited [$?]");
+ }
+ }
+
+ # If I had to choose between System V and 4.2, I'd resign. --Peter Honeyman
+ $SIG{CHLD} = \&REAPER;
+};
+
+
Property changes on: blossom/trunk/edgeproxy
___________________________________________________________________
Name: svn:executable
+ *
Added: blossom/trunk/exit.py
===================================================================
--- blossom/trunk/exit.py (rev 0)
+++ blossom/trunk/exit.py 2008-05-19 04:23:06 UTC (rev 14663)
@@ -0,0 +1,567 @@
+#!/usr/bin/env python
+# $Id: exit.py,v 1.16 2007-03-18 11:45:55 goodell Exp $
+
+import pickle
+import os
+import re
+import string
+import struct
+import time
+
+from TorCtl import *
+
+# constants
+
+URL_SHOST = "afs.eecs.harvard.edu"
+URL_CSS = "http://%s/~goodell/style.css"; % URL_SHOST
+URL_FLAGS = "http://%s/~goodell/flags"; % URL_SHOST
+URL_ICONS = "http://%s/~goodell/icons"; % URL_SHOST
+URL_OSICONS = "http://%s/~goodell/os-icons"; % URL_SHOST
+
+ADDRESS = "140.247.60.133"
+DIR_HOST = "cassandra.eecs.harvard.edu:9030"
+TITLE = "Tor Network Status"
+
+CCODES = "/afs/eecs.harvard.edu/user/goodell/blossom/src/country-codes.txt"
+HOSTNAMES = "/var/www/tor/cached-hostnames"
+METADATA = "/var/www/tor/cached-metadata"
+ROUTERS = "/var/www/tor/cached-routers"
+PICKLE_DIR = "/var/www/tor/pickle"
+
+EXITLIST = "/usr/bin/exitlist"
+
+# globals
+
+addr = {}
+router_set = {}
+i_ports = {}
+m_country = {}
+
+ports = []
+x_bandwidth = []
+
+start_time = time.time()
+
+def generateURI(structure):
+ return "&".join(map(lambda x: "%s=%s" % (x, structure[x]), structure.keys()))
+
+prompts = {}
+uri = os.environ.get("REQUEST_URI")
+if uri and re.search(r'\\?', uri):
+ elts = re.sub(r'.*\?', "", uri).split("&")
+ for elt in elts:
+ m = re.match(r'^(\S+)=(\S+)$', elt)
+ if m:
+ prompts[m.group(1)] = m.group(2)
+
+redirect_line = {}
+links_line = []
+captions = []
+
+# sort (s)
+
+diddle = {}
+for k in prompts.keys():
+ diddle[k] = prompts[k]
+if diddle.has_key("s") and "bytes" == diddle["s"]:
+ del diddle["s"]
+ caption = "by country"
+else:
+ diddle["s"] = "bytes"
+ caption = "by bandwidth"
+redirect_line[caption] = generateURI(diddle)
+captions.append(caption)
+
+# restrict (r)
+
+diddle = {}
+for k in prompts.keys():
+ diddle[k] = prompts[k]
+if diddle.has_key("r"):
+ del diddle["r"]
+for r in ["all", "valid", "running", "fast"]:
+ if "all" != r:
+ diddle["r"] = r
+ redirect_line[r] = generateURI(diddle)
+ captions.append(r)
+
+# exit ports (e)
+
+diddle = {}
+for k in prompts.keys():
+ diddle[k] = prompts[k]
+if diddle.has_key("e"):
+ del diddle["e"]
+ caption = "metadata"
+else:
+ diddle["e"] = "80,143,443,6667"
+ caption = "exit ports"
+redirect_line[caption] = generateURI(diddle)
+captions.append(caption)
+
+for link in captions:
+ links_line.append("[<a href=\"?%s\">%s</a>]" % (redirect_line[link], link))
+
+links_line = " ".join(links_line)
+
+if prompts.has_key("e"):
+ input = prompts["e"].split(",")
+ for str in input:
+ if re.match(r'^[0-9]+$', str):
+ ports.append(str)
+ if len(ports) > 6:
+ ports = ports[:6]
+
+ for port in ports:
+ i_ports[port] = {}
+ h = os.popen("%s %s:%s < %s" % (
+ EXITLIST,
+ ADDRESS,
+ port,
+ ROUTERS
+ ))
+
+ while 1:
+ try:
+ line = h.readline()
+ except AttributeError, e:
+ break
+ except EOFError, e:
+ break
+ if not line:
+ break
+ i_ports[port][line[:-1]] = 1
+
+# unpickle
+
+h = open("%s/routers" % PICKLE_DIR, "r")
+rtr = pickle.load(h)
+h.close()
+h = open("%s/bandwidth" % PICKLE_DIR, "r")
+i_bandwidth = pickle.load(h)
+h.close()
+h = open("%s/nickname" % PICKLE_DIR, "r")
+i_nickname = pickle.load(h)
+h.close()
+h = open("%s/versions" % PICKLE_DIR, "r")
+versions = pickle.load(h)
+h.close()
+
+# parse cached hostnames file
+
+h = open(HOSTNAMES, "r")
+while 1:
+ try:
+ line = h.readline()
+ except AttributeError, e:
+ break
+ except EOFError, e:
+ break
+ if not line:
+ break
+ m = line.split(" ")
+ a = m[1]
+ if not addr.has_key(a):
+ addr[a] = {}
+ addr[a] = {
+ "host" : re.sub(r'\n', "", m[2]),
+ "cc" : "~~",
+ "netname" : a
+ }
+
+h.close()
+
+# parse country codes file
+
+h = open(CCODES, "r")
+
+while 1:
+ try:
+ line = h.readline()
+ except AttributeError, e:
+ break
+ except EOFError, e:
+ break
+ if not line:
+ break
+ m = re.match(r'^(\S+)\s+(\S+.*)$', line)
+ if m:
+ m_country[m.group(1)] = m.group(2)
+
+h.close()
+
+# parse cached metadata file
+
+h = open(METADATA, "r")
+while 1:
+ try:
+ line = h.readline()
+ except AttributeError, e:
+ break
+ except EOFError, e:
+ break
+ if not line:
+ break
+ m = re.match(r'^(\S+)\s+(\S+)\s+(\S+)$', line)
+ if m:
+ a = m.group(1)
+ c = m.group(2)
+ n = m.group(3)
+ if not addr.has_key(a):
+ addr[a] = {}
+ if m_country.has_key(c):
+ addr[a]["cc"] = c
+ else:
+ addr[a]["cc"] = "~~"
+ addr[a]["netname"] = n
+
+h.close()
+
+# organize routers into countries
+
+names = i_nickname.keys()
+names.sort(lambda x, y: cmp(x.lower(), y.lower()))
+
+for name in names:
+ for r in i_nickname[name]:
+ if not rtr.has_key(r):
+ continue
+ a = rtr[r]["address"]
+ if not addr.has_key(a):
+ addr[a] = {
+ "host" : a,
+ "cc" : "~~",
+ "netname" : a
+ }
+ c = addr[a]["cc"]
+ if not router_set.has_key(c):
+ router_set[c] = []
+ router_set[c].append(r)
+ if m_country.has_key(c):
+ rtr[r]["country"] = m_country[c]
+ else:
+ rtr[r]["country"] = "~~"
+
+# organize bandwidths for statistics
+
+bandwidths = i_bandwidth.keys()
+bandwidths.sort()
+
+for bandwidth in bandwidths:
+ for i in range(i_bandwidth[bandwidth]):
+ x_bandwidth.append(bandwidth)
+
+# create entries for individual routers
+
+for r in rtr.keys():
+ extra = ""
+ dirport = ""
+ a = rtr[r]["address"]
+ s_platform = "".join(rtr[r]["platform"].split(" "))
+
+ if not addr.has_key(a):
+ addr[a] = {
+ "host" : a,
+ "cc" : "~~",
+ "netname" : a
+ }
+ if not rtr[r].has_key("version"):
+ rtr[r]["version"] = ""
+ if not rtr[r].has_key("bandwidth"):
+ rtr[r]["bandwidth"] = 0
+ if rtr[r]["bwsymbol"] == "bx":
+ s_bandwidth = "Not Valid"
+ elif rtr[r]["bwsymbol"] == "wx":
+ s_bandwidth = "Hibernating"
+ elif rtr[r]["bwsymbol"] == "rx":
+ s_bandwidth = "Not Available"
+ else:
+ s_bandwidth = "%s kB/s" % int(rtr[r]["bandwidth"]/1000)
+ if rtr[r]["status"].has_key("Fast"):
+ if rtr[r]["bandwidth"] > x_bandwidth[int(3*len(x_bandwidth)/4)]:
+ rtr[r]["bwsymbol"] = "v3"
+ elif rtr[r]["bandwidth"] > x_bandwidth[int(len(x_bandwidth)/2)]:
+ rtr[r]["bwsymbol"] = "v2"
+ for property in ["Stable", "Guard", "Authority", "Disputed"]:
+ property_str = property
+ if rtr[r]["status"].has_key(property):
+ extra = "<acronym title=\"%s\"><img src=\"%s/%s.png\"></acronym>" \
+ % (property_str, URL_ICONS, property)
+
+ if rtr[r]["orport"] == 443:
+ c_orport = "c5"
+ else:
+ c_orport = "centered"
+
+ if rtr[r]["dirport"] == 80:
+ c_dirport = "c5"
+ else:
+ c_dirport = "centered"
+
+ if versions.has_key(rtr[r]["version"]):
+ c_obsolete = "cell"
+ else:
+ c_obsolete = "red"
+
+ if not rtr[r].has_key("bytes"):
+ rtr[r]["bytes"] = 0
+
+ if not rtr[r].has_key("uptime"):
+ rtr[r]["uptime"] = 0
+
+ if len(ports):
+ extended_content = ""
+ for port in ports:
+ if i_ports[port].has_key(a):
+ if rtr[r]["status"].has_key("Fast"):
+ td_class = "entry"
+ else:
+ td_class = "dimentry"
+ extended_content += "<td class=\"%s\"><tt>%s</tt></td>\n" \
+ % (td_class, port)
+ else:
+ extended_content += "<td><tt></tt></td>\n"
+ else:
+ b = int(rtr[r]["bytes"]/86400)
+ bytes = b
+ suffix = " "
+ if bytes >= 4096:
+ bytes = b>>10
+ suffix = "k"
+ if bytes >= 4096:
+ bytes = b>>20
+ suffix = "M"
+ bytes = "%d %s" % (bytes, suffix)
+
+ u = int(rtr[r]["uptime"]/3600)
+ if u < 96:
+ uptime = "%d h" % u
+ else:
+ uptime = "<b>%d d</b>" % int(u/24)
+
+ extended_content = """<td class="%s">
+ <tt>%s</tt>
+</td><td class="%s">
+ <tt>%s</tt>
+</td><td class="%s">
+ <tt>%s</tt>
+</td><td class="number">
+ <tt>%s</tt>
+</td><td class="number">
+ <tt>%s</tt>
+</td></tr>
+""" % (
+ c_orport,
+ rtr[r]["orport"],
+ c_dirport,
+ rtr[r]["dirport"],
+ c_obsolete,
+ rtr[r]["version"],
+ bytes,
+ uptime
+)
+ rtr[r]["entry"] = """<tr><td>
+ <acronym title="%s"><img src="%s/%s.gif"></acronym>
+</td><td>
+ <acronym title="%s"><img src="%s/%s.gif"></acronym>
+</td><td class="%s">
+ <tt> <a class="%s" href="http://%s/tor/server/d/%s";>%s</a></tt>
+</td><td class="%s">
+ <tt> %s [<a class="%s" href="/cgi-bin/whois.pl?q=%s">%s</a>] </tt>
+</td><td>
+ %s
+</td><td>
+ <acronym title="%s"><img src="%s/%s.png"></acronym>
+</td>%s
+""" % (
+ rtr[r]["country"],
+ URL_FLAGS,
+ addr[a]["cc"].lower(),
+ s_bandwidth,
+ URL_ICONS,
+ rtr[r]["bwsymbol"],
+ rtr[r]["tdclass"],
+ rtr[r]["aclass"],
+ DIR_HOST,
+ rtr[r]["deschash"],
+ rtr[r]["nickname"],
+ rtr[r]["tdclass"],
+ addr[a]["host"],
+ rtr[r]["aclass"],
+ rtr[r]["address"],
+ addr[a]["netname"],
+ extra,
+ rtr[r]["platform"],
+ URL_OSICONS,
+ s_platform,
+ extended_content
+)
+
+# apply restrictive filter
+
+for r in rtr.keys():
+ restrict_list = []
+ if prompts.has_key("r"):
+ p = string.lower(prompts["r"])
+ if p == "valid":
+ restrict_list = ["bx"]
+ elif p == "active":
+ restrict_list = ["bx", "rx"]
+ elif p == "running":
+ restrict_list = ["bx", "rx", "wx"]
+ elif p == "fast":
+ restrict_list = ["bx", "rx", "wx", "v0"]
+ if len(filter(lambda x: rtr[r]["bwsymbol"] == x, restrict_list)):
+ del rtr[r]
+
+# organize routers into categories
+
+if prompts.has_key("s") and prompts["s"] == "bytes":
+ router_set = {}
+ names = rtr.keys()
+ names.sort(lambda x, y: cmp(rtr[x]["bytes"], rtr[y]["bytes"]))
+ size = len(rtr.keys())
+ c = 0
+ for r in names:
+ if c > 3*size/4:
+ rtr[r]["category"] = "First Quartile"
+ elif c > 2*size/4:
+ rtr[r]["category"] = "Second Quartile"
+ elif c > size/4:
+ rtr[r]["category"] = "Third Quartile"
+ else:
+ rtr[r]["category"] = "Fourth Quartile"
+ if not router_set.has_key(rtr[r]["category"]):
+ router_set[rtr[r]["category"]] = []
+ router_set[rtr[r]["category"]].append(r)
+ c += 1
+ for k in router_set.keys():
+ router_set[k].reverse()
+else:
+ for r in rtr.keys():
+ rtr[r]["category"] = rtr[r]["country"]
+
+# organize the entries into a report
+
+if prompts.has_key("s") and prompts["s"] == "bytes":
+ categories = [
+ "First Quartile",
+ "Second Quartile",
+ "Third Quartile",
+ "Fourth Quartile"
+ ]
+else:
+ categories = router_set.keys()
+ categories.sort(lambda x, y: cmp(m_country[x], m_country[y]))
+ categories.append("~~")
+
+output = ""
+total = 0
+f = 1
+uf = 1
+
+for c in categories:
+ rows = ""
+ if uf and "~~" == c:
+ uf = 0
+ continue
+ if not (prompts.has_key("s") and prompts["s"] == "bytes") and not m_country.has_key(c):
+ continue
+ if prompts.has_key("s") and prompts["s"] == "bytes":
+ category_name = c
+ else:
+ category_name = m_country[c]
+ for r in router_set[c]:
+ rows += rtr[r]["entry"]
+ count = len(router_set[c])
+ if count:
+ if f:
+ f = 0
+ else:
+ output += """<tr><td>
+</td><td>
+</td><td>
+</td><td>
+</td><td>
+</td><td>
+</td>"""
+ if len(ports):
+ for port in ports:
+ output += """<td>
+ <tt> </tt>
+</td>"""
+ output += "</tr>\n"
+ else:
+ output += """<td>
+ <tt> </tt>
+</td><td>
+ <tt> </tt>
+</td><td>
+</td><td>
+ <tt> </tt>
+</td><td>
+ <tt> </tt>
+</td></tr>
+"""
+ output += """<tr><td class="heading" colspan=6>
+ <tt>%s (%s)</tt>
+</td>""" % (category_name, count)
+ if len(ports):
+ output += """<td class="headingcentered" colspan=%s>
+ <tt> Exit Ports </tt>
+</td></tr>
+%s
+""" % (len(ports), rows)
+ else:
+ output += """<td class="headingcentered">
+ <tt>OR</tt>
+</td><td class="headingcentered">
+ <tt>Dir</tt>
+</td><td class="headingcentered">
+ <tt>Version</tt>
+</td><td class="headingcentered">
+ <tt>B/s</tt>
+</td><td class="headingcentered">
+ <tt>Uptime</tt>
+</td></tr>
+%s
+""" % rows
+ total += count
+
+# output as HTML
+
+print """Content-type: text/html
+
+<!doctype html public "-//W3C//DTD HTML 4.01//EN"
+ "http://www.w3.org/TR/html4/strict.dtd";>
+<html>
+<head>
+<title>%s</title>
+<meta name="Author" content="Geoffrey Goodell">
+<meta http-equiv="Content-Type" content="text/html; charset=utf-8">
+<meta http-equiv="Content-Style-Type" content="text/css">
+<link rel="stylesheet" type="text/css" href="%s">
+</head>
+
+<body>
+
+<h2>%s</h2>
+
+<div class="section">
+<p><tt>%s</tt></p>
+
+<table>
+%s
+</table>
+
+<p><tt>total: %d</tt></p>
+
+<p><tt>transaction completed in %d seconds</tt></p>
+
+<p><tt><a href="http://%s/~goodell/blossom/src/exit.py";>source code</a></tt></p>
+
+</div>
+
+</body>
+""" % (TITLE, URL_CSS, TITLE, links_line, output, total, time.time() - start_time, URL_SHOST)
+
Property changes on: blossom/trunk/exit.py
___________________________________________________________________
Name: svn:executable
+ *
Added: blossom/trunk/exitlist
===================================================================
--- blossom/trunk/exitlist (rev 0)
+++ blossom/trunk/exitlist 2008-05-19 04:23:06 UTC (rev 14663)
@@ -0,0 +1,285 @@
+#!/usr/bin/python
+# Copyright 2005-2006 Nick Mathewson
+# See the LICENSE file in the Tor distribution for licensing information.
+
+# Requires Python 2.2 or later.
+
+"""
+ exitlist -- Given a Tor directory on stdin, lists the Tor servers
+ that accept connections to given addreses.
+
+ example usage (Tor 0.1.0.x and earlier):
+
+ python exitlist 18.244.0.188:80 < ~/.tor/cached-directory
+
+ example usage (Tor 0.1.1.10-alpha and later):
+
+ cat ~/.tor/cached-routers* | python exitlist 18.244.0.188:80
+
+ If you're using Tor 0.1.1.18-rc or later, you should look at
+ the "FetchUselessDescriptors" config option in the man page.
+
+ Note that this script won't give you a perfect list of IP addresses
+ that might connect to you using Tor, since some Tor servers might exit
+ from other addresses than the one they publish.
+
+"""
+
+#
+# Change this to True if you want more verbose output. By default, we
+# only print the IPs of the servers that accept any the listed
+# addresses, one per line.
+#
+VERBOSE = False
+
+#
+# Change this to True if you want to reverse the output, and list the
+# servers that accept *none* of the listed addresses.
+#
+INVERSE = False
+
+#
+# Change this list to contain all of the target services you are interested
+# in. It must contain one entry per line, each consisting of an IPv4 address,
+# a colon, and a port number. This default is only used if we don't learn
+# about any addresses from the command-line.
+#
+ADDRESSES_OF_INTEREST = """
+ 1.2.3.4:80
+"""
+
+
+#
+# YOU DO NOT NEED TO EDIT AFTER THIS POINT.
+#
+
+import sys
+import re
+import getopt
+import socket
+import struct
+
+assert sys.version_info >= (2,2)
+
+
+def maskIP(ip,mask):
+ return "".join([chr(ord(a) & ord(b)) for a,b in zip(ip,mask)])
+
+def maskFromLong(lng):
+ return struct.pack("!L", lng)
+
+def maskByBits(n):
+ return maskFromLong(0xffffffffl ^ ((1L<<(32-n))-1))
+
+class Pattern:
+ """
+ >>> import socket
+ >>> ip1 = socket.inet_aton("192.169.64.11")
+ >>> ip2 = socket.inet_aton("192.168.64.11")
+ >>> ip3 = socket.inet_aton("18.244.0.188")
+
+ >>> print Pattern.parse("18.244.0.188")
+ 18.244.0.188/255.255.255.255:1-65535
+ >>> print Pattern.parse("18.244.0.188/16:*")
+ 18.244.0.0/255.255.0.0:1-65535
+ >>> print Pattern.parse("18.244.0.188/2.2.2.2:80")
+ 2.0.0.0/2.2.2.2:80-80
+ >>> print Pattern.parse("192.168.0.1/255.255.00.0:22-25")
+ 192.168.0.0/255.255.0.0:22-25
+ >>> p1 = Pattern.parse("192.168.0.1/255.255.00.0:22-25")
+ >>> import socket
+ >>> p1.appliesTo(ip1, 22)
+ False
+ >>> p1.appliesTo(ip2, 22)
+ True
+ >>> p1.appliesTo(ip2, 25)
+ True
+ >>> p1.appliesTo(ip2, 26)
+ False
+ """
+ def __init__(self, ip, mask, portMin, portMax):
+ self.ip = maskIP(ip,mask)
+ self.mask = mask
+ self.portMin = portMin
+ self.portMax = portMax
+
+ def __str__(self):
+ return "%s/%s:%s-%s"%(socket.inet_ntoa(self.ip),
+ socket.inet_ntoa(self.mask),
+ self.portMin,
+ self.portMax)
+
+ def parse(s):
+ if ":" in s:
+ addrspec, portspec = s.split(":",1)
+ else:
+ addrspec, portspec = s, "*"
+
+ if addrspec == '*':
+ ip,mask = "\x00\x00\x00\x00","\x00\x00\x00\x00"
+ elif '/' not in addrspec:
+ ip = socket.inet_aton(addrspec)
+ mask = "\xff\xff\xff\xff"
+ else:
+ ip,mask = addrspec.split("/",1)
+ ip = socket.inet_aton(ip)
+ if "." in mask:
+ mask = socket.inet_aton(mask)
+ else:
+ mask = maskByBits(int(mask))
+
+ if portspec == '*':
+ portMin = 1
+ portMax = 65535
+ elif '-' not in portspec:
+ portMin = portMax = int(portspec)
+ else:
+ portMin, portMax = map(int,portspec.split("-",1))
+
+ return Pattern(ip,mask,portMin,portMax)
+
+ parse = staticmethod(parse)
+
+ def appliesTo(self, ip, port):
+ return ((maskIP(ip,self.mask) == self.ip) and
+ (self.portMin <= port <= self.portMax))
+
+class Policy:
+ """
+ >>> import socket
+ >>> ip1 = socket.inet_aton("192.169.64.11")
+ >>> ip2 = socket.inet_aton("192.168.64.11")
+ >>> ip3 = socket.inet_aton("18.244.0.188")
+
+ >>> pol = Policy.parseLines(["reject *:80","accept 18.244.0.188:*"])
+ >>> print str(pol).strip()
+ reject 0.0.0.0/0.0.0.0:80-80
+ accept 18.244.0.188/255.255.255.255:1-65535
+ >>> pol.accepts(ip1,80)
+ False
+ >>> pol.accepts(ip3,80)
+ False
+ >>> pol.accepts(ip3,81)
+ True
+ """
+
+ def __init__(self, lst):
+ self.lst = lst
+
+ def parseLines(lines):
+ r = []
+ for item in lines:
+ a,p=item.split(" ",1)
+ if a == 'accept':
+ a = True
+ elif a == 'reject':
+ a = False
+ else:
+ raise ValueError("Unrecognized action %r",a)
+ p = Pattern.parse(p)
+ r.append((p,a))
+ return Policy(r)
+
+ parseLines = staticmethod(parseLines)
+
+ def __str__(self):
+ r = []
+ for pat, accept in self.lst:
+ rule = accept and "accept" or "reject"
+ r.append("%s %s\n"%(rule,pat))
+ return "".join(r)
+
+ def accepts(self, ip, port):
+ for pattern,accept in self.lst:
+ if pattern.appliesTo(ip,port):
+ return accept
+ return True
+
+class Server:
+ def __init__(self, name, ip, policy):
+ self.name = name
+ self.ip = ip
+ self.policy = policy
+
+def uniq_sort(lst):
+ d = {}
+ for item in lst: d[item] = 1
+ lst = d.keys()
+ lst.sort()
+ return lst
+
+def run():
+ global VERBOSE
+ global INVERSE
+ global ADDRESSES_OF_INTEREST
+
+ if len(sys.argv) > 1:
+ try:
+ opts, pargs = getopt.getopt(sys.argv[1:], "vx")
+ except getopt.GetoptError, e:
+ print """
+usage: %s [-v] [-x] [host:port [host:port [...]]]
+ -v verbose output
+ -x invert results
+""" % sys.argv[0]
+ sys.exit(0)
+
+ for o, a in opts:
+ if o == "-v":
+ VERBOSE = True
+ if o == "-x":
+ INVERSE = True
+ if len(pargs):
+ ADDRESSES_OF_INTEREST = "\n".join(pargs)
+
+ servers = []
+ policy = []
+ name = ip = None
+ for line in sys.stdin.xreadlines():
+ if line.startswith('router '):
+ if name:
+ servers.append(Server(name, ip, Policy.parseLines(policy)))
+ _, name, ip, rest = line.split(" ", 3)
+ policy = []
+ elif line.startswith('accept ') or line.startswith('reject '):
+ policy.append(line.strip())
+
+ if name:
+ servers.append(Server(name, ip, Policy.parseLines(policy)))
+
+ targets = []
+ for line in ADDRESSES_OF_INTEREST.split("\n"):
+ line = line.strip()
+ if not line: continue
+ p = Pattern.parse(line)
+ targets.append((p.ip, p.portMin))
+
+ accepters, rejecters = [], []
+ for s in servers:
+ for ip,port in targets:
+ if s.policy.accepts(ip,port):
+ accepters.append(s)
+ break
+ else:
+ rejecters.append(s)
+
+ if INVERSE:
+ printlist = rejecters
+ else:
+ printlist = accepters
+
+ ents = []
+ if VERBOSE:
+ ents = uniq_sort([ "%s\t%s"%(s.ip,s.name) for s in printlist ])
+ else:
+ ents = uniq_sort([ s.ip for s in printlist ])
+ for e in ents:
+ print e
+
+def _test():
+ import doctest, exitparse
+ return doctest.testmod(exitparse)
+#_test()
+
+run()
+
Property changes on: blossom/trunk/exitlist
___________________________________________________________________
Name: svn:executable
+ *