Damian Johnson <atagar1@xxxxxxxxx> wrote on or-talk@:
> Hi, I've uploaded a new tarball to:
> http://www.atagar.com/transfer/tmp/arm_bsdTest3.tar.bz2
> http://www.atagar.com/transfer/tmp/arm_bsdTest3.tar.bz2.asc
Seems to work great on my FreeBSD system, but when trying it
on one of my Tor relays (currently running Debian GNU/Linux)
I had to unset LANG to get the connection resolver working:
su -m debian-tor -c 'LANG=; /root/arm.git/arm -i 127.0.0.1:9051 -e 1'
Probably arm itself should do that.
Although there weren't that many connections:
"(171 inbound, 93 outbound, 1 control)"
I had to disable the connection panel anyway, as arm kept
hogging the CPU.
> Besides a modified version of Febian's patch to autodetect FreeBSD
> jails it most notably includes...
>
> - A replacement for the connection test function (which was a pita in
> my humble opinion). The new script [1] provides the resolver runtimes,
> a check if all the resolvers match, and a better method of dumping the
> connection results. If you modify the bsd resolvers then this should
> provide a nice sanity check that it's working as expected.
I haven't looked into using it yet, but it sounds great.
> - I forgot to account for the dns resolution exits do on behalf of the
> clients. The resolvers need to include UDP connections so, on *nix,
> they're now:
> - netstat -np | grep "ESTABLISHED <pid>/<process>"
> - sockstat | egrep "<process>\s*<pid>.*ESTABLISHED"
> - lsof -nPi | egrep "^<process>\s*<pid>.*((UDP.*)|(\(ESTABLISHED\)))"
> - ss -nptu | grep "ESTAB.*\"<process>\",<pid>"
>
> I'm guessing, for the FreeBSD resolvers, that sockstats already works
> and procstat just needs the 'grep TCP' to be removed (or maybe
> replaced with 'egrep "(TCP|UDP)"'). Is that right?
I think you are.
> > The connection doesn't leave the system because its a socks
> > connection with both the source and the destination address
> > located on the same system.
>
> Hm. Sounds like basic client connections (ie, things like firefox
> connecting to tor via the SocksPort). However, I tried running TBB and
> arm didn't list any of those connections. This is what I'd expect
> since the connection resolution is only fetching tor connections. Am I
> missing something here?
No, you're right, those socks connections are client connections.
> Regardless, I made a couple changes to address issues that have been
> brought up (socks connections and listing external addresses for
> private ip range connections - see lines 332-334 and 363-364 in
> src/interface/connPanel.py [2]). But without a working repro case I
> can't promises that this'll do the trick.
Those socks connections now show up properly as client connections, thanks.
> > With ^ added to the pattern it seems to work
>
> Great, it's happy with that on Linux as well so I'm now using:
> lsof -nPi | egrep "^<process>\s*<pid>.*((UDP.*)|(\(ESTABLISHED\)))"
>
> and including it among FreeBSD resolvers as the last fallback.
With the '\s' that's still not going to work. Even on the
Debian GNU/Linux system the egrep doesn't understand the "\s".
After fixing that and switching the resolver to lsof, I get the
following exception (may be a bit messed up as I had to scrape
it out of the connection panel):
File "/usr/lib/python2.5/threading.py", line 486, in __bootstrap_inner
self.run()
File "/root/arm.git/src/util/connections.py", line 339, in run
connResults = getConnections(resolver, self.processName, self.processPid) File "/root/arm.git/src/util/connections.py", line 151, in getConnections)
local, foreign = comp[8].split("->")
IndexError: list index out of range
At least for the lsof 4.78 I'm using, the 8 needs to be a 7.
I attached a couple of patches for the problems I noticed, and an
improperly tested hack to show the external address between the
local and the foreign one, if the local and the external one differ.
Fabian
From fac5423e4ef6ecee0c85e38dc95b47931aa9c96a Mon Sep 17 00:00:00 2001
From: Fabian Keil <fk@xxxxxxxxxxxxx>
Date: Tue, 14 Dec 2010 23:24:01 +0100
Subject: [PATCH 1/8] Fix spelling in a comment
---
src/test.py | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/src/test.py b/src/test.py
index 94b1126..aad901c 100644
--- a/src/test.py
+++ b/src/test.py
@@ -98,7 +98,7 @@ while True:
elif userInput == "3":
uiTools.demoGlyphs()
- # Switching to a curses context and back repetedy seems to screw up the
+ # Switching to a curses context and back repeatedly seems to screw up the
# terminal. Just to be safe this ends the process after the demo.
break
else:
--
1.7.3.3
From 0b342d330b3430f7351e2b7b23ef2f6ac72a2204 Mon Sep 17 00:00:00 2001
From: Fabian Keil <fk@xxxxxxxxxxxxx>
Date: Wed, 15 Dec 2010 21:34:44 +0100
Subject: [PATCH 2/8] Factor ipAddressIsPrivate() out of reset()
---
src/interface/connPanel.py | 12 ++++++++----
1 files changed, 8 insertions(+), 4 deletions(-)
diff --git a/src/interface/connPanel.py b/src/interface/connPanel.py
index e9abe00..81f3f76 100644
--- a/src/interface/connPanel.py
+++ b/src/interface/connPanel.py
@@ -105,6 +105,13 @@ def getSortType(sortLabel):
if sortLabel == label: return type
raise ValueError(sortLabel)
+def ipAddressIsPrivate(Ip):
+ # TODO: range should restrict to the following address ranges:
+ # 10.*, 172.16.* - 172.31.*, 192.168.*
+ # being lazy right now - fix the 172.* range when rewriting
+ isPrivateIp = Ip.startswith("10.") or Ip.startswith("192.168.") or Ip.startswith("172.")
+ return isPrivateIp
+
class ConnPanel(TorCtl.PostEventListener, panel.Panel):
"""
Lists tor related connection data.
@@ -357,10 +364,7 @@ class ConnPanel(TorCtl.PostEventListener, panel.Panel):
# replace nat address with external version if available and the
# external address isn't a private IP
- # TODO: range should restrict to the following address ranges:
- # 10.*, 172.16.* - 172.31.*, 192.168.*
- # being lazy right now - fix the 172.* range when rewriting
- isPrivateIp = fIp.startswith("10.") or fIp.startswith("192.168.") or fIp.startswith("172.")
+ isPrivateIp = ipAddressIsPrivate(fIp)
if self.address and type != "control" and not isPrivateIp: lIp = self.address
try:
--
1.7.3.3
From b3a491898b677c58dc24f2fbdef18da1de0ce222 Mon Sep 17 00:00:00 2001
From: Fabian Keil <fk@xxxxxxxxxxxxx>
Date: Fri, 17 Dec 2010 00:09:06 +0100
Subject: [PATCH 3/8] Treat addresses starting with '127.' as private, too.
---
src/interface/connPanel.py | 4 ++--
1 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/src/interface/connPanel.py b/src/interface/connPanel.py
index 81f3f76..1703077 100644
--- a/src/interface/connPanel.py
+++ b/src/interface/connPanel.py
@@ -107,9 +107,9 @@ def getSortType(sortLabel):
def ipAddressIsPrivate(Ip):
# TODO: range should restrict to the following address ranges:
- # 10.*, 172.16.* - 172.31.*, 192.168.*
+ # 10.*, 172.16.* - 172.31.*, 192.168.*, 127.*
# being lazy right now - fix the 172.* range when rewriting
- isPrivateIp = Ip.startswith("10.") or Ip.startswith("192.168.") or Ip.startswith("172.")
+ isPrivateIp = Ip.startswith("10.") or Ip.startswith("192.168.") or Ip.startswith("172.") or Ip.startswith("127.")
return isPrivateIp
class ConnPanel(TorCtl.PostEventListener, panel.Panel):
--
1.7.3.3
From e0e1f903272558610f509407c953b6d92265ccac Mon Sep 17 00:00:00 2001
From: Fabian Keil <fk@xxxxxxxxxxxxx>
Date: Wed, 15 Dec 2010 22:20:02 +0100
Subject: [PATCH 4/8] Don't add the country code to private addresses. XXX: the duplicated code could be factored out.
---
src/interface/connPanel.py | 21 +++++++++++++++++----
1 files changed, 17 insertions(+), 4 deletions(-)
diff --git a/src/interface/connPanel.py b/src/interface/connPanel.py
index 1703077..78a8bab 100644
--- a/src/interface/connPanel.py
+++ b/src/interface/connPanel.py
@@ -550,7 +550,9 @@ class ConnPanel(TorCtl.PostEventListener, panel.Panel):
if self.listingType == LIST_IP:
# base data requires 73 characters
src = "%s:%s" % (entry[CONN_L_IP], entry[CONN_L_PORT])
- dst = "%s:%s %s" % (entry[CONN_F_IP], entry[CONN_F_PORT], "" if type == "control" else "(%s)" % entry[CONN_COUNTRY])
+ dst = "%s:%s" % (entry[CONN_F_IP], entry[CONN_F_PORT])
+ if not ipAddressIsPrivate(entry[CONN_F_IP]):
+ dst += " (%s)" % entry[CONN_COUNTRY]
if isPrivate: dst = "<scrubbed>"
@@ -583,7 +585,11 @@ class ConnPanel(TorCtl.PostEventListener, panel.Panel):
foreignHostnameSpace -= 22
if isPrivate: ipEntry = "<scrubbed>"
- else: ipEntry = "%s %s" % (entry[CONN_F_IP], "" if type == "control" else "(%s)" % entry[CONN_COUNTRY])
+ else:
+ ipEntry = "%s:%s" % (entry[CONN_F_IP], entry[CONN_F_PORT])
+ if ipAddressIsPrivate(entry[CONN_F_IP]):
+ ipEntry += " (%s)" % entry[CONN_COUNTRY]
+
etc += "%-20s " % ipEntry
if width > 134 + xOffset:
@@ -631,7 +637,10 @@ class ConnPanel(TorCtl.PostEventListener, panel.Panel):
if width > 125 + xOffset:
# shows ip/port/locale (column width: 28 characters)
if isPrivate: ipEntry = "<scrubbed>"
- else: ipEntry = "%s:%s %s" % (entry[CONN_F_IP], entry[CONN_F_PORT], "" if type == "control" else "(%s)" % entry[CONN_COUNTRY])
+ else:
+ ipEntry = "%s:%s" % (entry[CONN_F_IP], entry[CONN_F_PORT])
+ if ipAddressIsPrivate(entry[CONN_F_IP]):
+ ipEntry += " (%s)" % entry[CONN_COUNTRY]
etc += "%-26s " % ipEntry
else:
# base data uses whatever extra room's available (using minimun of 50 characters)
@@ -653,7 +662,11 @@ class ConnPanel(TorCtl.PostEventListener, panel.Panel):
foreignNicknameSpace -= 28
if isPrivate: ipEntry = "<scrubbed>"
- else: ipEntry = "%s:%s %s" % (entry[CONN_F_IP], entry[CONN_F_PORT], "" if type == "control" else "(%s)" % entry[CONN_COUNTRY])
+ else:
+ ipEntry = "%s:%s" % (entry[CONN_F_IP], entry[CONN_F_PORT])
+ if ipAddressIsPrivate(entry[CONN_F_IP]):
+ ipEntry += " (%s)" % entry[CONN_COUNTRY]
+
etc += "%-26s " % ipEntry
dst = ("%%-%is" % foreignNicknameSpace) % dst
--
1.7.3.3
From 7633ef1e9eb166c0c1c8517aa95802eeac6a4b8e Mon Sep 17 00:00:00 2001
From: Fabian Keil <fk@xxxxxxxxxxxxx>
Date: Fri, 17 Dec 2010 19:03:56 +0100
Subject: [PATCH 5/8] If the fIp is private, don't bother trying to get a countryCode
---
src/interface/connPanel.py | 20 ++++++++++++--------
1 files changed, 12 insertions(+), 8 deletions(-)
diff --git a/src/interface/connPanel.py b/src/interface/connPanel.py
index 78a8bab..5e1b0d5 100644
--- a/src/interface/connPanel.py
+++ b/src/interface/connPanel.py
@@ -367,14 +367,18 @@ class ConnPanel(TorCtl.PostEventListener, panel.Panel):
isPrivateIp = ipAddressIsPrivate(fIp)
if self.address and type != "control" and not isPrivateIp: lIp = self.address
- try:
- countryCodeQuery = "ip-to-country/%s" % fIp
- countryCode = self.conn.get_info(countryCodeQuery)[countryCodeQuery]
- except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
- countryCode = "??"
- if not self.providedGeoipWarning:
- log.log(log.WARN, "Tor geoip database is unavailable.")
- self.providedGeoipWarning = True
+ if ipAddressIsPrivate(fIp):
+ # Should not be shown
+ countryCode = "???"
+ else:
+ try:
+ countryCodeQuery = "ip-to-country/%s" % fIp
+ countryCode = self.conn.get_info(countryCodeQuery)[countryCodeQuery]
+ except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
+ countryCode = "??"
+ if not self.providedGeoipWarning:
+ log.log(log.WARN, "Tor geoip database is unavailable.")
+ self.providedGeoipWarning = True
if (fIp, fPort) in connTimes: connTime = connTimes[(fIp, fPort)]
else: connTime = time.time()
--
1.7.3.3
From 7e2ca00f19505859ec02fbfbff9f4edfb14bd00d Mon Sep 17 00:00:00 2001 From: Fabian Keil <fk@xxxxxxxxxxxxx> Date: Sat, 18 Dec 2010 14:45:09 +0100 Subject: [PATCH 6/8] Replace the '\s' in RUN_LSOF with ' ' to get it working. --- src/util/connections.py | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/src/util/connections.py b/src/util/connections.py index abec3f6..d154288 100644 --- a/src/util/connections.py +++ b/src/util/connections.py @@ -60,7 +60,7 @@ RUN_SS = "ss -nptu | grep \"ESTAB.*\\\"%s\\\",%s\"" # oddly, using the -p flag via: # lsof lsof -nPi -p <pid> | grep "^<process>.*(ESTABLISHED)" # is much slower (11-28% in tests I ran) -RUN_LSOF = "lsof -nPi | egrep \"^%s\\s*%s.*((UDP.*)|(\\(ESTABLISHED\\)))\"" +RUN_LSOF = "lsof -nPi | egrep \"^%s *%s.*((UDP.*)|(\\(ESTABLISHED\\)))\"" # output: # atagar tor 3475 tcp4 127.0.0.1:9051 127.0.0.1:38942 ESTABLISHED -- 1.7.3.3
From e420434b29a64887a396e016921d8408cbbc80f6 Mon Sep 17 00:00:00 2001
From: Fabian Keil <fk@xxxxxxxxxxxxx>
Date: Sat, 18 Dec 2010 15:09:06 +0100
Subject: [PATCH 7/8] Fix splitting of the CMD_LSOF results
---
src/util/connections.py | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/src/util/connections.py b/src/util/connections.py
index d154288..9ca32be 100644
--- a/src/util/connections.py
+++ b/src/util/connections.py
@@ -148,7 +148,7 @@ def getConnections(resolutionCmd, processName, processPid = ""):
localIp, localPort = comp[4].split(":")
foreignIp, foreignPort = comp[5].split(":")
elif resolutionCmd == CMD_LSOF:
- local, foreign = comp[8].split("->")
+ local, foreign = comp[7].split("->")
localIp, localPort = local.split(":")
foreignIp, foreignPort = foreign.split(":")
elif resolutionCmd == CMD_SOCKSTAT:
--
1.7.3.3
From 91e0bb1208a010e9ab1fd73aec456b48ce321b48 Mon Sep 17 00:00:00 2001
From: Fabian Keil <fk@xxxxxxxxxxxxx>
Date: Sat, 11 Dec 2010 17:56:21 +0100
Subject: [PATCH 8/8] Hack to show the nat address behind the local address instead of showing it instead of the local address.
---
src/interface/connPanel.py | 20 +++++++++++++++-----
1 files changed, 15 insertions(+), 5 deletions(-)
diff --git a/src/interface/connPanel.py b/src/interface/connPanel.py
index 5e1b0d5..aef5c75 100644
--- a/src/interface/connPanel.py
+++ b/src/interface/connPanel.py
@@ -362,11 +362,6 @@ class ConnPanel(TorCtl.PostEventListener, panel.Panel):
connectionCountTmp[1] += 1
if SCRUB_PRIVATE_DATA and fIp not in self.fingerprintMappings.keys(): isPrivate = isExitAllowed(fIp, fPort, self.exitPolicy, self.exitRejectPrivate)
- # replace nat address with external version if available and the
- # external address isn't a private IP
- isPrivateIp = ipAddressIsPrivate(fIp)
- if self.address and type != "control" and not isPrivateIp: lIp = self.address
-
if ipAddressIsPrivate(fIp):
# Should not be shown
countryCode = "???"
@@ -558,6 +553,17 @@ class ConnPanel(TorCtl.PostEventListener, panel.Panel):
if not ipAddressIsPrivate(entry[CONN_F_IP]):
dst += " (%s)" % entry[CONN_COUNTRY]
+ # Hack to include the external address if it differs from the local one
+ if entry[CONN_L_IP] != self.address:
+ if not ipAddressIsPrivate(entry[CONN_F_IP]):
+ if type == "inbound":
+ # XXX: untested
+ src = self.address + " <-- " + src
+ else:
+ src += " --> " + self.address
+ else:
+ src += " " + " " * (len(self.address) + 1)
+
if isPrivate: dst = "<scrubbed>"
src, dst = "%-21s" % src, "%-26s" % dst
@@ -691,6 +697,10 @@ class ConnPanel(TorCtl.PostEventListener, panel.Panel):
else:
ipStart = etc.find("256")
if ipStart > -1: etc = etc[:ipStart] + ("%%-%is" % len(etc[ipStart:])) % "UNKNOWN"
+
+ if entry[CONN_L_IP] != self.address:
+ # Make room for the previously added external address
+ etc = etc.strip()
padding = width - (len(src) + len(dst) + len(etc) + 27) - xOffset # padding needed to fill full line
lineEntry = "<%s>%s --> %s %s%s%5s (<b>%s</b>)%s</%s>" % (color, src, dst, etc, " " * padding, timeLabel, type.upper(), " " * (9 - len(type)), color)
--
1.7.3.3
Attachment:
signature.asc
Description: PGP signature