Re: [tor-relays] [SOLVED] published descriptor missing from consensus

[Scott Bennett <bennett@xxxxxxx>]

teor <teor2345@xxxxxxxxx> wrote:

>
> > On 8 Jun 2017, at 18:44, Scott Bennett <bennett@xxxxxxx> wrote:
> > 
> > Roger Dingledine <arma@xxxxxxx> wrote:
> > 
> >     Or at least partially solved, at any rate.  The problem is solved,
> > but the mystery remains.
> > 
> >> On Sun, Jun 04, 2017 at 07:14:06PM -0500, Scott Bennett wrote:
> >>>     Which versions are the Running votes coming from versus the non-Running?
> >> 
> >> You can see the votes at
> >> https://www.seul.org/~arma/moria1-v3-status-votes
> >> 
> >>>     I have a few commands in a crontab entry that extract relay IP addresses
> >>> from the most recently received consensus, sort them, and load them into a
> >>> table in pf.  They run every 15 minutes.  Anything coming from the addresses
> >>> in the table is immediately passed.  Anything not passed gets checked against
> >>> a much larger table of probers, attempted intruders, etc. and is blocked if it
> >>> matches a table entry.
> >> 
> >> Well heck. Sounds like we have a winner here.
> >> 
> >> Turn off your weird censorship infrastructure, confirm that things
> >> start working after a while, and now you have something to debug. :)
> >> 
> >     The misuse of the word "censorship" aside, I tried disabling pf and
> > restarting tor.  To my surprise, the authorities connected to my relay
> > successfully and distributed its information in subsequent consensuses!
> > I do not know why, which is the remaining mystery.  However, having to
> > turn off one's firewall in order to run tor is not a solution to the
> > problem.
>
> If your firewall is blocking connections that tor needs to make, turning
> off that rule is precisely the solution to the problem.

     As noted more than once previously, the pf rules *pass* all traffic
from relay addresses *first*, so that traffic has already gone on to tor
before the block list is applied.  As also noted previously, the rules had
not (and still have not) been changed in a long time.  They were working
just fine for years, and then suddenly they were not.  Now they are working
just fine again since tor got rebuilt, so they are not the problem.
>
> > What I think I know is that the mismatched openssl stub and library versions
> > somehow interact with pf and pre-0.3.0.7 tor authorities in some mysterious
> > manner that causes connections from those authorities to my relay to fail,
> > but connections from 0.3.0.7 authorities to work just fine.  Having the stub
> > version and the library version be identical apparently avoids the erroneous
> > interaction(s).
>
> It's possible. But to confirm, you'd need to provide the sections of
> the firewall log that show what happens when a directory authority
> tries to check the reachability of your relay.

     That stuff isn't logged because it would soon exhaust the available
disk space with garbage.
>
> (As an aside, I'd encourage you not to keep logs at this level on an
> ongoing basis, they would contain client IP addresses and connection
> times.)

     That's one reason; the other is the space and kernel overhead
requirements.
>
> Or perhaps your firewall collected enough state to want to block tor
> while it was running, and lost that state when you turned it off and
> on again.

     I don't understand how an application could directly interfere with
the packet filter's operation.  I've previously posted the rules.  As I
described, it appears to be a problem of openssl misbehaving, apparently
due to mismatched stub and library versions, though I don't understand
how openssl trouble could occur with pf enabled and not occur with pf
disabled.  In any case, rebuilding tor so that the stub and library
versions match has eliminated the problem.  tor now plays nicely with the
authorities, regardless of whether pf is enabled.
     Now, having said that, I should mention that tor does mess around
with /dev/bpf, at least during startup, which is one reason why tor has
to have root privileges during startup.  I've never looked into exactly
what it does with /dev/bpf, however, so I have no idea whether that could
be involved and caused to malfunction somehow by openssl problems.
>
> You have a complex enough system that there are probably more
> possibilities neither of us have thought of.

     Complex?  Compared to what?  You've already seen the handful of rules
involved, which includes some near duplication of ORPort rules to provide
the same protection from miscreants to the DirPort that is given to the
ORPort.  In a nutshell, the rules look at incoming SYN packets to see if
they come from tor relay addresses and immediately lets pass those that do.
(By doing this step first, relay traffic is protected from the next step,
even if the relay's address has gotten onto the block list, e.g., an exit
that has been abused by some cracker or wannabe-cracker.)  If they do not,
then they are checked against a very large block list and, if found in the
block list, are dropped with no RST.  (Yes, I know dropping without RST
violates the TCP standards, but crackers deserve no courtesies.  Slowing
them down is good.  They can just wait until they time out.)  Any SYN
packets destined to the ORPort or DirPort that survive at this point are
either from clients or from relays that have just started up and therefore
do not yet appear in the relays list used in the first step.  If a relay
is new or has been down long enough to have been omitted from at least one
consensus *and* is in the block list, then its SYNs will be dropped until
up to 15 minutes after the next consensus that lists it has been downloaded.
That is the only, and very temporary, situation in which the rules block
traffic from legitimate source addresses.  Authorities will rarely, and only
briefly (i.e., not much over a couple of hours at most, certainly not
*days*), ever be in this situation.  There should *never* be a time when
a majority of authorities is in this situation simultaneously.
     It's worth pointing out that, without pf's very efficient protection
(or that of some other packet filter), tor itself is stuck with the load of
handling useless inbound connection attempts.  pf takes that load off of
tor when it is caused by IPv4 addresses that are known sources of trouble.
>
> I'd keep an eye on things over the next few weeks.
>
     I think I will be keeping an eye on openssl updates to make sure that
tor gets rebuilt each time the openssl library gets replaced, now that I
know that openssl updates can subvert one of the advantages of dynamically
linked libraries. :-(


                                  Scott Bennett, Comm. ASMELG, CFIAG
**********************************************************************
* Internet:   bennett at sdf.org   *xor*   bennett at freeshell.org  *
*--------------------------------------------------------------------*
* "A well regulated and disciplined militia, is at all times a good  *
* objection to the introduction of that bane of all free governments *
* -- a standing army."                                               *
*    -- Gov. John Hancock, New York Journal, 28 January 1790         *
**********************************************************************
_______________________________________________
tor-relays mailing list
tor-relays@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-relays