Re: [tor-bugs] #5232 [BridgeDB]: Import bridges into BridgeDB in a separate thread and database transaction

["Tor Bug Tracker & Wiki" <torproject-admin@xxxxxxxxxxxxxx>]

#5232: Import bridges into BridgeDB in a separate thread and database transaction
----------------------+-----------------------------------------------------
 Reporter:  karsten   |          Owner:  aagbsn
     Type:  defect    |         Status:  new   
 Priority:  major     |      Milestone:        
Component:  BridgeDB  |        Version:        
 Keywords:            |         Parent:  #4499 
   Points:            |   Actualpoints:        
----------------------+-----------------------------------------------------

Comment(by aagbsn):

 Replying to [comment:2 karsten]:
 > If you need to get a lock to swap the data structures and commit the
 transaction, does that mean readers also need to get a lock to get the
 data structure they're supposed to use?  Also, you may want to commit the
 transaction before swapping the data structures, as the commit may fail.
 >
 good catch, thanks. Yes, readers will need to be multithreading-aware.

 > This change sounds plausible to fix the problem, but it's also kinda
 invasive.  Are there things we could refactor before implementing this
 change, e.g., create a clean interface how distributors access the bridge
 copies in memory and in the database?  Ideally, the email distributor
 should not touch the database directly, and distributors shouldn't hold
 any references that we need to worry about when updating bridges.  I'm
 worried that this complexity is going to bite us.
 >

 the Email distributor holds a reference to the database as part of the
 email rate-limiting support. BridgeDB temporarily stores hashes of email
 addresses caught aggressively requesting bridges. These emails are stored
 in a separate table, however, sqlite may lock the entire database which
 could block if the transaction takes a long time to commit. Further
 investigation needed.

 If we think re-implementing the rate limiting support to avoid having a
 reference to the database we'll need to make sure state persists through a
 reload (HUP) of BridgeDB.

 And any multithreaded
 > Speaking of invasive changes: do we have good ways to test BridgeDB?  If
 not, is it a good time to add a bunch of unit/integration tests before
 starting to code on this change?
 >

 BridgeDB actually has a number of tests (Tests.py), and I have been adding
 new tests. The email distributor needs better test coverage.

 > We should also ask nickm for his opinion.  I don't know the BridgeDB
 code well enough to brainstorm about code changes like this.  I cc'ed him
 in this ticket.

-- 
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/5232#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs