Re: [tor-bugs] #16296 [Onionoo]: Implement lock file in a more robust way

["Tor Bug Tracker & Wiki" <blackhole@xxxxxxxxxxxxxx>]

#16296: Implement lock file in a more robust way
-------------------------+--------------------------
     Reporter:  karsten  |      Owner:
         Type:  defect   |     Status:  needs_review
     Priority:  normal   |  Milestone:
    Component:  Onionoo  |    Version:
   Resolution:           |   Keywords:
Actual Points:           |  Parent ID:
       Points:           |
-------------------------+--------------------------

Comment (by leeroy):

 I think it's not something to do lightly. By running two Java process, one
 for server, one for updater, each having access to the same heap size, you
 leave each to the mercy of the operating system. Java's GC works best
 within the context of a single JVM. When you leave the periodic updater
 running, even after releasing resources internally, the updater GC doesn't
 kick in until the OS says !`I'm low on memory`. Which means the updater
 will continue to occupy the allocated heap space. By running a cron job
 that memory is always freed at the end of the update. It should be
 flexible enough for operator choice and it should be fault tolerant. Just
 in case an operator decides to use some alternative automatic run
 mechanism, or accidentally runs more than one.

 A lock during an update means the developer of Onionoo went to the effort
 making corruption of data stores unlikely. A lock should be created in the
 updater instance, not the one-time version, or the periodic. The updater
 should itself be updated to spawn threads if needed for other types of
 data imports, and ideally, should allow multiple locks for different data
 types (having sub-process to handle those imports). A lock could then be
 created for recent imports, and archives for mm-yyyy, where a signal is
 passed to indicate the import is done and ready for merging into request-
 able documents.

 I'll take a look at the other ticket later today and consider improvements
 further. The topic of better updater (and archive import in general) has
 been a topic on my mind recently (while evaluating if a db is solving the
 real problems for Onionoo as deployed).

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/16296#comment:4>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs