Re: [tor-bugs] #16401 [Onionoo]: Fail-fast if file access permissions for out and status prevent an update

["Tor Bug Tracker & Wiki" <blackhole@xxxxxxxxxxxxxx>]

#16401: Fail-fast if file access permissions for out and status prevent an update
-----------------------------+-----------------
     Reporter:  leeroy       |      Owner:
         Type:  enhancement  |     Status:  new
     Priority:  normal       |  Milestone:
    Component:  Onionoo      |    Version:
   Resolution:               |   Keywords:
Actual Points:               |  Parent ID:
       Points:               |
-----------------------------+-----------------

Comment (by karsten):

 Replying to [comment:2 leeroy]:
 > Sure I'll look into it. This happened when I tried to transfer a backup
 to a new instance of Onionoo running branch-task-13600. I just forgot to
 recursively change owner on out and status and found the server flooding
 the logs and rotating over dozens of log files.
 >
 > I pretty much do the same thing for backups. Because I've been testing
 multiple server versions I usually try to delay importing the latest
 recent data. Wait until I'm between updates, tar.bz2 the folders, then
 transfer them elsewhere. I could probably make my life easier by doing all
 this as the onionoo user and using a location which they can access.

 Sounds like a special case, but if it's easy to avoid flooding the logs by
 making sure that `out/` either does not exist or is a directory, let's do
 it.

 > __On a related issue:__ By my estimation, each month adds ~460MB and
 ~64K uncompressed files (in the past 1.5 years). So a fully deployed
 server can be expected to have to backup ~40GB. A lot of small files too,
 so lots of IO. Should Onionoo include functionality for backup/restore?
 This might work better with an hourly updater + archives, and could be
 done using java.util.zip or apache.commons.compress.

 I'm yet unsure how Onionoo would include functionality for backup/restore.
 Want to create a new ticket for that and explain in more detail?

 > __On another related issue__: Should I make a ticket for having the
 server stop trying to process the update if a certain number of errors
 occur? If the server keeps trying to update this might make a (hardware)
 failure worse. Maybe the server should stop trying to update if
 encountering many errors. The server admin will get a notice somehow (if
 they configured it right) so they probably have other things to worry
 about at this point.

 Maybe there's an easy way to rate-limit log statements?  But yes, creating
 a new ticket for that might be good.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/16401#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs