[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [tor-bugs] #25815 [Metrics/Onionoo]: Speed up hourly updater performance
#25815: Speed up hourly updater performance
-----------------------------+--------------------------
Reporter: karsten | Owner: karsten
Type: enhancement | Status: accepted
Priority: Medium | Milestone:
Component: Metrics/Onionoo | Version:
Severity: Normal | Resolution:
Keywords: | Actual Points:
Parent ID: | Points:
Reviewer: | Sponsor:
-----------------------------+--------------------------
Comment (by karsten):
So, I ran some performance measurements here. In particular, I used a
recent backup from one of the public Onionoo hosts and ran an update. By
doing so I identified two possible areas of improvements:
1. While writing details documents, we spend almost 40% of CPU time on
(de-)serializing JSON documents. We might want to try out another JSON
library than Gson. A quick search says that Jackson might be a lot faster.
Maybe we can save 10% or 20% of the overall time here.
2. Also while writing details documents, we read almost all details
status files and (re-)write a corresponding details document file. That's
almost 1M files and takes roughly 55% of the overall CPU time, including
the 40% for JSON handling. If we can find a way to exclude files that
don't need to be updated, we might be able to save another 10% or 20% of
overall time. (The overall savings are related to what we do to JSON
handling.)
I'd like to keep this ticket for a while longer and dive deeper into these
two ideas. But I thought I'd share these first results to hear early if
they're a bad idea for some reason.
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/25815#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs