[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [tor-bugs] #15844 [Onionoo]: Develop database schema to support Onionoo's search parameter efficiently
#15844: Develop database schema to support Onionoo's search parameter efficiently
-----------------------------+-----------------
Reporter: karsten | Owner:
Type: enhancement | Status: new
Priority: normal | Milestone:
Component: Onionoo | Version:
Resolution: | Keywords:
Actual Points: | Parent ID:
Points: |
-----------------------------+-----------------
Comment (by leeroy):
Thank you for all the clarifications. To start with:
1. Create an updated normal form with tables to ensure document coverage.
1. Create related SQL for document queries.
1. Deploy a test database, and a test Onionoo instance, using a
restricted data-set for comparison.
1. Benchmark them for similar queries.
1. Iteratively make adjustments to database to achieve desired quality of
service.
With updates and refinements occurring in this ticket along the way.
----
>Slow clients would show up in the next line, "Milliseconds to build
response", which includes reading documents from disk as well as writing
them to the response. (We could split up that line, but then we'd have
to buffer complete responses in memory, which might be bad.)
>Happy to add new statistics there, if they help us investigate potential
bottlenecks.
It might be useful to look at how much time is spent just building the
JSON part of the response. In particular, the data is serialized into JSON
using `GsonBuilder` in `ResponseBuilder` . That class appears to contain
non-trivial branching and logic. PostgreSQL can serialize the response
into JSON too. I heard it's shiny.
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/15844#comment:13>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs