[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
[tor-bugs] #4673 [Metrics Website]: Partition the 67, 000, 000 row statusentry table in the metrics database
#4673: Partition the 67,000,000 row statusentry table in the metrics database
-----------------------------+----------------------------------------------
Reporter: karsten | Owner: karsten
Type: enhancement | Status: new
Priority: major | Milestone:
Component: Metrics Website | Version:
Keywords: | Parent:
Points: | Actualpoints:
-----------------------------+----------------------------------------------
Thomas Termin suggested splitting the huge statusentry table in the
metrics database into multiple tables to solve some of our metrics
website/database performance problems. Tim Wilde chimed in saying they're
already doing that for large tables down to the hour level.
There was some more discussion about splitting the whole table covering 3+
years of data into 36+ month tables and adding a new month table every
month. Another suggestion was to move old data into a history table of
some kind using a cron-job-like stored procedure. I later saw Tim explain
something about using a year table, month tables, etc. down to hour tables
and deciding in the application which table(s) to query. Basically, there
was some discussion whether to do the splitting and merging in the
database or in the application.
A good next step would be to look at the PostgreSQL documentation for
[http://www.postgresql.org/docs/8.4/static/ddl-partitioning.html
partitioning tables]. I'm also going to look more at the schema and Java
code to find the places which are affected by such a change.
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/4673>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs