[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [tor-bugs] #25161 [Metrics/CollecTor]: Fix another memory problem with the webstats bulk import
#25161: Fix another memory problem with the webstats bulk import
-------------------------------+------------------------------
Reporter: karsten | Owner: metrics-team
Type: defect | Status: new
Priority: Medium | Milestone:
Component: Metrics/CollecTor | Version:
Severity: Normal | Resolution:
Keywords: | Actual Points:
Parent ID: | Points:
Reviewer: | Sponsor:
-------------------------------+------------------------------
Comment (by iwakeh):
I didn't take a close look at the proposed code changes/additions, yet.
In #25100 I suggested to partition the import by date, b/c that gives the
reduction of heap usage and also is fine for the bulk import where we know
one-file-one-date is true.
> Or is this just the operation where we happen to run out of memory from
accumulating stuff over time, and where fixing this issue would just mean
that we're failing somewhere else, shortly after?
I think this is the case. We're also opposed to all other CollecTor
modules compressing before writing.
What heap setting is used, 8G? How many cores are available?
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/25161#comment:1>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs