[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

Re: [tor-bugs] #21236 [Metrics/Metrics website]: Put a visualization of Tor Browser downloads and updates on the Metrics website

#21236: Put a visualization of Tor Browser downloads and updates on the Metrics
website
-------------------------------------+--------------------------------
 Reporter:  karsten                  |          Owner:  karsten
     Type:  enhancement              |         Status:  needs_revision
 Priority:  High                     |      Milestone:
Component:  Metrics/Metrics website  |        Version:
 Severity:  Normal                   |     Resolution:
 Keywords:                           |  Actual Points:
Parent ID:                           |         Points:
 Reviewer:                           |        Sponsor:
-------------------------------------+--------------------------------
Changes (by karsten):

 * status:  merge_ready => needs_revision


Comment:

 I just ran into this bug while importing logs on the Metrics server:

 {{{
      [java] 18:58:10.454 [main] DEBUG org.torproject.metrics.webstats.Main
 - Importing log file
 https://webstats.torproject.org/out/archeotrichon.torproject.org/dist.torproject
 .org-access.log-20151219.xz.
      [java] Exception in thread "main" java.lang.OutOfMemoryError: Java
 heap space
      [java]     at java.util.Arrays.copyOfRange(Arrays.java:2694)
      [java]     at java.lang.String.<init>(String.java:203)
      [java]     at
 java.io.BufferedReader.readLine(BufferedReader.java:349)
      [java]     at
 java.io.BufferedReader.readLine(BufferedReader.java:382)
      [java]     at
 org.torproject.metrics.webstats.Main.downloadLogFile(Main.java:219)
      [java]     at
 org.torproject.metrics.webstats.Main.importLogFiles(Main.java:175)
      [java]     at org.torproject.metrics.webstats.Main.main(Main.java:94)
      [java] Java Result: 1
 }}}

 That log file is 282K compressed and '''1.0G''' uncompressed.  Looks like
 we'll have to try harder not to keep the file in memory at once.  I'm too
 tired to look into this today, but I'll try to fix this tomorrow, in
 particular how we solved this in the [https://gitweb.torproject.org
 /metrics-tasks.git/tree/task-20008/src/Importer.java ad-hoc analysis of
 webstats] in #20008.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/21236#comment:16>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs