[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [tor-bugs] #2687 [Torperf]: Update filter.R to parse Torperf's new .mergedata format
#2687: Update filter.R to parse Torperf's new .mergedata format
-------------------------+--------------------------------------------------
Reporter: karsten | Owner: karsten
Type: enhancement | Status: needs_review
Priority: major | Milestone:
Component: Torperf | Version:
Keywords: | Parent:
Points: 4 | Actualpoints:
-------------------------+--------------------------------------------------
Comment(by tomb):
I broke karsten's and my code down into it's barest essentials.
Even with everything but the actual parsing and output removed they are
far too slow.
I ran R's native profiling to find out what they were spending their time
on, and found it to be string manipulation and output.
I found no significant difference between aggregation of output in data
frame, vector, or immediate appending to the native output buffer. I went
through about a dozen possible implementations and found none of them to
be more than a small constant factor different in run time.
I profiled memory consumption and found that all the versions I
experimented with had modest O(n) memory consumption.
Tentative conclusion: R is ill suited to significant string manipulation
Tentative recommendation: Let R crunch numbers and stats, but do the
string manipulation in a different language. Why not move the string
manipulation into the programs that provide the .data and .mergedata?
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/2687#comment:18>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs