[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [tor-bugs] #28866 [Core Tor/sbws]: ResultDump.queue.put() can hang if the queue is full
#28866: ResultDump.queue.put() can hang if the queue is full
---------------------------+-----------------------------------
Reporter: teor | Owner: (none)
Type: defect | Status: new
Priority: Medium | Milestone: sbws: 1.0.x-final
Component: Core Tor/sbws | Version: sbws: 1.0.2
Severity: Normal | Resolution:
Keywords: | Actual Points:
Parent ID: #28663 | Points:
Reviewer: | Sponsor:
---------------------------+-----------------------------------
Comment (by juga):
After looking more at this.
Replying to [ticket:28866 teor]:
> sbws calls `ResultDump.queue.put()` in blocking mode:
>
https://github.com/torproject/sbws/blob/ee64d76df54ceb3a3c9e1e2a797fd70d68bb0035/sbws/core/scanner.py#L303
put is not blocking because it happens in result_dump's thread
(result_dump launches its own thread before the the pool is instantiated)
>
> But multiprocessing callbacks need to return immediately:
>
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool.apply_async
>
> So sbws should call put() without blocking, or with a (very small)
timeout:
> https://docs.python.org/3/library/queue.html#queue.Queue.put
With put timeout it will always return Full unless the slot is inmediatly
available (https://docs.python.org/3/library/queue.html#queue.Queue.put)
It's also very unlikely that the queue will be full, since results happen
with a difference of seconds and worst case there're only 6000.
A way to solve this without a different thread, queue and lock in
result_putter, would be to use a deque, but i don't think it's necessary,
since i think the bug is in #28897.
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/28866#comment:2>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs