[tor-bugs] #20617 [Core Tor/Tor]: Write a unit test abstraction for tests that need to be repeated multiple times to prove failure

["Tor Bug Tracker & Wiki" <blackhole@xxxxxxxxxxxxxx>]

#20617: Write a unit test abstraction for tests that need to be repeated multiple
times to prove failure
------------------------------+------------------------------
     Reporter:  teor          |      Owner:
         Type:  enhancement   |     Status:  new
     Priority:  Medium        |  Milestone:  Tor: unspecified
    Component:  Core Tor/Tor  |    Version:
     Severity:  Normal        |   Keywords:  test
Actual Points:                |  Parent ID:
       Points:  2             |   Reviewer:
      Sponsor:                |
------------------------------+------------------------------
 Some unit tests succeed at random with a certain probability, so we need
 to repeat them a number of times until the failure rate is acceptable.

 Typically, we make sure the failure rate is less than the memory bit flip
 rate.

 But we could standardise this, by taking the following inputs:
 * test function with success/fail return value,
 * the probability of success regardless of whether the code actually
 works,
 * the desired overall probability of failure by chance (for example, the
 bit flip rate).

 The abstraction would then repeat the test enough times to make sure the
 failure rate is less than the bit flip rate.

 We might also want to check that the distribution of returned values is
 uniform, but I think that's a separate (and more complex) ticket.

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/20617>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs