Hi Nicolas, some remarks are below. Nicolas Vigier: > In order to help me doing that, I'm very interested to receive from > developers of any tor components : > > - a description or ticket number of bugs that you wish could have been > detected earlier with automated tests https://trac.torproject.org/projects/tor/ticket/8143 comes to mind here. > - any wish or specific needs that you may have You write: "Tor Browser includes some patches to make the build reproducible. We could have a test that check the reproducibility of the build by building the browser twice." While this is indeed a good idea, it won't be enough as we had bugs in the past that were only visible when builds on different machines got compared. So, what I'd like to have (in addition to running browser builds twice? on different machines?) are tests that cover specific bugs we avoided (see: the "Remaining Build Reproducibility Issues" in Mike's blog post covering the technical details of the Gitian build) or tracked down. See: https://trac.torproject.org/projects/tor/ticket/10159 for an example for the latter. (There, one could write a test that automates the creation of the browser.manifest which would eventually (i.e. if run a couple of times) show whether this bug still exists or whether not).) > - anything else that you think might be useful for me to know You write: "We can produce some packages for Tor Browser, to make testing of the browser easier." In which regard is it easier to test the browser if you have packages? And how should that look like outside of the Tor Browser Bundle? I fear we create extra bugs if we move the browser outside the environment it will be used in (i.e. the TBB) just for testing purposes. Even if we won't create extra bugs this way we might miss some. To be clear, running the tests you call "Usablility tests" and "Reproducible build test" outside the bundle, seems fine to me, while my concerns apply to the fingerprinting and privacy tests. The test helpers are a good idea. You might want to rename "Cookies tool" to something like "Identifier tool" matching the Tor Browser specification more closely (especially as you actually mean "Identifier tool" as "Later versions could be extended to also use other techniques for storing informations in the browser" indicates). Georg
Attachment:
signature.asc
Description: OpenPGP digital signature
_______________________________________________ tor-dev mailing list tor-dev@xxxxxxxxxxxxxxxxxxxx https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev