[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

Re: [tor-bugs] #27827 [Obfuscation/Snowflake]: Reproducibility issue of the snowflake osx64 build

To: undisclosed-recipients: ;
Subject: Re: [tor-bugs] #27827 [Obfuscation/Snowflake]: Reproducibility issue of the snowflake osx64 build
From: "Tor Bug Tracker & Wiki" <blackhole@xxxxxxxxxxxxxx>
Date: Sun, 23 Sep 2018 21:15:40 -0000
Auto-submitted: auto-generated
Delivered-to: archiver@xxxxxxxx
Delivery-date: Sun, 23 Sep 2018 17:15:53 -0400
In-reply-to: <045.6384ffcb1bdadc8d9c7ed47d69e9d761@torproject.org>
List-archive: <http://lists.torproject.org/pipermail/tor-bugs/>
List-help: <mailto:tor-bugs-request@lists.torproject.org?subject=help>
List-id: "auto: Tor bug tracker status mails" <tor-bugs.lists.torproject.org>
List-post: <mailto:tor-bugs@lists.torproject.org>
List-subscribe: <https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs>, <mailto:tor-bugs-request@lists.torproject.org?subject=subscribe>
List-unsubscribe: <https://lists.torproject.org/cgi-bin/mailman/options/tor-bugs>, <mailto:tor-bugs-request@lists.torproject.org?subject=unsubscribe>
References: <045.6384ffcb1bdadc8d9c7ed47d69e9d761@torproject.org>
Reply-to: no-reply@xxxxxxxxxxxxxx, tor-assistants@xxxxxxxxxxxxxx
Sender: "tor-bugs" <tor-bugs-bounces@xxxxxxxxxxxxxxxxxxxx>

#27827: Reproducibility issue of the snowflake osx64 build
-----------------------------------+------------------------
 Reporter:  boklm                  |          Owner:  (none)
     Type:  defect                 |         Status:  new
 Priority:  Medium                 |      Milestone:
Component:  Obfuscation/Snowflake  |        Version:
 Severity:  Normal                 |     Resolution:
 Keywords:                         |  Actual Points:
Parent ID:                         |         Points:
 Reviewer:                         |        Sponsor:
-----------------------------------+------------------------

Old description:

> The build of Snowflake for MacOS is often producing the same result, but
> not always.
>
> Arthur has been rebuilding Snowflake 8 times, with 4 different results:
> https://gist.github.com/arthuredelstein/73860df088c565ea0b2ca6eef586063a

New description:

 The build of Snowflake for MacOS is often producing the same result, but
 not always.

 Arthur has been rebuilding Snowflake 8 times, with 4 different results:
 https://gist.github.com/arthuredelstein/73860df088c565ea0b2ca6eef586063a
 {{{
 fish script:

 for x in (seq 8)
     rm out/snowflake/snowflake-6077141f4aff-osx-x86_64-3b578d.tar.gz
     ./rbm/rbm build snowflake  --target alpha --target torbrowser-osx-
 x86_64
     tar xvf out/snowflake/snowflake-6077141f4aff-osx-x86_64-3b578d.tar.gz
     echo (sha256sum ./Contents/MacOS/Tor/PluggableTransports/snowflake-
 client
 end

 Results:

 b060b42cfd0c8fb2781dbb0fd45d42804dbb414473fec0597d9c2fb7d6d12aa8
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 1ee0dd2a0b228988e22c663d62b696b23a6ac48dc742a57dfa8f854aa3992bc3
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 22557c38d913e478e480dd3581efc00019fe2989c4273d9207f1719c34b6e399
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 22557c38d913e478e480dd3581efc00019fe2989c4273d9207f1719c34b6e399
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 1ee0dd2a0b228988e22c663d62b696b23a6ac48dc742a57dfa8f854aa3992bc3
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 1ee0dd2a0b228988e22c663d62b696b23a6ac48dc742a57dfa8f854aa3992bc3
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 1ee0dd2a0b228988e22c663d62b696b23a6ac48dc742a57dfa8f854aa3992bc3
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 6d008bc7d29e8543608491b67d4b11da7bd6589741d9f52ac5fd50dd39d84f29
 ./Contents/MacOS/Tor/PluggableTransports/snowflake-client
 }}}

--

Comment (by dcf):

 The difference is different ordering of some symbols/path in the
 snowflake-client binary. Notice how (except for `cgo-gcc-prolog`), it's
 different orderings of three blocks, color coded below.

 Notice also that each block contains a `/tmp/go-buildXXXXXXXXX` string.
 These come from [https://gitweb.torproject.org/builders/tor-browser-
 build.git/tree/projects/snowflake/build?h=tbb-8.5a2-build3#n38 the
 snowflake build descriptor]: the `XXXXXXXXX` overwrites a random number.
 What I suspect is happening is, the go compiler is generating three
 distinct random `/tmp/go-buildXXXXXXXXX` paths and sorting the blocks on
 them; then the build descriptor overwrites them all.

 The overwriting is a workaround for the upstream Go bug
 [https://github.com/golang/go/issues/9206 #9206], which is now marked
 closed. The first thing I would try is upgrading to a newer Go and seeing
 if it just gets fixed.

 {{{#!th
 1ee0dd2a0b228988e22c663d62b696b23a6ac48dc742a57dfa8f854aa3992bc3
 }}}
 {{{#!th
 22557c38d913e478e480dd3581efc00019fe2989c4273d9207f1719c34b6e399
 }}}
 {{{#!th
 6d008bc7d29e8543608491b67d4b11da7bd6589741d9f52ac5fd50dd39d84f29
 }}}
 |-----------------------------------
 {{{#!td
 {{{#!html
 <small>
 <pre style="background-color: red">
 /tmp/go-build/net/_obj/
 _cgo_export.c
 /tmp/go-buildXXXXXXXXX/net/_obj/_cgo_export.o
 __cgo_9f1b05c52f96_Cfunc__Cmalloc
 cgo_resnew.cgo2.c
 /tmp/go-buildXXXXXXXXX/net/_obj/cgo_resnew.cgo2.o
 __cgo_9f1b05c52f96_C2func_getnameinfo
 <span style="background-color: gray">cgo-gcc-prolog</span>
 __cgo_9f1b05c52f96_Cfunc_getnameinfo
 cgo_unix.cgo2.c
 /tmp/go-buildXXXXXXXXX/net/_obj/cgo_unix.cgo2.o
 __cgo_9f1b05c52f96_C2func_getaddrinfo
 __cgo_9f1b05c52f96_Cfunc_free
 __cgo_9f1b05c52f96_Cfunc_freeaddrinfo
 __cgo_9f1b05c52f96_Cfunc_gai_strerror
 __cgo_9f1b05c52f96_Cfunc_getaddrinfo
 </pre><pre style="background-color: green">
 /tmp/go-build/crypto/x509/_obj/
 root_cgo_darwin.cgo2.c
 /tmp/go-buildXXXXXXXXX/crypto/x509/_obj/root_cgo_darwin.cgo2.o
 _FetchPEMRoots_MountainLion
 /var/tmp/dist/go/src/crypto/x509/root_cgo_darwin.go
 _useOldCode
 _FetchPEMRoots
 __cgo_62033c69288a_Cfunc_CFDataGetBytePtr
 __cgo_62033c69288a_Cfunc_CFDataGetLength
 __cgo_62033c69288a_Cfunc_CFRelease
 __cgo_62033c69288a_Cfunc_FetchPEMRoots
 </pre><pre style="background-color: lightblue">
 /var/tmp/dist/go/src/runtime/cgo/
 gcc_context.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_context.o
 __cgo_release_context
 gcc_darwin_amd64.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_darwin_amd64.o
 _x_cgo_init
 __cgo_sys_thread_start
 _threadentry
 _k1
 gcc_libinit.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_libinit.o
 _x_cgo_sys_thread_create
 __cgo_try_pthread_create
 __cgo_wait_runtime_init_done
 _x_cgo_notify_runtime_init_done
 _x_cgo_set_context_function
 __cgo_get_context_function
 _runtime_init_mu
 _runtime_init_cond
 _runtime_init_done
 _cgo_context_function
 gcc_setenv.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_setenv.o
 _x_cgo_setenv
 _x_cgo_unsetenv
 gcc_util.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_util.o
 _x_cgo_thread_start
 gcc_amd64.S
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_amd64.o
 _crosscall_amd64
 </pre>
 }}}
 }}}
 {{{#!td
 {{{#!html
 <small>
 <pre style="background-color: red">
 /tmp/go-build/net/_obj/
 _cgo_export.c
 /tmp/go-buildXXXXXXXXX/net/_obj/_cgo_export.o
 __cgo_9f1b05c52f96_Cfunc__Cmalloc
 cgo_resnew.cgo2.c
 /tmp/go-buildXXXXXXXXX/net/_obj/cgo_resnew.cgo2.o
 __cgo_9f1b05c52f96_C2func_getnameinfo
 <span style="background-color: gray">cgo-gcc-prolog</span>
 __cgo_9f1b05c52f96_Cfunc_getnameinfo
 cgo_unix.cgo2.c
 /tmp/go-buildXXXXXXXXX/net/_obj/cgo_unix.cgo2.o
 __cgo_9f1b05c52f96_C2func_getaddrinfo
 __cgo_9f1b05c52f96_Cfunc_free
 __cgo_9f1b05c52f96_Cfunc_freeaddrinfo
 __cgo_9f1b05c52f96_Cfunc_gai_strerror
 __cgo_9f1b05c52f96_Cfunc_getaddrinfo
 </pre><pre style="background-color: lightblue">
 /var/tmp/dist/go/src/runtime/cgo/
 gcc_context.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_context.o
 __cgo_release_context
 gcc_darwin_amd64.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_darwin_amd64.o
 _x_cgo_init
 __cgo_sys_thread_start
 _threadentry
 _k1
 gcc_libinit.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_libinit.o
 _x_cgo_sys_thread_create
 __cgo_try_pthread_create
 __cgo_wait_runtime_init_done
 _x_cgo_notify_runtime_init_done
 _x_cgo_set_context_function
 __cgo_get_context_function
 _runtime_init_mu
 _runtime_init_cond
 _runtime_init_done
 _cgo_context_function
 gcc_setenv.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_setenv.o
 _x_cgo_setenv
 _x_cgo_unsetenv
 gcc_util.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_util.o
 _x_cgo_thread_start
 gcc_amd64.S
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_amd64.o
 _crosscall_amd64
 </pre><pre style="background-color: green">
 /tmp/go-build/crypto/x509/_obj/
 root_cgo_darwin.cgo2.c
 /tmp/go-buildXXXXXXXXX/crypto/x509/_obj/root_cgo_darwin.cgo2.o
 _FetchPEMRoots_MountainLion
 /var/tmp/dist/go/src/crypto/x509/root_cgo_darwin.go
 _useOldCode
 _FetchPEMRoots
 __cgo_62033c69288a_Cfunc_CFDataGetBytePtr
 __cgo_62033c69288a_Cfunc_CFDataGetLength
 __cgo_62033c69288a_Cfunc_CFRelease
 __cgo_62033c69288a_Cfunc_FetchPEMRoots
 </pre>
 }}}
 }}}
 {{{#!td
 {{{#!html
 <small>
 <pre style="background-color: lightblue">
 /var/tmp/dist/go/src/runtime/cgo/
 gcc_context.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_context.o
 __cgo_release_context
 gcc_darwin_amd64.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_darwin_amd64.o
 _x_cgo_init
 __cgo_sys_thread_start
 _threadentry
 _k1
 gcc_libinit.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_libinit.o
 _x_cgo_sys_thread_create
 __cgo_try_pthread_create
 __cgo_wait_runtime_init_done
 _x_cgo_notify_runtime_init_done
 _x_cgo_set_context_function
 __cgo_get_context_function
 _runtime_init_mu
 _runtime_init_cond
 _runtime_init_done
 _cgo_context_function
 gcc_setenv.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_setenv.o
 _x_cgo_setenv
 _x_cgo_unsetenv
 gcc_util.c
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_util.o
 _x_cgo_thread_start
 gcc_amd64.S
 /tmp/go-buildXXXXXXXXX/runtime/cgo/_obj/gcc_amd64.o
 _crosscall_amd64
 </pre><pre style="background-color: green">
 /tmp/go-build/crypto/x509/_obj/
 root_cgo_darwin.cgo2.c
 /tmp/go-buildXXXXXXXXX/crypto/x509/_obj/root_cgo_darwin.cgo2.o
 _FetchPEMRoots_MountainLion
 /var/tmp/dist/go/src/crypto/x509/root_cgo_darwin.go
 _useOldCode
 _FetchPEMRoots
 __cgo_62033c69288a_Cfunc_CFDataGetBytePtr
 <span style="background-color: gray">cgo-gcc-prolog</span>
 __cgo_62033c69288a_Cfunc_CFDataGetLength
 __cgo_62033c69288a_Cfunc_CFRelease
 __cgo_62033c69288a_Cfunc_FetchPEMRoots
 </pre><pre style="background-color: red">
 /tmp/go-build/net/_obj/
 _cgo_export.c
 /tmp/go-buildXXXXXXXXX/net/_obj/_cgo_export.o
 __cgo_9f1b05c52f96_Cfunc__Cmalloc
 cgo_resnew.cgo2.c
 /tmp/go-buildXXXXXXXXX/net/_obj/cgo_resnew.cgo2.o
 __cgo_9f1b05c52f96_C2func_getnameinfo
 __cgo_9f1b05c52f96_Cfunc_getnameinfo
 cgo_unix.cgo2.c
 /tmp/go-buildXXXXXXXXX/net/_obj/cgo_unix.cgo2.o
 __cgo_9f1b05c52f96_C2func_getaddrinfo
 __cgo_9f1b05c52f96_Cfunc_free
 __cgo_9f1b05c52f96_Cfunc_freeaddrinfo
 __cgo_9f1b05c52f96_Cfunc_gai_strerror
 __cgo_9f1b05c52f96_Cfunc_getaddrinfo
 </pre>
 }}}
 }}}

--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/27827#comment:3>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online

_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs

References:
- [tor-bugs] #27827 [Obfuscation/Snowflake]: Reproducibility issue of the snowflake osx64 build
  - From: Tor Bug Tracker & Wiki

Prev by Author: Re: [tor-bugs] #27824 [Applications/Tor Browser]: TorBrowser or NoScript 10 prevents cookies even if cookie exceptions are present
Next by Author: Re: [tor-bugs] #25702 [Applications/Tor Browser]: Activity 1.1 Update Tor Browser icon to follow design guidelines.
Previous by thread: Re: [tor-bugs] #27827 [Obfuscation/Snowflake]: Reproducibility issue of the snowflake osx64 build
Next by thread: Re: [tor-bugs] #27827 [Obfuscation/Snowflake]: Reproducibility issue of the snowflake osx64 build
Index(es):
- Author
- Thread