Hi, Nicolas Vigier: > Hello, > > You can find at this URL a proposal to refactor the tor browser bundle > build process, using an other tool to replace gitian: > https://people.torproject.org/~boklm/automation/tor-automation-proposals.html#build-tool > (also added as attached file to this email) it seems to be much more than a proposal for refactoring the TBB build process as building packages of components is not relevant in the latter. So, I won't comment on the details but like to get consensus on the big picture first. Taking the main improvements you listed below as a starting point seems therefore fine to me: > The main improvements in this prototype from the current build process > are: > > - all components are built separately, and include in their output file > name the commit hash or version, architecture and OS (if architecture > dependant). This allows us to keep previous builds if the > commit/version/architecture/OS didn't change. So we can rebuild a > bundle very quickly when the browser didn't change. Yes, we start with #10120 (which I'd like to start working on in the next weeks). But avoiding to rebuild other parts of the bundle (torbutton etc.) should be easily doable as well (but keep the starting/updating/stopping-the-VM-overhead in mind). > - the gitian replacment has features to download tarballs and verify them > with sha256sum or gpg signature, so this can replace the fetch-inputs.sh > script. Yes, but we already have fetch-inputs.sh. So the advantage of burps seems not to be so big here. > - we can remove the linux/windows/macosx descriptors duplication, and > instead use template directives for the parts that differ between > those builds (it's still possible to use separate files if they differ > completly). Yes, that is good. Although I am not sure how much this buys us in a full-fledged gitian-like setup. And I guess the gitian people would be happy to take patches. :) > - we can define variables based on selected OS. This allows for instance > to build python 2.7 when building on Ubuntu Lucid, but avoid building > it on other distros which already provide python 2.7. Well, building Python already ourselves is actually a feature as we are no longer dependent on some Python package. In the very loooooooooong run the aim is to build everything ourselves. A better example might therefore be building Binutils which we only need for Windows. But that boils basically down to #10120 again (we might even be able to be so smart to build the platform dependent tools only if the user really wants to build for that particular platform: there is no need to build Binutils if the user only wants to build Linux bundles). > - we can define variables based on "targets" that are set on command > line. For instance in the prototype using "--target enable_pt" instructs > to include the portable transports (only pyptlib in this prototype) in > the bundle. The portable transports are supposed to get included into the stable TBB rather sooner than later. Thus, that feature is not needed either. > - we can easily switch from building in a VM to building locally True, but I am not sure why that is a feature compared to Gitian as we need the VM for creating reproducible builds. Thus, this one does not count here as an improvement IMO. > - build descriptors can depend on the result of another build descriptor. > This remove needs for scripts like mkbundle-*.sh. Good idea. And looking at https://github.com/bitcoin/bitcoin/tree/master/contrib/gitian-descriptors the Bitcoin people might be interested in this as one. > And I think those improvements should make it easier to rebuild a new > bundle automatically when any of the components of the bundle receives > a new commit, and then run tests on this bundle. I understand why the first improvement makes rebuilding the bundles easier. But why does that hold for the other features as well? To get the discussion properly started I think we should ask additionally why there has to be a new tool for building TBB. Why not improving Gitian? Is it broken beyond repair? Others using Gitian could benefit as well and it would save maintenance costs (due to creating yet another tool doing similar things *and* maintaining that one etc.). As far as I can see none of your improvements is so specific to your tool that they can't get included into Gitian. This point is especially worth considering as you don't want to get rid of Gitian's functionality entirely but only of Gitian for driving the build process if I understood that correctly. All those tricky things concerning VM creation/handling are kept (and improved :) ) "- creation and start/stop of the Ubuntu build VMs. We can keep the gitian scripts for that, and improve them later." So, from my current understanding I tend to think there should be a couple of bugs get filed against gitian-builder (and that are good bugs you point out, I think!). They should get fixed then and upstreamed. That said, maybe having the whole packaging in the same tool as well changes things, I don't know (it might be worth thinking about the additional complexity due to burps being a packaging tool, too: e.g. does the lsb_release/release + lsb_release/id combination not matter for TBBs). But that is probably a different discussion (or is it not?). Georg
Attachment:
signature.asc
Description: OpenPGP digital signature
_______________________________________________ tor-dev mailing list tor-dev@xxxxxxxxxxxxxxxxxxxx https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-dev