[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

[tor-commits] [Git][tpo/applications/tor-browser-build][main] 3 commits: Bug 41444: Fix android artifact generation

Title: GitLab

brizental pushed to branch main at The Tor Project / Applications / tor-browser-build

Commits:

  • 83acc092
    by Beatriz Rizental at 2025-06-18T18:36:17+02:00 
    Bug 41444: Fix android artifact generation
  • dde97fb2
    by Beatriz Rizental at 2025-06-18T18:36:19+02:00 
    Bug 41444: Fix macos and windows artifact generation

There were multiple issues.

For MacOS there were missing tools, namely hfsplus and libdmg,
the location of some bundle files were different than expected
and teh generated artifacts were not the same as the ones in
Linux, so we were copying the wrong things. Finally, we were not
taking into account universal builds when copying artifacts.

For Windows we were just not copying the right artifacts.
  • d5e2db25
    by Beatriz Rizental at 2025-06-19T14:16:37+02:00 
    Bug 41444: Put geckoview build artifacts in separate folders

Otherwise different architecture artifacts override each other.

Also stop compressing the final folder. It's not worth it, the size
difference is minimal.

27 changed files:

Changes:

  • projects/browser/build
    ... ... @@ -217,7 +217,7 @@ tar -C "${TB_STAGE_DIR}" -xf [% c('input_files_by_name/firefox') %]/browser.tar.
    217 217 
        END; %]
    218 218
    219 219 
     [% IF c("var/macos") %]
    220 
    -  tar -C "Bundle-Data/[% c('var/ProjectName') %].dmg" -c . | tar -C "$TB_STAGE_DIR" -x
    220 
    +  tar -C "$rootdir/dmg-root/[% c('var/ProjectName') %].dmg" -c . | tar -C "$TB_STAGE_DIR" -x
    221 221 
       pushd "$TB_STAGE_DIR"
    222 222 
       cp [% c('var/channel') %].DS_Store .DS_Store
    223 223 
       rm *.DS_Store
    ... ... @@ -429,7 +429,14 @@ cd $distdir
    429 429 
       popd
    430 430 
     [% END %]
    431 431
    432 
    -[% IF c("var/nightly") && c("var/linux") -%]
    433 
    -  mkdir -p "$OUTDIR"/artifacts
    434 
    -  cp -r $rootdir/[% c('input_files_by_name/firefox') %]/artifacts "$OUTDIR"
    432 
    +[% IF c("var/nightly") -%]
    433 
    +  [% IF c("var/macos_universal") -%]
    434 
    +    mkdir -p "$OUTDIR"/artifacts/aarch64
    435 
    +    cp -a $rootdir/[% c('input_files_by_name/firefox-aarch64') %]/artifacts/. "$OUTDIR/artifacts/aarch64"
    436 
    +    mkdir -p "$OUTDIR"/artifacts/x86_64
    437 
    +    cp -a $rootdir/[% c('input_files_by_name/firefox') %]/artifacts/. "$OUTDIR/artifacts/x86_64"
    438 
    +  [% ELSE -%]
    439 
    +    mkdir -p "$OUTDIR"/artifacts
    440 
    +    cp -a $rootdir/[% c('input_files_by_name/firefox') %]/artifacts "$OUTDIR"
    441 
    +  [% END -%]
    435 442 
     [% END %]

  • projects/browser/build.android
    ... ... @@ -2,8 +2,6 @@
    2 2 
     [% c("var/set_default_env") -%]
    3 3 
     [% pc(c('var/compiler'), 'var/setup', { compiler_tarfile => c('input_files_by_name/' _ c('var/compiler')) }) %]
    4 4
    5 
    -tar -xf $rootdir/[% c('input_files_by_name/fenix') %]
    6 
    -
    7 5 
     # Bundle our extension(s).
    8 6 
     #
    9 7 
     # NoScript will be copied over to the profile folder
    ... ... @@ -97,20 +95,22 @@ function generate_apk {
    97 95 
       rm $aligned_apk
    98 96 
     }
    99 97
    98 
    +geckoviewdir="$rootdir/[% c('input_files_by_name/fenix') %]/geckoview"
    99 
    +
    100 100 
     # Generate the QA APK and unsign patch.
    101 101 
     qa_apk=[% dest_dir %]/[% c('filename') %]/[% c("var/project-name") %]-qa-[% c("var/osname") %]-[% c("version") %].apk
    102 102 
     qa_unsign_patch="[% dest_dir %]/[% c('filename') %]/[% c('var/project-name') %]-qa-unsign-[% c('var/osname') %]-[% c('version') %].bspatch"
    103 
    -apk=$rootdir/geckoview/*-[% c("var/abi") %]-*.apk
    103 
    +apk=$geckoviewdir/*-[% c("var/abi") %]-*.apk
    104 104 
     generate_apk $qa_apk $apk $qa_unsign_patch
    105 105
    106 106 
     # Generate the non-optimized APK used for testing and its unsign patch.
    107 107 
     noopt_apk=[% dest_dir %]/[% c('filename') %]/[% c("var/project-name") %]-noopt-[% c("var/osname") %]-[% c("version") %].apk
    108 
    -noopt_final_apk=$rootdir/geckoview/tests/*-[% c("var/abi") %]-*.apk
    108 
    +noopt_final_apk=$geckoviewdir/tests/*-[% c("var/abi") %]-*.apk
    109 109 
     generate_apk $noopt_apk $noopt_final_apk
    110 110
    111 111 
     # Copy and sign the Android test APK.
    112 112 
     test_apk=[% dest_dir %]/[% c('filename') %]/[% c("var/project-name") %]-[% c("version") %]-androidTest.apk
    113 
    -cp $rootdir/geckoview/tests/*-androidTest.apk $test_apk
    113 
    +cp $geckoviewdir/tests/*-androidTest.apk $test_apk
    114 114 
     apksigner sign --verbose --min-sdk-version [% c("var/android_min_api") %] --ks $rootdir/android-qa.keystore --out $test_apk --in $test_apk --ks-key-alias androidqakey --key-pass pass:android --ks-pass pass:android
    115 115
    116 116 
     [%IF c("var/tor-browser") -%]
    ... ... @@ -122,5 +122,5 @@ apksigner sign --verbose --min-sdk-version [% c("var/android_min_api") %] --ks $
    122 122
    123 123 
     [% IF c("var/nightly") -%]
    124 124 
       mkdir -p "[% dest_dir %]/[% c('filename') %]/artifacts"
    125 
    -  cp -r $rootdir/geckoview/artifacts [% dest_dir %]/[% c('filename') %]
    125 
    +  cp -r $rootdir/[% c('input_files_by_name/fenix') %]/artifacts [% dest_dir %]/[% c('filename') %]
    126 126 
     [% END %]

  • projects/browser/config
    ... ... @@ -111,6 +111,8 @@ input_files:
    111 111 
         enable: '[% ! c("var/android") %]'
    112 112 
       - filename: Bundle-Data
    113 113 
         enable: '[% ! c("var/android") %]'
    114 
    +  - filename: dmg-root
    115 
    +    enable: '[% ! c("var/android") %]'
    114 116 
       - URL: https://addons.mozilla.org/firefox/downloads/file/4411102/noscript-12.1.1.xpi
    115 117 
         name: noscript
    116 118 
         sha256sum: f9639e63ffcfc352036de00e4ff6694bb0ca65a0bb8fbd103bd08f32dc1ff31a

  • projects/browser/Bundle-Data/BaseBrowser.dmg/.VolumeIcon.icns → projects/common/dmg-root/BaseBrowser.dmg/.VolumeIcon.icns
    No preview for this file type
  • projects/browser/Bundle-Data/BaseBrowser.dmg/.background/background.tiff → projects/common/dmg-root/BaseBrowser.dmg/.background/background.tiff
    No preview for this file type
  • projects/browser/Bundle-Data/BaseBrowser.dmg/alpha.DS_Store → projects/common/dmg-root/BaseBrowser.dmg/alpha.DS_Store
    No preview for this file type
  • projects/browser/Bundle-Data/BaseBrowser.dmg/nightly.DS_Store → projects/common/dmg-root/BaseBrowser.dmg/nightly.DS_Store
    No preview for this file type
  • projects/browser/Bundle-Data/BaseBrowser.dmg/release.DS_Store → projects/common/dmg-root/BaseBrowser.dmg/release.DS_Store
    No preview for this file type
  • projects/browser/Bundle-Data/MullvadBrowser.dmg/.VolumeIcon.icns → projects/common/dmg-root/MullvadBrowser.dmg/.VolumeIcon.icns
    No preview for this file type
  • projects/browser/Bundle-Data/MullvadBrowser.dmg/.background/background.png → projects/common/dmg-root/MullvadBrowser.dmg/.background/background.png
    No preview for this file type
  • projects/browser/Bundle-Data/MullvadBrowser.dmg/alpha.DS_Store → projects/common/dmg-root/MullvadBrowser.dmg/alpha.DS_Store
    No preview for this file type
  • projects/browser/Bundle-Data/MullvadBrowser.dmg/nightly.DS_Store → projects/common/dmg-root/MullvadBrowser.dmg/nightly.DS_Store
    No preview for this file type
  • projects/browser/Bundle-Data/MullvadBrowser.dmg/release.DS_Store → projects/common/dmg-root/MullvadBrowser.dmg/release.DS_Store
    No preview for this file type
  • projects/browser/Bundle-Data/TorBrowser.dmg/.VolumeIcon.icns → projects/common/dmg-root/TorBrowser.dmg/.VolumeIcon.icns
    No preview for this file type
  • projects/browser/Bundle-Data/TorBrowser.dmg/.background/dmg-window_2x.png → projects/common/dmg-root/TorBrowser.dmg/.background/dmg-window_2x.png
    No preview for this file type
  • projects/browser/Bundle-Data/TorBrowser.dmg/alpha.DS_Store → projects/common/dmg-root/TorBrowser.dmg/alpha.DS_Store
    No preview for this file type
  • projects/browser/Bundle-Data/TorBrowser.dmg/nightly.DS_Store → projects/common/dmg-root/TorBrowser.dmg/nightly.DS_Store
    No preview for this file type
  • projects/browser/Bundle-Data/TorBrowser.dmg/release.DS_Store → projects/common/dmg-root/TorBrowser.dmg/release.DS_Store
    No preview for this file type
  • projects/firefox/build
    ... ... @@ -49,6 +49,11 @@ export PATH="/var/tmp/dist/rust/bin:/var/tmp/dist/cbindgen:/var/tmp/dist/node/bi
    49 49 
       [% END -%]
    50 50 
     [% END -%]
    51 51
    52 
    +[% IF c("var/macos") && c("var/nightly") %]
    53 
    +  tar -C /var/tmp/dist -xf $rootdir/[% c('input_files_by_name/hfsplus-tools') %]
    54 
    +  tar -C /var/tmp/dist -xf $rootdir/[% c('input_files_by_name/libdmg') %]
    55 
    +[% END %]
    56 
    +
    52 57 
     [% IF c("var/rlbox") -%]
    53 58 
       tar -C /var/tmp/dist -xf [% c('input_files_by_name/wasi-sysroot') %]
    54 59 
       export WASI_SYSROOT=/var/tmp/dist/wasi-sysroot
    ... ... @@ -128,16 +133,36 @@ echo "Starting ./mach build $(date)"
    128 133 
       ./mach build stage-package
    129 134 
     [% END -%]
    130 135
    131 
    -[% IF c("var/nightly") && c("var/linux") -%]
    136 
    +[% IF c("var/nightly") -%]
    132 137 
       echo "Building development artifacts"
    138 
    +
    139 
    +  [% IF c("var/macos") -%]
    140 
    +    export MOZ_PKG_MAC_BACKGROUND=$rootdir/dmg-root/[% c('var/ProjectName') %].dmg/.background/background.tiff
    141 
    +    export MOZ_PKG_MAC_DSSTORE=$rootdir/dmg-root/[% c('var/ProjectName') %].dmg/nightly.DS_Store
    142 
    +    export MOZ_PKG_MAC_ICON=$rootdir/dmg-root/[% c('var/ProjectName') %].dmg/.VolumeIcon.icns
    143 
    +  [% END -%]
    133 144 
       ./mach package
    134 145
    135 146 
       artifactsdir=[% out_dir %]/artifacts
    136 147 
       mkdir $artifactsdir
    148 
    +
    137 149 
       # Copy the artifacts to the target directory
    138 150 
       # Naming convention is the same as Mozilla uses for their artifacts
    139 
    -  cp -a obj-*/dist/*.tar.xz $artifactsdir/target.tar.xz
    140 
    -  cp -a obj-*/dist/*.zip $artifactsdir/target.xpt_artifacts.zip
    151 
    +  mv obj-*/dist/*.xpt_artifacts.zip $artifactsdir/target.xpt_artifacts.zip
    152 
    +
    153 
    +  [% IF c("var/macos") -%]
    154 
    +    mv obj-*/dist/*.dmg $artifactsdir/target.dmg
    155 
    +    mv obj-*/dist/*.update_framework_artifacts.zip $artifactsdir/target.update_framework_artifacts.zip
    156 
    +  [% END -%]
    157 
    +
    158 
    +  [% IF c("var/linux") -%]
    159 
    +    mv obj-*/dist/*.tar.xz $artifactsdir/target.tar.xz
    160 
    +  [% END -%]
    161 
    +
    162 
    +  [% IF c("var/windows") -%]
    163 
    +    mv obj-*/dist/*.zip $artifactsdir/target.zip
    164 
    +  [% END -%]
    165 
    +
    141 166 
       ./mach python -m mozbuild.action.test_archive common $artifactsdir/target.common.tests.tar.gz
    142 167 
     [% END %]
    143 168

  • projects/firefox/config
    ... ... @@ -249,3 +249,11 @@ input_files:
    249 249 
         # TorButton patch authored by Arthur Edelstein, from https://github.com/arthuredelstein/torbutton/ branch 2.1.10-namecoin
    250 250 
       - filename: namecoin-etld.patch
    251 251 
         enable: '[% c("var/namecoin") %]'
    252 
    +  - filename: dmg-root
    253 
    +    enable: '[% c("var/macos") && c("var/nightly") %]'
    254 
    +  - project: hfsplus-tools
    255 
    +    name: hfsplus-tools
    256 
    +    enable: '[% c("var/macos") && c("var/nightly") %]'
    257 
    +  - project: libdmg-hfsplus
    258 
    +    name: libdmg
    259 
    +    enable: '[% c("var/macos") && c("var/nightly") %]'

  • projects/firefox/mozconfig
    ... ... @@ -56,6 +56,12 @@
    56 56 
       export TOOLCHAIN_PREFIX=$CROSS_CCTOOLS_PATH/bin/[% c('var/build_target') %]-
    57 57 
       export DSYMUTIL="$TOOLCHAIN_DIR/clang/bin/dsymutil"
    58 58
    59 
    +  [% IF c("var/nightly") -%]
    60 
    +    export MKFSHFS=/var/tmp/dist/hfsplus-tools/newfs_hfs
    61 
    +    export DMG_TOOL=/var/tmp/dist/libdmg-hfsplus/dmg
    62 
    +    export HFS_TOOL=/var/tmp/dist/libdmg-hfsplus/hfsplus
    63 
    +  [% END -%]
    64 
    +
    59 65 
       export HOST_CFLAGS="-g"
    60 66 
       export HOST_CXXFLAGS="-g"
    61 67 
       export HOST_LDFLAGS="-g"

  • projects/geckoview/build
    ... ... @@ -38,7 +38,7 @@ echo "Starting ./mach build $(date)"
    38 38 
       echo "Building development artifacts"
    39 39 
       ./mach package
    40 40
    41 
    -  artifactsdir=[% dest_dir %]/artifacts
    41 
    +  artifactsdir=$outdir/artifacts
    42 42 
       mkdir -p $artifactsdir
    43 43 
       # Copy the artifacts to the target directory
    44 44 
       # Naming convention is the same as Mozilla uses for their artifacts
    ... ... @@ -50,16 +50,9 @@ echo "Starting ./mach build $(date)"
    50 50 
     echo "Build finished, copying the AAR to the to the destination directory $(date)"
    51 51
    52 52 
     [% IF !c("var/android_single_arch") -%]
    53 
    -  mkdir "$distdir/[% project %]"
    53 
    +  mkdir -p "$outdir/[% project %]"
    54 54 
       # We don't want the debug or "exoplayer" .aars, but the .aar that has `omni` in its name.
    55 
    -  find obj-* -type f -name geckoview*omni*.aar -exec cp {} $distdir/[% project %] \;
    55 
    +  find obj-* -type f -name geckoview*omni*.aar -exec mv {} $outdir/[% project %] \;
    56 56 
     [% ELSE -%]
    57 57 
     [% INCLUDE 'build_ac_fenix' %]
    58 58 
     [% END -%]
    59 
    -
    60 
    -echo "Starting to package artifacts $(date)"
    61 
    -cd $distdir
    62 
    -[% c('tar', {
    63 
    -    tar_src => [ project ],
    64 
    -    tar_args => '-caf ' _ dest_dir _ '/' _ c('filename'),
    65 
    -  }) %]

  • projects/geckoview/build_ac_fenix
    ... ... @@ -27,7 +27,7 @@ echo autoPublish.android-components.dir=../android-components > local.properties
    27 27 
       # Add our localized strings
    28 28 
       supported_locales="[% tmpl(c('var/locales_mobile').join(' ')) %]"
    29 29 
       for lang in $supported_locales; do
    30 
    -    cp "/var/tmp/dist/translation-fenix/$lang/torbrowser_strings.xml" "app/src/main/res/values-$lang/"
    30 
    +    cp "$distdir/translation-fenix/$lang/torbrowser_strings.xml" "app/src/main/res/values-$lang/"
    31 31 
       done
    32 32 
     [% END -%]
    33 33
    ... ... @@ -41,8 +41,8 @@ echo "Building $variant Fenix APK"
    41 41 
     gradle $GRADLE_FLAGS -PversionName="$version_name" "assemble$variant"
    42 42
    43 43 
     echo "Build finished, copying the APK(s) to the destination directory $(date)"
    44 
    -mkdir $distdir/[% project %]
    45 
    -cp $objdir/gradle/build/mobile/android/fenix/app/outputs/apk/fenix/$variant/*.apk $distdir/[% project %]
    44 
    +mkdir -p $outdir/[% project %]
    45 
    +mv $objdir/gradle/build/mobile/android/fenix/app/outputs/apk/fenix/$variant/*.apk $outdir/[% project %]
    46 46
    47 47 
     echo "Building non optimized $variant Fenix APK for testing"
    48 48 
     gradle $GRADLE_FLAGS -PversionName="$version_name" -PdisableOptimization "assemble$variant"
    ... ... @@ -52,9 +52,8 @@ gradle $GRADLE_FLAGS -PversionName="$version_name" -PtestBuildType="$variant" -P
    52 52
    53 53 
     echo "Test build finished, copying the APKs to the destination directory $(date)"
    54 54
    55 
    -mkdir $distdir/[% project %]/tests
    56 
    -cp $objdir/gradle/build/mobile/android/fenix/app/outputs/apk/fenix/$variant/*.apk $distdir/[% project %]/tests
    57 
    -cp $objdir/gradle/build/mobile/android/fenix/app/outputs/apk/androidTest/fenix/$variant/*.apk $distdir/[% project %]/tests
    55 
    +mkdir -p $outdir/[% project %]/tests
    56 
    +mv $objdir/gradle/build/mobile/android/fenix/app/outputs/apk/fenix/$variant/*.apk $outdir/[% project %]/tests
    57 
    +mv $objdir/gradle/build/mobile/android/fenix/app/outputs/apk/androidTest/fenix/$variant/*.apk $outdir/[% project %]/tests
    58 58
    59 59 
     popd
    60 
    -

  • projects/geckoview/build_apk
    ... ... @@ -38,9 +38,3 @@ echo "ac_add_options --with-branding=$branding_dir" >> mozconfig-android-all
    38 38 
     ./mach build --verbose
    39 39
    40 40 
     [% INCLUDE 'build_ac_fenix' %]
    41 
    -
    42 
    -cd $distdir/
    43 
    -[% c('tar', {
    44 
    -    tar_src => [ project ],
    45 
    -    tar_args => '-caf ' _ dest_dir _ '/' _ c('filename'),
    46 
    -  }) %]

  • projects/geckoview/build_common
    ... ... @@ -5,7 +5,8 @@
    5 5 
     }) %]
    6 6 
     distdir=/var/tmp/dist
    7 7 
     builddir=/var/tmp/build
    8 
    -mkdir -p $builddir $distdir
    8 
    +outdir="[%  dest_dir _ '/' _ c('filename') -%]"
    9 
    +mkdir -p $builddir $distdir $outdir
    9 10
    10 11 
     tar -C $distdir -xf [% c('input_files_by_name/node') %]
    11 12 
     export PATH="/var/tmp/dist/node/bin:$PATH"

  • projects/geckoview/config
    1 1 
     # vim: filetype=yaml sw=2
    2 2 
     version: '[% c("abbrev") %]'
    3 
    -filename: 'geckoview-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %].tar.[% c("compress_tar") %]'
    3 
    +filename: 'geckoview-[% c("version") %]-[% c("var/osname") %]-[% c("var/build_id") %]'
    4 4 
     git_hash: 'tor-browser-[% c("var/geckoview_version") %]-[% c("var/browser_branch") %]-build[% c("var/browser_build") %]'
    5 5 
     tag_gpg_id: 1
    6 6 
     git_url: https://gitlab.torproject.org/tpo/applications/tor-browser.git
    ... ... @@ -59,7 +59,7 @@ targets:
    59 59
    60 60 
     steps:
    61 61 
       build_apk:
    62 
    -    filename: 'geckoview-[% c("version") %]-apks-[% c("var/build_id") %].tar.[% c("compress_tar") %]'
    62 
    +    filename: 'geckoview-[% c("version") %]-apks-[% c("var/build_id") %]'
    63 63 
         version: '[% c("abbrev") %]'
    64 64 
         build_apk: '[% INCLUDE build_apk %]'
    65 65 
         input_files:

  • tools/signing/gatekeeper-bundling.sh
    ... ... @@ -64,7 +64,7 @@ test -d "$macos_signed_dir" || mkdir "$macos_signed_dir"
    64 64 
     tmpdir="$macos_stapled_dir/tmp"
    65 65 
     rm -Rf "$tmpdir"
    66 66 
     mkdir "$tmpdir"
    67 
    -cp -rT "$script_dir/../../projects/browser/Bundle-Data/$ProjName.dmg" "$tmpdir/dmg"
    67 
    +cp -rT "$script_dir/../../projects/common/dmg-root/$ProjName.dmg" "$tmpdir/dmg"
    68 68
    69 69 
     tar -C "$tmpdir" -xf "$libdmg_file"
    70 70 
     tar -C "$tmpdir" -xf "$hfstools_file"

    • —
    View it on GitLab.
    You're receiving this email because of your account on gitlab.torproject.org. Manage all notifications · Help Notification message regarding https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/compare/82357805279e6da528c7c2f8d8b201e6fcb6b69f...d5e2db251655fafb2f9f128f595a5470b8dda6f8 at 1750341230 

    _______________________________________________
tor-commits mailing list -- tor-commits@xxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to tor-commits-leave@xxxxxxxxxxxxxxxxxxxx