[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

[tor-commits] [Git][tpo/applications/tor-browser][base-browser-140.2.0esr-15.0-1] 6 commits: BB 43564: Modify ./mach bootstrap for Base Browser

Title: GitLab

brizental pushed to branch base-browser-140.2.0esr-15.0-1 at The Tor Project / Applications / Tor Browser

Commits:

  • 85b4d8d5
    by Beatriz Rizental at 2025-09-08T19:22:50+02:00 
    BB 43564: Modify ./mach bootstrap for Base Browser
  • 02a18b4d
    by Beatriz Rizental at 2025-09-08T19:22:58+02:00 
    fixup! BB 43564: Modify ./mach bootstrap for Base Browser

EXTRA: Stop asking to configure git during bootstrap.
  • 4937c1b6
    by Beatriz Rizental at 2025-09-08T19:23:06+02:00 
    fixup! BB 43564: Modify ./mach bootstrap for Base Browser
  • 98a01dbd
    by Beatriz Rizental at 2025-09-08T19:23:32+02:00 
    fixup! BB 41803: Add some developer tools for working on tor-browser.
  • 10ce4ee0
    by Beatriz Rizental at 2025-09-08T19:28:44+02:00 
    fixup! BB 43564: Modify ./mach bootstrap for Base Browser
  • e841a083
    by Beatriz Rizental at 2025-09-08T19:28:49+02:00 
    fixup! BB 43564: Modify ./mach bootstrap for Base Browser

25 changed files:

Changes:

  • build/moz.configure/basebrowser-resources.configure
    1 
    +# Helpers
    2 
    +# -------------------------------------------------
    3 
    +
    4 
    +
    5 
    +@depends(build_project)
    6 
    +def is_desktop_build(build_project):
    7 
    +    return build_project == "browser"
    8 
    +
    9 
    +
    10 
    +# Bootstrap resources
    11 
    +# -------------------------------------------------
    12 
    +
    13 
    +
    14 
    +option(
    15 
    +    "--with-noscript",
    16 
    +    env="NOSCRIPT",
    17 
    +    nargs=1,
    18 
    +    default=None,
    19 
    +    help="Path to noscript .xpi extension archive.",
    20 
    +)
    21 
    +
    22 
    +
    23 
    +@depends(
    24 
    +    "--with-noscript",
    25 
    +    mozbuild_state_path,
    26 
    +    bootstrap_path(
    27 
    +        "noscript", no_unpack=True, when=depends("--with-noscript")(lambda x: not x)
    28 
    +    ),
    29 
    +)
    30 
    +@checking("for noscript")
    31 
    +@imports(_from="pathlib", _import="Path")
    32 
    +def noscript(value, mozbuild_state_path, _bootstrapped):
    33 
    +    if value:
    34 
    +        path = Path(value[0])
    35 
    +        if path.is_file() and path.suffix == ".xpi":
    36 
    +            return value[0]
    37 
    +        else:
    38 
    +            die("--with-noscript must be an existing .xpi file")
    39 
    +
    40 
    +    bootstrapped_location = Path(mozbuild_state_path) / "browser"
    41 
    +    for file in bootstrapped_location.glob(f"*.xpi"):
    42 
    +        if "noscript" in file.name:
    43 
    +            return str(bootstrapped_location / file)
    44 
    +
    45 
    +    # noscript is not required for building.
    46 
    +    return None
    47 
    +
    48 
    +
    49 
    +set_config("NOSCRIPT", noscript)
    50 
    +
    51 
    +
    52 
    +option(
    53 
    +    "--with-tor-browser-fonts",
    54 
    +    env="TOR_BROWSER_FONTS",
    55 
    +    nargs=1,
    56 
    +    default=None,
    57 
    +    help="Path to location of fonts directory.",
    58 
    +)
    59 
    +
    60 
    +
    61 
    +@depends(
    62 
    +    "--with-tor-browser-fonts",
    63 
    +    mozbuild_state_path,
    64 
    +    bootstrap_path(
    65 
    +        "fonts",
    66 
    +        when=depends("--with-tor-browser-fonts")(lambda x: not x) & is_desktop_build,
    67 
    +    ),
    68 
    +)
    69 
    +@checking("for tor-browser fonts directory")
    70 
    +@imports(_from="pathlib", _import="Path")
    71 
    +def tor_browser_fonts(value, mozbuild_state_path, _bootstrapped):
    72 
    +    if value:
    73 
    +        path = Path(value[0])
    74 
    +        # TODO: Do a more thorough check on the directory.
    75 
    +        if path.is_dir():
    76 
    +            return value[0]
    77 
    +        else:
    78 
    +            die("--with-tor-browser-fonts must point to a real directory.")
    79 
    +
    80 
    +    bootstrapped_location = Path(mozbuild_state_path) / "fonts"
    81 
    +    if bootstrapped_location.is_dir():
    82 
    +        return str(bootstrapped_location)
    83 
    +
    84 
    +    # tor browser fonts directory is not required for building.
    85 
    +    return None
    86 
    +
    87 
    +
    88 
    +set_config("TOR_BROWSER_FONTS", tor_browser_fonts)

  • build/moz.configure/bootstrap.configure
    ... ... @@ -4,6 +4,29 @@
    4 4 
     # License, v. 2.0. If a copy of the MPL was not distributed with this
    5 5 
     # file, You can obtain one at http://mozilla.org/MPL/2.0/.
    6 6
    7 
    +option(
    8 
    +    "--with-tor-browser-build-out",
    9 
    +    env="TOR_BROWSER_BUILD_OUT",
    10 
    +    nargs=1,
    11 
    +    default="https://tb-build-06.torproject.org/~tb-builder/tor-browser-build/out",
    12 
    +    help="URL pointing to a Tor Browser Build out folder, served over HTTP[S].",
    13 
    +)
    14 
    +
    15 
    +
    16 
    +@depends("--with-tor-browser-build-out")
    17 
    +def tor_browser_build_out(value):
    18 
    +    if value:
    19 
    +        return value[0]
    20 
    +
    21 
    +
    22 
    +option(
    23 
    +    "--enable-tor-browser-build-only-bootstrap",
    24 
    +    env="TBB_ONLY_BOOTSTRAP",
    25 
    +    default=False,
    26 
    +    help="Flag that disables bootstrapping any artifact from Mozilla's Taskcluster. Will only bootstrap artifacts from tor-browser-build.",
    27 
    +)
    28 
    +
    29 
    +
    7 30 
     option(
    8 31 
         env="MOZ_FETCHES_DIR",
    9 32 
         nargs=1,
    ... ... @@ -115,9 +138,10 @@ def bootstrap_toolchain_tasks(host):
    115 138 
     def bootstrap_path(path, **kwargs):
    116 139 
         when = kwargs.pop("when", None)
    117 140 
         allow_failure = kwargs.pop("allow_failure", None)
    141 
    +    no_unpack = kwargs.pop("no_unpack", False)
    118 142 
         if kwargs:
    119 143 
             configure_error(
    120 
    -            "bootstrap_path only takes `when` and `allow_failure` as a keyword argument"
    144 
    +            "bootstrap_path only takes `when`, `allow_failure` and `no_unpack` as keyword arguments"
    121 145 
             )
    122 146
    123 147 
         @depends(
    ... ... @@ -129,11 +153,16 @@ def bootstrap_path(path, **kwargs):
    129 153 
             build_environment,
    130 154 
             dependable(path),
    131 155 
             dependable(allow_failure),
    156 
    +        dependable(no_unpack),
    157 
    +        tor_browser_build_out,
    158 
    +        "--enable-tor-browser-build-only-bootstrap",
    159 
    +        target,
    132 160 
             when=when,
    133 161 
         )
    134 162 
         @imports("os")
    135 163 
         @imports("subprocess")
    136 164 
         @imports("sys")
    165 
    +    @imports("mozbuild.tbbutils")
    137 166 
         @imports(_from="mozbuild.dirutils", _import="ensureParentDir")
    138 167 
         @imports(_from="importlib", _import="import_module")
    139 168 
         @imports(_from="shutil", _import="rmtree")
    ... ... @@ -148,6 +177,10 @@ def bootstrap_path(path, **kwargs):
    148 177 
             build_env,
    149 178 
             path,
    150 179 
             allow_failure,
    180 
    +        no_unpack,
    181 
    +        tor_browser_build_out,
    182 
    +        tbb_only_bootstrap,
    183 
    +        target,
    151 184 
         ):
    152 185 
             if not path:
    153 186 
                 return
    ... ... @@ -158,6 +191,83 @@ def bootstrap_path(path, **kwargs):
    158 191 
             if path_parts[0] == "clang-tools":
    159 192 
                 path_prefix = path_parts.pop(0)
    160 193
    194 
    +        # Small hack because noscript is inside the browser folder.
    195 
    +        if path_parts[0] == "noscript":
    196 
    +            path_prefix = "browser"
    197 
    +
    198 
    +        def try_tbb_bootstrap(exists):
    199 
    +            if not tor_browser_build_out:
    200 
    +                return False
    201 
    +
    202 
    +            # Tor browser build doesn't have artifacts for all targets supported
    203 
    +            # by the Firefox build system. When this is empty it means we are
    204 
    +            # building for a platform which tbb doesn't support.
    205 
    +            if not target.tor_browser_build_alias:
    206 
    +                return False
    207 
    +
    208 
    +            artifact = mozbuild.tbbutils.get_artifact_name(path_parts[0], tasks.prefix)
    209 
    +            if not artifact:
    210 
    +                log.info("%s is not mapped to a tbb artifact", path_parts[0])
    211 
    +                return False
    212 
    +
    213 
    +            artifact_path = mozbuild.tbbutils.get_artifact_path(
    214 
    +                tor_browser_build_out,
    215 
    +                artifact,
    216 
    +                target,
    217 
    +                prefix=path_prefix,
    218 
    +                log=log.warning,
    219 
    +            )
    220 
    +            if not artifact_path:
    221 
    +                log.info("no path found in tbb/out for %s", artifact)
    222 
    +                return False
    223 
    +
    224 
    +            # We will use the name of the artifact as the index.
    225 
    +            #
    226 
    +            # It's usually unique to the artifact version, but each artifact follows
    227 
    +            # a different naming convention, so we can't really get more specific here.
    228 
    +            artifact_index = artifact_path.rsplit("/", 1)[-1]
    229 
    +            index_file = os.path.join(toolchains_base_dir, "indices", artifact)
    230 
    +            try:
    231 
    +                with open(index_file) as fh:
    232 
    +                    index = fh.read().strip()
    233 
    +            except Exception:
    234 
    +                index = None
    235 
    +            if index == artifact_index and exists:
    236 
    +                log.debug("%s is up-to-date", artifact)
    237 
    +                return True
    238 
    +
    239 
    +            command = ["artifact", "toolchain", "--from-url", artifact_path]
    240 
    +
    241 
    +            if no_unpack:
    242 
    +                command.append("--no-unpack")
    243 
    +
    244 
    +            # Note to rebasers:
    245 
    +            # From here on, it's a slightly modified copy/paste
    246 
    +            # from the end of the try_bootstrap function
    247 
    +            log.info(
    248 
    +                "%s bootstrapped toolchain from TBB in %s",
    249 
    +                "Updating" if exists else "Installing",
    250 
    +                os.path.join(toolchains_base_dir, path_prefix, artifact),
    251 
    +            )
    252 
    +            os.makedirs(os.path.join(toolchains_base_dir, path_prefix), exist_ok=True)
    253 
    +            proc = subprocess.run(
    254 
    +                [
    255 
    +                    sys.executable,
    256 
    +                    os.path.join(build_env.topsrcdir, "mach"),
    257 
    +                    "--log-no-times",
    258 
    +                ]
    259 
    +                + command,
    260 
    +                cwd=os.path.join(toolchains_base_dir, path_prefix),
    261 
    +                check=not allow_failure,
    262 
    +            )
    263 
    +            if proc.returncode != 0 and allow_failure:
    264 
    +                return False
    265 
    +            ensureParentDir(index_file)
    266 
    +            with open(index_file, "w") as fh:
    267 
    +                fh.write(artifact_index)
    268 
    +
    269 
    +            return True
    270 
    +
    161 271 
             def try_bootstrap(exists):
    162 272 
                 if not tasks:
    163 273 
                     return False
    ... ... @@ -280,9 +390,10 @@ def bootstrap_path(path, **kwargs):
    280 390 
                 try:
    281 391 
                     # With --enable-bootstrap=no-update, we don't `try_bootstrap`, except
    282 392 
                     # when the toolchain can't be found.
    283 
    -                if (
    284 
    -                    "no-update" not in enable_bootstrap or not exists
    285 
    -                ) and not try_bootstrap(exists):
    393 
    +                if ("no-update" not in enable_bootstrap or not exists) and not (
    394 
    +                    try_tbb_bootstrap(exists)
    395 
    +                    or (not tbb_only_bootstrap and try_bootstrap(exists))
    396 
    +                ):
    286 397 
                         # If there aren't toolchain artifacts to use for this build,
    287 398 
                         # don't return a path.
    288 399 
                         return None

  • build/moz.configure/init.configure
    ... ... @@ -590,6 +590,21 @@ def split_triplet(triplet, allow_wasi=False):
    590 590 
         else:
    591 591 
             toolchain = "%s-%s" % (cpu, os)
    592 592
    593 
    +    # In tor-browser-build we use slightly different terminology for
    594 
    +    # the supported platforms. Let's prepare that OS string here.
    595 
    +    #
    596 
    +    # Not all possible platforms listed here are supported in tbb,
    597 
    +    # so this value will be empty sometimes.
    598 
    +    tor_browser_build_alias = None
    599 
    +    if canonical_os == "Android" and canonical_kernel == "Linux":
    600 
    +        tor_browser_build_alias = f"android"
    601 
    +    elif canonical_os == "GNU" and canonical_kernel == "Linux":
    602 
    +        tor_browser_build_alias = f"linux"
    603 
    +    elif canonical_os == "OSX" and canonical_kernel == "Darwin":
    604 
    +        tor_browser_build_alias = f"macos"
    605 
    +    elif canonical_os == "WINNT" and canonical_kernel == "WINNT":
    606 
    +        tor_browser_build_alias = f"windows"
    607 
    +
    593 608 
         return namespace(
    594 609 
             alias=triplet,
    595 610 
             cpu=CPU(canonical_cpu),
    ... ... @@ -604,6 +619,7 @@ def split_triplet(triplet, allow_wasi=False):
    604 619 
             toolchain=toolchain,
    605 620 
             vendor=vendor,
    606 621 
             sub_configure_alias=sub_configure_alias,
    622 
    +        tor_browser_build_alias=tor_browser_build_alias,
    607 623 
         )
    608 624
    609 625

  • moz.configure
    ... ... @@ -229,6 +229,7 @@ check_prog("WGET", ("wget",), allow_missing=True)
    229 229
    230 230
    231 231 
     include("build/moz.configure/toolchain.configure", when="--enable-compile-environment")
    232 
    +include("build/moz.configure/basebrowser-resources.configure")
    232 233
    233 234 
     include("build/moz.configure/pkg.configure")
    234 235 
     include("build/moz.configure/memory.configure", when="--enable-compile-environment")

  • python/mozboot/mozboot/bootstrap.py
    ... ... @@ -52,21 +52,28 @@ Note on Artifact Mode:
    52 52 
     Artifact builds download prebuilt C++ components rather than building
    53 53 
     them locally. Artifact builds are faster!
    54 54
    55 
    -Artifact builds are recommended for people working on Firefox or
    56 
    -Firefox for Android frontends, or the GeckoView Java API. They are unsuitable
    55 
    +Artifact builds are recommended for people working on Tor Browser or
    56 
    +Base Browser for Android frontends, or the GeckoView Java API. They are unsuitable
    57 57 
     for those working on C++ code. For more information see:
    58 58 
     https://firefox-source-docs.mozilla.org/contributing/build/artifact_builds.html.
    59 59
    60 
    -Please choose the version of Firefox you want to build (see note above):
    60 
    +# Note to Base Browser developers
    61 
    +
    62 
    +This is still highly experimental. Expect bugs!
    63 
    +
    64 
    +Please choose the version of Base Browser you want to build (see note above):
    61 65 
     %s
    62 66 
     Your choice: """
    63 67
    64 68 
     APPLICATIONS = OrderedDict(
    65 69 
         [
    66 
    -        ("Firefox for Desktop Artifact Mode", "browser_artifact_mode"),
    67 
    -        ("Firefox for Desktop", "browser"),
    68 
    -        ("GeckoView/Firefox for Android Artifact Mode", "mobile_android_artifact_mode"),
    69 
    -        ("GeckoView/Firefox for Android", "mobile_android"),
    70 
    +        ("Base Browser for Desktop Artifact Mode", "browser_artifact_mode"),
    71 
    +        ("Base Browser for Desktop", "browser"),
    72 
    +        (
    73 
    +            "GeckoView/Base Browser for Android Artifact Mode",
    74 
    +            "mobile_android_artifact_mode",
    75 
    +        ),
    76 
    +        ("GeckoView/Base Browser for Android", "mobile_android"),
    70 77 
             ("SpiderMonkey _javascript_ engine", "js"),
    71 78 
         ]
    72 79 
     )
    ... ... @@ -360,6 +367,8 @@ class Bootstrapper:
    360 367 
             getattr(self.instance, "ensure_%s_packages" % application)()
    361 368
    362 369 
         def check_code_submission(self, checkout_root: Path):
    370 
    +        return
    371 
    +
    363 372 
             if self.instance.no_interactive or which("moz-phab"):
    364 373 
                 return
    365 374
    ... ... @@ -474,8 +483,7 @@ class Bootstrapper:
    474 483 
                     configure_mercurial(hg, state_dir)
    475 484
    476 485 
             # Offer to configure Git, if the current checkout or repo type is Git.
    477 
    -        elif git and checkout_type == "git":
    478 
    -            should_configure_git = False
    486 
    +        elif False and git and checkout_type == "git":
    479 487 
                 if not self.instance.no_interactive:
    480 488 
                     should_configure_git = self.instance.prompt_yesno(prompt=CONFIGURE_GIT)
    481 489 
                 else:

  • python/mozbuild/mozbuild/action/tooltool.py
    ... ... @@ -1029,14 +1029,29 @@ def unpack_file(filename):
    1029 1029 
         """Untar `filename`, assuming it is uncompressed or compressed with bzip2,
    1030 1030 
         xz, gzip, zst, or unzip a zip file. The file is assumed to contain a single
    1031 1031 
         directory with a name matching the base of the given filename.
    1032 
    -    Xz support is handled by shelling out to 'tar'."""
    1032 
    +    Xz support is handled by shelling out to 'tar'.
    1033 
    +
    1034 
    +    tor-browser#41564 - For supporting tor-browser-build artifacts that contain
    1035 
    +    multiple directories, the archive is extracted into a directory with the
    1036 
    +    same name as the base of the filename. This modification is only applied to
    1037 
    +    tar archives, because that is all that was necessary.
    1038 
    +    """
    1033 1039 
         if os.path.isfile(filename) and tarfile.is_tarfile(filename):
    1034 1040 
             tar_file, zip_ext = os.path.splitext(filename)
    1035 1041 
             base_file, tar_ext = os.path.splitext(tar_file)
    1036 1042 
             clean_path(base_file)
    1037 1043 
             log.info('untarring "%s"' % filename)
    1038 1044 
             with TarFile.open(filename) as tar:
    1039 
    -            safe_extract(tar)
    1045 
    +            top_level_directories = set()
    1046 
    +            for name in tar.getnames():
    1047 
    +                dir = name.split("/", 1)[0]
    1048 
    +                top_level_directories.add(dir)
    1049 
    +            if len(top_level_directories) == 1:
    1050 
    +                safe_extract(tar)
    1051 
    +            else:
    1052 
    +                safe_extract(
    1053 
    +                    tar, path=os.path.join(os.path.dirname(filename), base_file)
    1054 
    +                )
    1040 1055 
         elif os.path.isfile(filename) and filename.endswith(".tar.zst"):
    1041 1056 
             import zstandard
    1042 1057

  • python/mozbuild/mozbuild/artifact_commands.py
    ... ... @@ -244,6 +244,12 @@ def artifact_clear_cache(command_context, tree=None, job=None, verbose=False):
    244 244 
         nargs="+",
    245 245 
         help="Download toolchain artifact from a given task.",
    246 246 
     )
    247 
    +@CommandArgument(
    248 
    +    "--from-url",
    249 
    +    metavar="URL",
    250 
    +    nargs="+",
    251 
    +    help="Download toolchain artifact from an arbitrary address.",
    252 
    +)
    247 253 
     @CommandArgument(
    248 254 
         "--tooltool-manifest",
    249 255 
         metavar="MANIFEST",
    ... ... @@ -273,6 +279,7 @@ def artifact_toolchain(
    273 279 
         skip_cache=False,
    274 280 
         from_build=(),
    275 281 
         from_task=(),
    282 
    +    from_url=[],
    276 283 
         tooltool_manifest=None,
    277 284 
         no_unpack=False,
    278 285 
         retry=0,
    ... ... @@ -504,6 +511,13 @@ def artifact_toolchain(
    504 511 
             record = ArtifactRecord(task_id, name)
    505 512 
             records[record.filename] = record
    506 513
    514 
    +    if from_url:
    515 
    +        for file in from_url:
    516 
    +            record = DownloadRecord(
    517 
    +                file, file.rsplit("/", 1)[-1], None, None, None, True
    518 
    +            )
    519 
    +            records[record.filename] = record
    520 
    +
    507 521 
         for record in records.values():
    508 522 
             command_context.log(
    509 523 
                 logging.INFO,

  • python/mozbuild/mozbuild/backend/base.py
    ... ... @@ -2,11 +2,14 @@
    2 2 
     # License, v. 2.0. If a copy of the MPL was not distributed with this
    3 3 
     # file, You can obtain one at http://mozilla.org/MPL/2.0/.
    4 4
    5 
    +import errno
    5 6 
     import itertools
    7 
    +import logging
    6 8 
     import os
    7 9 
     import time
    8 10 
     from abc import ABCMeta, abstractmethod
    9 11 
     from contextlib import contextmanager
    12 
    +from pathlib import Path
    10 13
    11 14 
     import mozpack.path as mozpath
    12 15 
     from mach.mixin.logging import LoggingMixin
    ... ... @@ -239,6 +242,72 @@ class BuildBackend(LoggingMixin):
    239 242 
                 with open(mozpath.join(dir, ".purgecaches"), "w") as f:
    240 243 
                     f.write("\n")
    241 244
    245 
    +    def _setup_tor_browser_environment(self, config):
    246 
    +        app = config.substs["MOZ_BUILD_APP"]
    247 
    +
    248 
    +        noscript_target_filename = "{73a6fe31-595d-460b-a920-fcc0f8843232}.xpi"
    249 
    +        noscript_location = config.substs.get("NOSCRIPT")
    250 
    +
    251 
    +        def _infallible_symlink(src, dst):
    252 
    +            try:
    253 
    +                os.symlink(src, dst)
    254 
    +            except OSError as e:
    255 
    +                if e.errno == errno.EEXIST:
    256 
    +                    # If the symlink already exists, remove it and try again.
    257 
    +                    os.remove(dst)
    258 
    +                    os.symlink(src, dst)
    259 
    +                else:
    260 
    +                    return
    261 
    +
    262 
    +        if app == "browser":
    263 
    +            tbdir = Path(config.topobjdir) / "dist" / "bin"
    264 
    +
    265 
    +            if config.substs.get("OS_TARGET") == "Darwin":
    266 
    +                tbdir = next(tbdir.glob("*.app"))
    267 
    +                paths = {
    268 
    +                    "docs": tbdir / "Contents/Resources/TorBrowser/Docs",
    269 
    +                    "exts": tbdir / "Contents/Resources/distribution/extensions",
    270 
    +                    "fonts": tbdir / "Resources/fonts",
    271 
    +                }
    272 
    +            else:
    273 
    +                paths = {
    274 
    +                    "docs": tbdir / "TorBrowser/Docs",
    275 
    +                    "exts": tbdir / "distribution/extensions",
    276 
    +                    "fonts": tbdir / "fonts",
    277 
    +                }
    278 
    +
    279 
    +            fonts_location = config.substs.get("TOR_BROWSER_FONTS")
    280 
    +            if fonts_location:
    281 
    +                self.log(
    282 
    +                    logging.INFO,
    283 
    +                    "_setup_tor_browser_environment",
    284 
    +                    {
    285 
    +                        "fonts_location": fonts_location,
    286 
    +                        "fonts_target": str(paths["fonts"]),
    287 
    +                    },
    288 
    +                    "Creating symlink for fonts files from {fonts_location} to {fonts_target}",
    289 
    +                )
    290 
    +
    291 
    +                for file in Path(fonts_location).iterdir():
    292 
    +                    target = paths["fonts"] / file.name
    293 
    +                    _infallible_symlink(file, target)
    294 
    +
    295 
    +            # Set up NoScript extension
    296 
    +            if noscript_location:
    297 
    +                noscript_target = paths["exts"] / noscript_target_filename
    298 
    +                self.log(
    299 
    +                    logging.INFO,
    300 
    +                    "_setup_tor_browser_environment",
    301 
    +                    {
    302 
    +                        "noscript_location": noscript_location,
    303 
    +                        "noscript_target": str(noscript_target),
    304 
    +                    },
    305 
    +                    "Creating symlink for NoScript from {noscript_location} to {noscript_target}",
    306 
    +                )
    307 
    +
    308 
    +                paths["exts"].mkdir(parents=True, exist_ok=True)
    309 
    +                _infallible_symlink(noscript_location, noscript_target)
    310 
    +
    242 311 
         def post_build(self, config, output, jobs, verbose, status):
    243 312 
             """Called late during 'mach build' execution, after `build(...)` has finished.
    244 313
    ... ... @@ -257,6 +326,9 @@ class BuildBackend(LoggingMixin):
    257 326 
             """
    258 327 
             self._write_purgecaches(config)
    259 328
    329 
    +        if status == 0:
    330 
    +            self._setup_tor_browser_environment(config)
    331 
    +
    260 332 
             return status
    261 333
    262 334 
         @contextmanager

  • python/mozbuild/mozbuild/tbbutils.py
    1 
    +import re
    2 
    +from urllib.request import Request, urlopen
    3 
    +
    4 
    +
    5 
    +def list_files_http(url):
    6 
    +    try:
    7 
    +        req = Request(url, method="GET")
    8 
    +        with urlopen(req) as response:
    9 
    +            if response.status != 200:
    10 
    +                return []
    11 
    +            html = response.read().decode()
    12 
    +    except Exception:
    13 
    +        return []
    14 
    +
    15 
    +    links = []
    16 
    +    for href in re.findall(r'<a href="">"([^"]+)"', html):
    17 
    +        if href == "../":
    18 
    +            continue
    19 
    +
    20 
    +        links.append(href)
    21 
    +
    22 
    +    return links
    23 
    +
    24 
    +
    25 
    +TOR_BROWSER_BUILD_ARTIFACTS = [
    26 
    +    # Tor Browser Build-only artifacts, these artifacts are not common with Firefox.
    27 
    +    "noscript",
    28 
    +    "fonts",
    29 
    +]
    30 
    +
    31 
    +# Mapping of artifacts from taskcluster to tor-browser-build.
    32 
    +ARTIFACT_NAME_MAP = {
    33 
    +    "cbindgen": "cbindgen",
    34 
    +    # FIXME (tor-browser-build#41471): nasm is more or less ready to go, but it needs to have the
    35 
    +    # executable in the root of the artifact folder instead of nasm/bin.
    36 
    +    # "nasm": "nasm",
    37 
    +    # FIXME (tor-browser-build#41421): the clang project as is, is not ready to use. It needs
    38 
    +    # to be repackaged with a bunch of things that differ per platform. Fun stuff.
    39 
    +    # "clang": "clang",
    40 
    +    "node": "node",
    41 
    +}
    42 
    +
    43 
    +
    44 
    +def get_artifact_name(original_artifact_name, host):
    45 
    +    # These are not build artifacts, they are pre-built artifacts to be added to the final build,
    46 
    +    # therefore this check can come before the host check.
    47 
    +    if original_artifact_name in TOR_BROWSER_BUILD_ARTIFACTS:
    48 
    +        return original_artifact_name
    49 
    +
    50 
    +    if host != "linux64":
    51 
    +        # Tor browser build only has development artifacts for linux64 host systems.
    52 
    +        return None
    53 
    +
    54 
    +    return ARTIFACT_NAME_MAP.get(original_artifact_name)
    55 
    +
    56 
    +
    57 
    +def get_artifact_path(url, artifact, target, prefix="", log=lambda *args, **kwargs: {}):
    58 
    +    if prefix:
    59 
    +        path = prefix
    60 
    +    else:
    61 
    +        path = artifact
    62 
    +
    63 
    +    # The `?C=M;O=D` parameters make it so links are ordered by
    64 
    +    # the last modified date. This here to make us get the latest
    65 
    +    # version of file in the case there are multiple and we just
    66 
    +    # grab the first one.
    67 
    +    files = list_files_http(f"{url}/{path}?C=M;O=D")
    68 
    +
    69 
    +    if not files:
    70 
    +        log(f"No files found in {url} for {artifact}.")
    71 
    +        return None
    72 
    +
    73 
    +    def filter_files(files, keyword):
    74 
    +        return [file for file in files if keyword in file]
    75 
    +
    76 
    +    artifact_files = [file for file in files if file.startswith(artifact)]
    77 
    +
    78 
    +    if len(artifact_files) == 0:
    79 
    +        log(f"No files found in {url} for {artifact}.")
    80 
    +        return None
    81 
    +
    82 
    +    if len(artifact_files) == 1:
    83 
    +        return f"{url}/{path}/{artifact_files[0]}"
    84 
    +
    85 
    +    files_per_os = filter_files(artifact_files, target.tor_browser_build_alias)
    86 
    +
    87 
    +    # If there are files in the folder, but they don't have the OS in the name
    88 
    +    # it probably means we can get any of them because they can be used to build
    89 
    +    # for any OS. So let's just get the first one.
    90 
    +    #
    91 
    +    # Note: It could be the case that the artifact _is_ OS dependant, but there
    92 
    +    # just are no files for the OS we are looking for. In that case, this will
    93 
    +    # return an incorrect artifact. This should not happen often though and is
    94 
    +    # something we cannot address until artifact names are standardized on tbb.
    95 
    +    if len(files_per_os) == 0:
    96 
    +        return f"{url}/{artifact}/{artifact_files[0]}"
    97 
    +
    98 
    +    elif len(files_per_os) == 1:
    99 
    +        return f"{url}/{artifact}/{files_per_os[0]}"
    100 
    +
    101 
    +    matches = filter_files(files_per_os, target.cpu)
    102 
    +
    103 
    +    return f"{url}/{artifact}/{matches[0]}" if matches else None

  • python/mozbuild/mozbuild/test/python.toml
    ... ... @@ -111,6 +111,9 @@ subsuite = "mozbuild"
    111 111
    112 112 
     ["test_rewrite_mozbuild.py"]
    113 113
    114 
    +["test_tbbutils.py"]
    115 
    +subsuite = "base-browser"
    116 
    +
    114 117 
     ["test_telemetry.py"]
    115 118
    116 119 
     ["test_telemetry_settings.py"]

  • python/mozbuild/mozbuild/test/test_tbbutils.py
    1 
    +import unittest
    2 
    +from types import SimpleNamespace
    3 
    +from unittest.mock import MagicMock, patch
    4 
    +
    5 
    +import mozunit
    6 
    +
    7 
    +from mozbuild.tbbutils import get_artifact_path, list_files_http
    8 
    +
    9 
    +
    10 
    +class TestGetArtifactName(unittest.TestCase):
    11 
    +    def setUp(self):
    12 
    +        self.artifact = "artifact"
    13 
    +        self.host = "linux64"
    14 
    +
    15 
    +    @patch("mozbuild.tbbutils.TOR_BROWSER_BUILD_ARTIFACTS", new=["artifact"])
    16 
    +    def test_artifact_in_tbb_artifacts(self):
    17 
    +        from mozbuild.tbbutils import get_artifact_name
    18 
    +
    19 
    +        result = get_artifact_name(self.artifact, self.host)
    20 
    +        self.assertEqual(result, self.artifact)
    21 
    +
    22 
    +    @patch("mozbuild.tbbutils.ARTIFACT_NAME_MAP", new={"artifact": "tcafitra"})
    23 
    +    def test_host_is_not_linux64(self):
    24 
    +        from mozbuild.tbbutils import get_artifact_name
    25 
    +
    26 
    +        result = get_artifact_name(self.artifact, "linux64-aarch64")
    27 
    +        self.assertIsNone(result)
    28 
    +
    29 
    +    @patch("mozbuild.tbbutils.ARTIFACT_NAME_MAP", new={"artifact": "tcafitra"})
    30 
    +    def test_mapped_artifact(self):
    31 
    +        from mozbuild.tbbutils import get_artifact_name
    32 
    +
    33 
    +        result = get_artifact_name(self.artifact, self.host)
    34 
    +        self.assertEqual(result, self.artifact[::-1])
    35 
    +
    36 
    +
    37 
    +class TestGetArtifactPath(unittest.TestCase):
    38 
    +    def setUp(self):
    39 
    +        self.url = "http://example.com"
    40 
    +        self.artifact = "artifact"
    41 
    +        # This is just an example target which is valid. But it doesn't make
    42 
    +        # any difference and could be anything for these tests.
    43 
    +        self.target = SimpleNamespace(tor_browser_build_alias="linux", cpu="x86_64")
    44 
    +
    45 
    +    @patch("mozbuild.tbbutils.list_files_http")
    46 
    +    def test_no_files_returns_none(self, mock_list_files):
    47 
    +        mock_list_files.return_value = []
    48 
    +        result = get_artifact_path(self.url, self.artifact, self.target)
    49 
    +        self.assertIsNone(result)
    50 
    +
    51 
    +    @patch("mozbuild.tbbutils.list_files_http")
    52 
    +    def test_no_matching_files_returns_none(self, mock_list_files):
    53 
    +        mock_list_files.return_value = ["somethingelse.zip", "yetanotherthing.zip"]
    54 
    +        result = get_artifact_path(self.url, self.artifact, self.target)
    55 
    +        self.assertIsNone(result)
    56 
    +
    57 
    +    @patch("mozbuild.tbbutils.list_files_http")
    58 
    +    def test_single_artifact_match(self, mock_list_files):
    59 
    +        mock_list_files.return_value = ["artifact-1.zip"]
    60 
    +        result = get_artifact_path(self.url, self.artifact, self.target)
    61 
    +        self.assertEqual(result, f"{self.url}/{self.artifact}/artifact-1.zip")
    62 
    +
    63 
    +    @patch("mozbuild.tbbutils.list_files_http")
    64 
    +    def test_artifact_without_os_returns_first(self, mock_list_files):
    65 
    +        mock_list_files.return_value = ["artifact-1.zip", "artifact-2.zip"]
    66 
    +        result = get_artifact_path(self.url, self.artifact, self.target)
    67 
    +        self.assertTrue(result.startswith(f"{self.url}/{self.artifact}/"))
    68 
    +        self.assertIn("artifact-", result)
    69 
    +
    70 
    +    @patch("mozbuild.tbbutils.list_files_http")
    71 
    +    def test_artifact_with_os_match(self, mock_list_files):
    72 
    +        mock_list_files.return_value = [
    73 
    +            "artifact-windows.zip",
    74 
    +            "artifact-linux.zip",
    75 
    +        ]
    76 
    +        result = get_artifact_path(self.url, self.artifact, self.target)
    77 
    +        self.assertEqual(result, f"{self.url}/{self.artifact}/artifact-linux.zip")
    78 
    +
    79 
    +    @patch("mozbuild.tbbutils.list_files_http")
    80 
    +    def test_artifact_with_cpu_match(self, mock_list_files):
    81 
    +        mock_list_files.return_value = [
    82 
    +            "artifact-linux-arm.zip",
    83 
    +            "artifact-linux-x86_64.zip",
    84 
    +        ]
    85 
    +        result = get_artifact_path(self.url, self.artifact, self.target)
    86 
    +        self.assertEqual(
    87 
    +            result, f"{self.url}/{self.artifact}/artifact-linux-x86_64.zip"
    88 
    +        )
    89 
    +
    90 
    +    @patch("mozbuild.tbbutils.list_files_http")
    91 
    +    def test_artifact_with_prefix(self, mock_list_files):
    92 
    +        mock_list_files.return_value = ["artifact-1.zip"]
    93 
    +
    94 
    +        prefix = "prefix"
    95 
    +        result = get_artifact_path(self.url, self.artifact, self.target, prefix=prefix)
    96 
    +        self.assertEqual(result, f"{self.url}/{prefix}/artifact-1.zip")
    97 
    +        mock_list_files.assert_called_with(f"{self.url}/{prefix}?C=M;O=D")
    98 
    +
    99 
    +
    100 
    +class TestListFilesHttp(unittest.TestCase):
    101 
    +    def setUp(self):
    102 
    +        self.url = "http://example.com"
    103 
    +
    104 
    +    @patch("mozbuild.tbbutils.urlopen")
    105 
    +    def test_non_200_status_returns_empty(self, mock_urlopen):
    106 
    +        mock_resp = MagicMock()
    107 
    +        mock_resp.status = 404
    108 
    +        mock_resp.read.return_value = b""
    109 
    +        mock_urlopen.return_value.__enter__.return_value = mock_resp
    110 
    +
    111 
    +        result = list_files_http(self.url)
    112 
    +        self.assertEqual(result, [])
    113 
    +
    114 
    +    @patch("mozbuild.tbbutils.urlopen")
    115 
    +    def test_exception_returns_empty(self, mock_urlopen):
    116 
    +        mock_urlopen.side_effect = Exception("network error")
    117 
    +        result = list_files_http(self.url)
    118 
    +        self.assertEqual(result, [])
    119 
    +
    120 
    +    @patch("mozbuild.tbbutils.urlopen")
    121 
    +    def test_regular_links(self, mock_urlopen):
    122 
    +        html = b"""
    123 
    +        <html><body>
    124 
    +        <a href="">"../">Parent</a>
    125 
    +        <a href="">"file1.zip">file1</a>
    126 
    +        <a href="">"file2.zip">file2</a>
    127 
    +        </body></html>
    128 
    +        """
    129 
    +        mock_resp = MagicMock()
    130 
    +        mock_resp.status = 200
    131 
    +        mock_resp.read.return_value = html
    132 
    +        mock_urlopen.return_value.__enter__.return_value = mock_resp
    133 
    +
    134 
    +        result = list_files_http(self.url)
    135 
    +        self.assertEqual(result, ["file1.zip", "file2.zip"])
    136 
    +
    137 
    +
    138 
    +if __name__ == "__main__":
    139 
    +    mozunit.main()

  • tools/base-browser/l10n/combine/tests/README deleted
    1 
    -python tests to be run with pytest.
    2 
    -Requires the compare-locales package.

  • tools/base-browser/git-rebase-fixup-preprocessor → tools/base_browser/git-rebase-fixup-preprocessor

  • tools/base-browser/l10n/combine-translation-versions.py → tools/base_browser/l10n/combine-translation-versions.py

  • tools/base-browser/l10n/combine/__init__.py → tools/base_browser/l10n/combine/__init__.py

  • tools/base-browser/l10n/combine/combine.py → tools/base_browser/l10n/combine/combine.py

  • tools/base-browser/l10n/combine/tests/__init__.py → tools/base_browser/l10n/combine/tests/__init__.py

  • tools/base_browser/l10n/combine/tests/python.toml
    1 
    +[DEFAULT]
    2 
    +subsuite = "base-browser"
    3 
    +
    4 
    +["test_android.py"]
    5 
    +
    6 
    +["test_dtd.py"]
    7 
    +
    8 
    +["test_fluent.py"]
    9 
    +
    10 
    +["test_properties.py"]

  • tools/base-browser/l10n/combine/tests/test_android.py → tools/base_browser/l10n/combine/tests/test_android.py
    1 1 
     import textwrap
    2 2
    3 
    -from combine import combine_files
    3 
    +import mozunit
    4 
    +from base_browser.l10n.combine import combine_files
    4 5
    5 6
    6 7 
     def wrap_in_xml(content):
    ... ... @@ -413,3 +414,7 @@ def test_alternatives():
    413 414 
             <string name="string_4_alt">Other string</string>
    414 415 
             """,
    415 416 
         )
    417 
    +
    418 
    +
    419 
    +if __name__ == "__main__":
    420 
    +    mozunit.main()

  • tools/base-browser/l10n/combine/tests/test_dtd.py → tools/base_browser/l10n/combine/tests/test_dtd.py
    1 1 
     import textwrap
    2 2
    3 
    -from combine import combine_files
    3 
    +import mozunit
    4 
    +from base_browser.l10n.combine import combine_files
    4 5
    5 6
    6 7 
     def assert_result(new_content, old_content, expect):
    ... ... @@ -411,3 +412,7 @@ def test_alternatives():
    411 412 
             <!ENTITY string.4.alt "Other string">
    412 413 
             """,
    413 414 
         )
    415 
    +
    416 
    +
    417 
    +if __name__ == "__main__":
    418 
    +    mozunit.main()

  • tools/base-browser/l10n/combine/tests/test_fluent.py → tools/base_browser/l10n/combine/tests/test_fluent.py
    1 1 
     import textwrap
    2 2
    3 
    -from combine import combine_files
    3 
    +import mozunit
    4 
    +from base_browser.l10n.combine import combine_files
    4 5
    5 6
    6 7 
     def assert_result(new_content, old_content, expect):
    ... ... @@ -475,3 +476,7 @@ def test_alternatives():
    475 476 
             -string-4-alt = Other string
    476 477 
             """,
    477 478 
         )
    479 
    +
    480 
    +
    481 
    +if __name__ == "__main__":
    482 
    +    mozunit.main()

  • tools/base-browser/l10n/combine/tests/test_properties.py → tools/base_browser/l10n/combine/tests/test_properties.py
    1 1 
     import textwrap
    2 2
    3 
    -from combine import combine_files
    3 
    +import mozunit
    4 
    +from base_browser.l10n.combine import combine_files
    4 5
    5 6
    6 7 
     def assert_result(new_content, old_content, expect):
    ... ... @@ -408,3 +409,7 @@ def test_alternatives():
    408 409 
             string.4.alt = Other string
    409 410 
             """,
    410 411 
         )
    412 
    +
    413 
    +
    414 
    +if __name__ == "__main__":
    415 
    +    mozunit.main()

  • tools/base-browser/missing-css-variables.py → tools/base_browser/missing-css-variables.py

  • tools/base-browser/tb-dev → tools/base_browser/tb-dev

  • tools/moz.build
    ... ... @@ -71,6 +71,7 @@ with Files("tryselect/docs/**"):
    71 71 
         SCHEDULES.exclusive = ["docs"]
    72 72
    73 73 
     PYTHON_UNITTEST_MANIFESTS += [
    74 
    +    "base_browser/l10n/combine/tests/python.toml",
    74 75 
         "fuzzing/smoke/python.toml",
    75 76 
         "lint/test/python.toml",
    76 77 
         "tryselect/test/python.toml",

    • —
    View it on GitLab.
    You're receiving this email because of your account on gitlab.torproject.org. Manage all notifications · Help Notification message regarding https://gitlab.torproject.org/tpo/applications/tor-browser/-/compare/53201cbee6c69e19bdf881c13c3e15d8339b5370...e841a083f6bca69d240f723acad2d0d63734f322 at 1757352633 

    _______________________________________________
tor-commits mailing list -- tor-commits@xxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to tor-commits-leave@xxxxxxxxxxxxxxxxxxxx