[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

[tor-commits] [Git][tpo/applications/tor-browser-build][maint-13.5] 3 commits: Bug 41284: Stop sync'ing changelogs between channels.

Title: GitLab

morgan pushed to branch maint-13.5 at The Tor Project / Applications / tor-browser-build

Commits:

  • 72f49dc1
    by Pier Angelo Vendrame at 2024-11-21T17:35:24+01:00 
    Bug 41284: Stop sync'ing changelogs between channels.
  • bf089520
    by Pier Angelo Vendrame at 2024-11-21T17:45:32+01:00 
    Bug 41200: Remove the tools for allowed_addons.json.

For tor-browser#42618 and tor-browser#42619, we stopped using the
allowed_addons.json, but initially we kept the tools to update it.
However, the file creates some confusion, because it is used only for
Andorid, but should be updated also when doing desktop-only releases
when they update NoScript, or Android nightly builds might fail.
So, since as a matter of fact we do not use that file anymore, we
decided to stop updating it and remove the related tools.
  • 84e2ad6a
    by Pier Angelo Vendrame at 2024-11-21T18:10:39+01:00 
    Bug 41310: 13.5-specific fixes for relprep.py.

When I wrote relprep.py, I did not think about a legacy channel, as we
never had one before.
Our approach was to keep everything to build releases as stable, so we
decided to comment or remove the undesired functionalities rather than
implementing a legacy channel in the script.

11 changed files:

Changes:

  • .gitattributes deleted
    1 
    -projects/browser/allowed_addons.json -diff

  • .gitlab/issue_templates/Release Prep - Tor Browser Alpha.md
    ... ... @@ -59,8 +59,6 @@ Tor Browser Alpha (and Nightly) are on the `main` branch
    59 59 
         - [ ] `fenix_version` : update to match alpha `firefox-android` build tag
    60 60 
         - [ ] `browser_branch` : update to match alpha `firefox-android` build tag
    61 61 
         - [ ] `browser_build` : update to match alpha `firefox-android` build tag
    62 
    -  - [ ] Update allowed_addons.json by running (from `tor-browser-build` root):
    63 
    -    - `./tools/fetch_allowed_addons.py > projects/browser/allowed_addons.json`
    64 62 
     - [ ] Update `projects/translation/config`:
    65 63 
       - [ ] run `make list_translation_updates-alpha` to get updated hashes
    66 64 
       - [ ] `steps/base-browser/git_hash` : update with `HEAD` commit of project's `base-browser` branch

  • .gitlab/issue_templates/Release Prep - Tor Browser Stable.md
    ... ... @@ -60,8 +60,6 @@ Tor Browser Stable lives in the various `maint-$(TOR_BROWSER_MAJOR).$(TOR_BROWSE
    60 60 
         - [ ] `browser_branch` : update to match stable `firefox-android` build tag
    61 61 
         - [ ] `browser_build` : update to match stable `firefox-android` build tag
    62 62 
       variant: Beta
    63 
    -  - [ ] Update allowed_addons.json by running (from `tor-browser-build` root):
    64 
    -    - `./tools/fetch_allowed_addons.py > projects/browser/allowed_addons.json`
    65 63 
     - [ ] Update `projects/translation/config`:
    66 64 
       - [ ] run `make list_translation_updates-release` to get updated hashes
    67 65 
       - [ ] `steps/base-browser/git_hash` : update with `HEAD` commit of project's `base-browser` branch

  • projects/browser/allowed_addons.json deleted The diff for this file was not included because it is too large.
  • projects/browser/build.android
    ... ... @@ -39,15 +39,6 @@ unzip ../omni.ja
    39 39 
           }) %]
    40 40 
     popd
    41 41
    42 
    -
    43 
    -[% IF c("var/verify_allowed_addons") %]
    44 
    -  # Check that allowed_addons.json contains the right versions of our bundled extension(s).
    45 
    -  # If so, replace the default allowed_addons.json by ours in the apk assets folder.
    46 
    -  $rootdir/verify_allowed_addons.py "$rootdir/allowed_addons.json" "$noscript_path"
    47 
    -[% END %]
    48 
    -
    49 
    -mv $rootdir/allowed_addons.json $assets_dir/allowed_addons.json
    50 
    -
    51 42 
     mkdir apk
    52 43 
     pushd apk
    53 44 
     7zz x "$apk"

  • projects/browser/config
    ... ... @@ -49,7 +49,6 @@ targets:
    49 49 
       android:
    50 50 
         build: '[% INCLUDE build.android %]'
    51 51 
         var:
    52 
    -      verify_allowed_addons: 1
    53 52 
           arch_deps:
    54 53 
             - 7zip
    55 54 
             - openjdk-17-jdk-headless
    ... ... @@ -159,10 +158,6 @@ input_files:
    159 158 
         enable: '[% c("var/namecoin") %]'
    160 159 
       - filename: namecoin.patch
    161 160 
         enable: '[% c("var/namecoin") %]'
    162 
    -  - filename: allowed_addons.json
    163 
    -    enable: '[% c("var/android") %]'
    164 
    -  - filename: verify_allowed_addons.py
    165 
    -    enable: '[% c("var/android") && c("var/verify_allowed_addons") %]'
    166 161 
       - project: manual
    167 162 
         name: manual
    168 163 
         enable: '[% ! c("var/android") && c("var/tor-browser") %]'

  • projects/browser/verify_allowed_addons.py deleted
    1 
    -#!/usr/bin/env python3
    2 
    -
    3 
    -import json
    4 
    -import sys
    5 
    -import hashlib
    6 
    -import zipfile
    7 
    -
    8 
    -def find_addon(addons, addon_id):
    9 
    -  results = addons['results']
    10 
    -  for x in results:
    11 
    -    addon = x['addon']
    12 
    -    if addon['guid'] == addon_id:
    13 
    -      return addon
    14 
    -  sys.exit("Error: cannot find addon " + addon_id)
    15 
    -
    16 
    -def verify_extension_version(addons, addon_id, version):
    17 
    -  addon = find_addon(addons, addon_id)
    18 
    -  expected_version = addon['current_version']['version']
    19 
    -  if version != expected_version:
    20 
    -    sys.exit("Error: version " + version + " != " + expected_version)
    21 
    -
    22 
    -def verify_extension_hash(addons, addon_id, hash):
    23 
    -  addon = find_addon(addons, addon_id)
    24 
    -  expected_hash = addon["current_version"]["files"][0]["hash"]
    25 
    -  if hash != expected_hash:
    26 
    -    sys.exit("Error: hash " + hash + " != " + expected_hash)
    27 
    -
    28 
    -def read_extension_manifest(path):
    29 
    -  return json.loads(zipfile.ZipFile(path, 'r').read('manifest.json'))
    30 
    -
    31 
    -def main(argv):
    32 
    -  allowed_addons_path = argv[0]
    33 
    -  noscript_path = argv[1]
    34 
    -
    35 
    -  addons = None
    36 
    -  with open(allowed_addons_path, 'r') as file:
    37 
    -    addons = json.loads(file.read())
    38 
    -
    39 
    -  noscript_hash = None
    40 
    -  with open(noscript_path, 'rb') as file:
    41 
    -    noscript_hash = "sha256:" + hashlib.sha256(file.read()).hexdigest()
    42 
    -
    43 
    -  noscript_version = read_extension_manifest(noscript_path)["version"]
    44 
    -
    45 
    -  verify_extension_hash(addons, '{73a6fe31-595d-460b-a920-fcc0f8843232}', noscript_hash)
    46 
    -  verify_extension_version(addons, '{73a6fe31-595d-460b-a920-fcc0f8843232}', noscript_version)
    47 
    -
    48 
    -if __name__ == "__main__":
    49 
    -   main(sys.argv[1:])

  • projects/manual/config
    1 1 
     # vim: filetype=yaml sw=2
    2 2 
     # To update, see doc/how-to-update-the-manual.txt
    3 3 
     # Remember to update also the package's hash, with the version!
    4 
    -version: 222718
    4 
    +version: 210938
    5 5 
     filename: 'manual-[% c("version") %]-[% c("var/build_id") %].tar.[% c("compress_tar") %]'
    6 6 
     container:
    7 7 
       use_container: 1
    ... ... @@ -23,6 +23,6 @@ input_files:
    23 23 
       - project: container-image
    24 24 
       - URL: 'https://build-sources.tbb.torproject.org/manual_[% c("version") %].zip'
    25 25 
         name: manual
    26 
    -    sha256sum: 051174ba012fa2241e865cc604658a0af116d3bbf9d02474025277fff1b34636
    26 
    +    sha256sum: eb83259f0525a14dae1a1c3944e1e5ac3a2f8111a42834ab0f401628c8a38791
    27 27 
       - filename: packagemanual.py
    28 28 
         name: package_script

  • tools/fetch_allowed_addons.py deleted
    1 
    -#!/usr/bin/env python3
    2 
    -
    3 
    -import urllib.request
    4 
    -import json
    5 
    -import base64
    6 
    -import sys
    7 
    -
    8 
    -NOSCRIPT = "{73a6fe31-595d-460b-a920-fcc0f8843232}"
    9 
    -
    10 
    -
    11 
    -def fetch(x):
    12 
    -    with urllib.request.urlopen(x) as response:
    13 
    -        return response.read()
    14 
    -
    15 
    -
    16 
    -def find_addon(addons, addon_id):
    17 
    -    results = addons["results"]
    18 
    -    for x in results:
    19 
    -        addon = x["addon"]
    20 
    -        if addon["guid"] == addon_id:
    21 
    -            return addon
    22 
    -
    23 
    -
    24 
    -def fetch_and_embed_icons(addons):
    25 
    -    results = addons["results"]
    26 
    -    for x in results:
    27 
    -        addon = x["addon"]
    28 
    -        icon_data = fetch(addon["icon_url"])
    29 
    -        addon["icon_url"] = "data:image/png;base64," + str(
    30 
    -            base64.b64encode(icon_data), "utf8"
    31 
    -        )
    32 
    -
    33 
    -
    34 
    -def fetch_allowed_addons(amo_collection=None):
    35 
    -    if amo_collection is None:
    36 
    -        amo_collection = "83a9cccfe6e24a34bd7b155ff9ee32"
    37 
    -    url = f"https://services.addons.mozilla.org/api/v4/accounts/account/mozilla/collections/{amo_collection}/addons/"
    38 
    -    data = json.loads(fetch(url))
    39 
    -    fetch_and_embed_icons(data)
    40 
    -    data["results"].sort(key=lambda x: x["addon"]["guid"])
    41 
    -    return data
    42 
    -
    43 
    -
    44 
    -def main(argv):
    45 
    -    data = fetch_allowed_addons(argv[0] if len(argv) > 1 else None)
    46 
    -    # Check that NoScript is present
    47 
    -    if find_addon(data, NOSCRIPT) is None:
    48 
    -        sys.exit("Error: cannot find NoScript.")
    49 
    -    print(json.dumps(data, indent=2, ensure_ascii=False))
    50 
    -
    51 
    -
    52 
    -if __name__ == "__main__":
    53 
    -    main(sys.argv[1:])

  • tools/fetch_changelogs.py
    ... ... @@ -21,12 +21,10 @@ class EntryType(enum.IntFlag):
    21 21
    22 22
    23 23 
     class Platform(enum.IntFlag):
    24 
    -    WINDOWS = 8
    25 
    -    MACOS = 4
    26 
    -    LINUX = 2
    27 
    -    ANDROID = 1
    28 
    -    DESKTOP = 8 | 4 | 2
    29 
    -    ALL_PLATFORMS = 8 | 4 | 2 | 1
    24 
    +    WINDOWS = 2
    25 
    +    MACOS = 1
    26 
    +    DESKTOP = 2 | 1
    27 
    +    ALL_PLATFORMS = 2 | 1
    30 28
    31 29
    32 30 
     class ChangelogEntry:
    ... ... @@ -52,10 +50,6 @@ class ChangelogEntry:
    52 50 
                 platforms.append("Windows")
    53 51 
             if self.platform & Platform.MACOS:
    54 52 
                 platforms.append("macOS")
    55 
    -        if self.platform & Platform.LINUX:
    56 
    -            platforms.append("Linux")
    57 
    -        if self.platform & Platform.ANDROID:
    58 
    -            platforms.append("Android")
    59 53 
             return " + ".join(platforms)
    60 54
    61 55 
         def __lt__(self, other):
    ... ... @@ -78,15 +72,8 @@ class ChangelogEntry:
    78 72
    79 73 
     class UpdateEntry(ChangelogEntry):
    80 74 
         def __init__(self, name, version, is_mb):
    81 
    -        if name == "Firefox" and not is_mb:
    82 
    -            platform = Platform.DESKTOP
    83 
    -            num_platforms = 3
    84 
    -        elif name == "GeckoView" or name == "Zstandard":
    85 
    -            platform = Platform.ANDROID
    86 
    -            num_platforms = 1
    87 
    -        else:
    88 
    -            platform = Platform.ALL_PLATFORMS
    89 
    -            num_platforms = 4
    75 
    +        platform = Platform.ALL_PLATFORMS
    76 
    +        num_platforms = 2
    90 77 
             super().__init__(
    91 78 
                 EntryType.UPDATE, platform, num_platforms, name == "Go", is_mb
    92 79 
             )
    ... ... @@ -107,8 +94,8 @@ class Issue(ChangelogEntry):
    107 94 
             platform = 0
    108 95 
             num_platforms = 0
    109 96 
             if "Desktop" in j["labels"]:
    110 
    -            platform = Platform.DESKTOP
    111 
    -            num_platforms += 3
    97 
    +            platform = Platform.ALL_PLATFORMS
    98 
    +            num_platforms += 2
    112 99 
             else:
    113 100 
                 if "Windows" in j["labels"]:
    114 101 
                     platform |= Platform.WINDOWS
    ... ... @@ -116,20 +103,13 @@ class Issue(ChangelogEntry):
    116 103 
                 if "MacOS" in j["labels"]:
    117 104 
                     platform |= Platform.MACOS
    118 105 
                     num_platforms += 1
    119 
    -            if "Linux" in j["labels"]:
    120 
    -                platform |= Platform.LINUX
    121 
    -                num_platforms += 1
    122 
    -        if "Android" in j["labels"]:
    123 
    -            if is_mb and num_platforms == 0:
    106 
    +        if not platform:
    107 
    +            if "Android" in j["labels"] or "Linux" in j["labels"]:
    124 108 
                     raise Exception(
    125 
    -                    f"Android-only issue on Mullvad Browser: {j['references']['full']}!"
    109 
    +                    f"The legacy channel should include only fixes for macOS and/or Windows, please check {self.project}#{self.number}."
    126 110 
                     )
    127 
    -            elif not is_mb:
    128 
    -                platform |= Platform.ANDROID
    129 
    -                num_platforms += 1
    130 
    -        if not platform or (is_mb and platform == Platform.DESKTOP):
    131 111 
                 platform = Platform.ALL_PLATFORMS
    132 
    -            num_platforms = 4
    112 
    +            num_platforms = 2
    133 113 
             is_build = "Build System" in j["labels"]
    134 114 
             super().__init__(
    135 115 
                 EntryType.ISSUE, platform, num_platforms, is_build, is_mb

  • tools/relprep.py
    ... ... @@ -4,7 +4,6 @@ from collections import namedtuple
    4 4 
     import configparser
    5 5 
     from datetime import datetime, timezone
    6 6 
     from hashlib import sha256
    7 
    -import json
    8 7 
     import locale
    9 8 
     import logging
    10 9 
     from pathlib import Path
    ... ... @@ -16,7 +15,6 @@ from git import Repo
    16 15 
     import requests
    17 16 
     import ruamel.yaml
    18 17
    19 
    -from fetch_allowed_addons import NOSCRIPT, fetch_allowed_addons, find_addon
    20 18 
     import fetch_changelogs
    21 19 
     from update_manual import update_manual
    22 20
    ... ... @@ -93,13 +91,10 @@ class ReleasePreparation:
    93 91 
             self.base_path = Path(repo_path)
    94 92 
             self.repo = Repo(self.base_path)
    95 93
    96 
    -        self.tor_browser = bool(kwargs.get("tor_browser", True))
    97 
    -        self.mullvad_browser = bool(kwargs.get("mullvad_browser", True))
    98 
    -        if not self.tor_browser and not self.mullvad_browser:
    99 
    -            raise ValueError("Nothing to do")
    100 
    -        self.android = kwargs.get("android", self.tor_browser)
    101 
    -        if not self.tor_browser and self.android:
    102 
    -            raise ValueError("Only Tor Browser supports Android")
    94 
    +        # Legacy channel, always do Tor Browser desktop only.
    95 
    +        self.tor_browser = True
    96 
    +        self.mullvad_browser = False
    97 
    +        self.android = False
    103 98
    104 99 
             logger.debug(
    105 100 
                 "Tor Browser: %s; Mullvad Browser: %s; Android: %s",
    ... ... @@ -142,7 +137,8 @@ class ReleasePreparation:
    142 137 
                 # Do not update Go anymore: 1.21.x is not listed anymore in
    143 138 
                 # the download page as it is EOL as of August 13, 2024.
    144 139 
                 # self.update_go()
    145 
    -            self.update_manual()
    140 
    +            # Freeze the manual to before 14.0.
    141 
    +            # self.update_manual()
    146 142
    147 143 
             self.update_changelogs()
    148 144 
             self.update_rbm_conf()
    ... ... @@ -304,7 +300,6 @@ class ReleasePreparation:
    304 300 
             targets = ["base-browser"]
    305 301 
             if self.tor_browser:
    306 302 
                 targets.append("tor-browser")
    307 
    -            targets.append("fenix")
    308 303 
             if self.mullvad_browser:
    309 304 
                 targets.append("mullvad-browser")
    310 305 
             for i in targets:
    ... ... @@ -319,28 +314,27 @@ class ReleasePreparation:
    319 314 
             logger.info("Updating addons")
    320 315 
             config = self.load_config("browser")
    321 316
    322 
    -        amo_data = fetch_allowed_addons()
    323 
    -        logger.debug("Fetched AMO data")
    324 
    -        if self.android:
    325 
    -            with (
    326 
    -                self.base_path / "projects/browser/allowed_addons.json"
    327 
    -            ).open("w") as f:
    328 
    -                json.dump(amo_data, f, indent=2)
    329 
    -
    330 
    -        noscript = find_addon(amo_data, NOSCRIPT)
    331 317 
             logger.debug("Updating NoScript")
    332 
    -        self.update_addon_amo(config, "noscript", noscript)
    318 
    +        self.update_addon_amo(
    319 
    +            config, "noscript", "{73a6fe31-595d-460b-a920-fcc0f8843232}"
    320 
    +        )
    333 321 
             if self.mullvad_browser:
    334 322 
                 logger.debug("Updating uBlock Origin")
    335 
    -            ublock = find_addon(amo_data, "uBlock0@xxxxxxxxxxxxxxx")
    336 
    -            self.update_addon_amo(config, "ublock-origin", ublock)
    323 
    +            self.update_addon_amo(
    324 
    +                config, "ublock-origin", "uBlock0@xxxxxxxxxxxxxxx"
    325 
    +            )
    337 326 
                 logger.debug("Updating the Mullvad Browser extension")
    338 327 
                 self.update_mullvad_addon(config)
    339 328
    340 329 
             self.save_config("browser", config)
    341 330
    342 
    -    def update_addon_amo(self, config, name, addon):
    343 
    -        addon = addon["current_version"]["files"][0]
    331 
    +    def update_addon_amo(self, config, name, addon_id):
    332 
    +        r = requests.get(
    333 
    +            f"https://services.addons.mozilla.org/api/v4/addons/addon/{addon_id}"
    334 
    +        )
    335 
    +        r.raise_for_status()
    336 
    +        amo_data = r.json()
    337 
    +        addon = amo_data["current_version"]["files"][0]
    344 338 
             assert addon["hash"].startswith("sha256:")
    345 339 
             addon_input = self.find_input(config, name)
    346 340 
             addon_input["URL"] = addon["url"]
    ... ... @@ -523,6 +517,10 @@ class ReleasePreparation:
    523 517 
                             self.build_number,
    524 518 
                         )
    525 519 
                     continue
    520 
    +            if version.major > self.version.major:
    521 
    +                # Ignore more recent version.
    522 
    +                # E.g., for the legacy channel.
    523 
    +                continue
    526 524 
                 key = (project, version.channel)
    527 525 
                 if key not in self.last_releases:
    528 526 
                     self.last_releases[key] = []
    ... ... @@ -590,14 +588,9 @@ class ReleasePreparation:
    590 588 
             changelogs = cb.create(**kwargs)
    591 589
    592 590 
             path = f"projects/browser/Bundle-Data/Docs-{tag_prefix.upper()}/ChangeLog.txt"
    593 
    -        stable_tag = self.last_releases[(tag_prefix, "release")][0].tag
    594 
    -        alpha_tag = self.last_releases[(tag_prefix, "alpha")][0].tag
    595 
    -        if stable_tag.tagged_date > alpha_tag.tagged_date:
    596 
    -            last_tag = stable_tag
    597 
    -        else:
    598 
    -            last_tag = alpha_tag
    599 
    -        logger.debug("Using %s to add the new changelogs to.", last_tag.tag)
    600 
    -        last_changelogs = self.repo.git.show(f"{last_tag.tag}:{path}")
    591 
    +        # Take HEAD to reset any changes we might already have from a
    592 
    +        # previous run.
    593 
    +        last_changelogs = self.repo.git.show(f"HEAD:{path}")
    601 594 
             with (self.base_path / path).open("w") as f:
    602 595 
                 f.write(changelogs + "\n" + last_changelogs + "\n")
    603 596
    ... ... @@ -705,8 +698,6 @@ if __name__ == "__main__":
    705 698 
             default=Path(__file__).parent.parent,
    706 699 
             help="Path to a tor-browser-build.git clone",
    707 700 
         )
    708 
    -    parser.add_argument("--tor-browser", action="store_true")
    709 
    -    parser.add_argument("--mullvad-browser", action="store_true")
    710 701 
         parser.add_argument(
    711 702 
             "--date",
    712 703 
             help="Release date and optionally time for changelog purposes. "
    ... ... @@ -747,12 +738,7 @@ if __name__ == "__main__":
    747 738 
         )
    748 739 
         logger.addHandler(ch)
    749 740
    750 
    -    tbb = bool(args.tor_browser)
    751 
    -    mb = bool(args.mullvad_browser)
    752 741 
         kwargs = {}
    753 
    -    if tbb or mb:
    754 
    -        kwargs["tor_browser"] = tbb
    755 
    -        kwargs["mullvad_browser"] = mb
    756 742 
         if args.date:
    757 743 
             try:
    758 744 
                 kwargs["changelog_date"] = datetime.fromisoformat(args.date)

    • —
    View it on GitLab.
    You're receiving this email because of your account on gitlab.torproject.org. Manage all notifications · Help 

    _______________________________________________
tor-commits mailing list -- tor-commits@xxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to tor-commits-leave@xxxxxxxxxxxxxxxxxxxx