[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

[tor-commits] [Git][tpo/applications/tor-browser-build][main] Bug 41711: Update our manual machinery for its new home

Title: GitLab

boklm pushed to branch main at The Tor Project / Applications / tor-browser-build

Commits:

  • 8b8abf69
    by Nicolas Vigier at 2026-07-28T13:41:58+02:00 
    Bug 41711: Update our manual machinery for its new home

7 changed files:

Changes:

  • doc/how-to-update-the-manual.txt
    1 
    -1. Go to https://gitlab.torproject.org/tpo/web/manual/-/jobs/
    2 
    -2. Open the latest build stage
    3 
    -3. Download the artifacts (they come in a .zip file).
    4 
    -4. Rename it to manual_$PIPELINEID.zip
    5 
    -5. Upload it to people.tpo
    6 
    -6. Update projects/manual/config
    7 
    -   6.1 Change the version to $PIPELINEID
    8 
    -   6.2 Update the hash in the input_files section
    9 
    -   6.3 Update the URL if it was different from the previous uploader
    1 
    +1. Run ./tools/update_manual.py
    2 
    +2. Upload the new file to tb-build-02.torproject.org:~tb-builder/public_html/
    3 
    +3. Commit the changes made by the script to projects/manual/config

  • projects/browser/build
    ... ... @@ -184,7 +184,7 @@ tar -C "${TB_STAGE_DIR}" -xf [% c('input_files_by_name/firefox') %]/browser.tar.
    184 184 
       find . -exec [% c("touch") %] {} \;
    185 185 
       for tbdir in "${TBDIRS[@]}"
    186 186 
       do
    187 
    -    find chrome/ | sort | zip -X -@ "$tbdir[% IF c('var/macos') %]/Contents/Resources[% END %]/browser/omni.ja"
    187 
    +    find chrome/ defaults/preferences/tor-browser-manual.js | sort | zip -X -@ "$tbdir[% IF c('var/macos') %]/Contents/Resources[% END %]/browser/omni.ja"
    188 188 
       done
    189 189 
       popd
    190 190 
       rm -rf $TMP_MANUAL_PATH

  • projects/manual/README.md
    ... ... @@ -10,7 +10,11 @@ So, since the manual doesn't contain binary executables, we decided not to build
    10 10 
     it, but to mirror the latest result of the CI to make sure a build stays
    11 11 
     reproducible (otherwise, CI artifacts would be deleted after a week).
    12 12
    13 
    -However, the CI artifacts include a lot of files used by other Tor sites but not
    14 
    -from the manual, so we delete them in our build script to save space in the
    15 
    -builds. Also, the paths to resources need to be updated, because we serve the
    16 
    -manual as an about page, so we need to convert them to full `chrome://` URIs.
    13 
    +The download of the latest CI artifacts can be done by the script
    14 
    +`tools/update_manual.py`.
    15 
    +
    16 
    +However, the CI artifacts include a lot of files used by other Tor sites
    17 
    +but not from the manual, so they need to be deleted, and some of the
    18 
    +files need to be adapted for offline use. This is done by the script
    19 
    +`package_marble_build_artifacts.py` (which has tests in
    20 
    +`test_package_marble_build_artifacts.py`).

  • projects/manual/build
    1 1 
     #!/bin/bash
    2 2 
     [% c("var/set_default_env") -%]
    3 3
    4 
    -mkdir -p chrome/browser/content/browser/manual
    5 
    -unzip $rootdir/[% c('input_files_by_name/manual') %]
    6 
    -python3 $rootdir/[% c('input_files_by_name/package_script') %] public chrome/browser/content/browser/manual
    4 
    +manual_dir='chrome/browser/content/browser/aboutmanual'
    5 
    +mkdir -p "$manual_dir" manual-src
    6 
    +unzip -d manual-src $rootdir/[% c('input_files_by_name/manual') %]
    7 
    +python3 $rootdir/[% c('input_files_by_name/package_script') %] \
    8 
    +	--in-dir manual-src/public --out-dir "$manual_dir" \
    9 
    +	--out-locales "$rootdir/supported-locales.txt"
    7 10
    8 
    -pushd chrome/browser/content/browser/manual/static
    9 
    -find -name '*.map' -delete
    10 
    -rm -rf fonts
    11 
    -rm -rf css/digital-climate-strike css/images
    12 
    -# We prune unused images in the Python script, but empty directories remain
    13 
    -find images -type d -empty -delete
    14 
    -popd
    11 
    +# Check for the expected characters, which will also ensure this can
    12 
    +# form a valid _javascript_ string literal.
    13 
    +# Note that $() will strip any trailing newlines, if any.
    14 
    +supported_locales="$(cat "$rootdir/supported-locales.txt")"
    15 
    +if [[ ! "$supported_locales" =~ ^[a-zA-Z,-]*$ ]]; then
    16 
    +  echo 'Error: unexpected characters in supported-locales.txt' >&2
    17 
    +  exit 1
    18 
    +fi
    19 
    +
    20 
    +mkdir -p defaults/preferences
    21 
    +echo 'pref("torbrowser.manual.available-locales", "'"$supported_locales"'");' > defaults/preferences/tor-browser-manual.js
    15 22
    16 23 
     [% c('tar', {
    17 
    -    tar_src => [ 'chrome' ],
    24 
    +    tar_src => [ 'chrome', 'defaults' ],
    18 25 
         tar_args => '-caf ' _ dest_dir _ '/' _ c('filename'),
    19 26 
       }) %]

  • projects/manual/config
    1 1 
     # vim: filetype=yaml sw=2
    2 2 
     # To update, see doc/how-to-update-the-manual.txt
    3 3 
     # Remember to update also the package's hash, with the version!
    4 
    -version: 321660
    4 
    +version: 402523
    5 5 
     filename: 'manual-[% c("version") %]-[% c("var/build_id") %].tar.[% c("compress_tar") %]'
    6 6 
     container:
    7 7 
       use_container: 1
    ... ... @@ -21,8 +21,8 @@ var:
    21 21
    22 22 
     input_files:
    23 23 
       - project: container-image
    24 
    -  - URL: 'https://build-sources.tbb.torproject.org/manual_[% c("version") %].zip'
    24 
    +  - URL: 'https://build-sources.tbb.torproject.org/marble-support_[% c("version") %].zip'
    25 25 
         name: manual
    26 
    -    sha256sum: 9adb7d8ca2c9cb7dbd8d13d46135ea9f42636eafd053192fc5e233f2426cd8a3
    27 
    -  - filename: packagemanual.py
    26 
    +    sha256sum: e0f09c869719bf9dc0ae2761f121cae7cd9b5324f817f2cc0a5861c58442f3ff
    27 
    +  - filename: package_marble_build_artifacts.py
    28 28 
         name: package_script

  • projects/manual/packagemanual.py deleted
    1 
    -#!/usr/bin/env python3
    2 
    -from pathlib import Path
    3 
    -import re
    4 
    -import sys
    5 
    -import shutil
    6 
    -
    7 
    -
    8 
    -PAGE_NAME = 'offline-docs'
    9 
    -STATIC_PATH = 'chrome://browser/content/manual/static'
    10 
    -
    11 
    -if len(sys.argv) < 3:
    12 
    -    print(f'Usage: {sys.argv[0]} lektor-out-directory target-directory')
    13 
    -    sys.exit(1)
    14 
    -
    15 
    -source = Path(sys.argv[1])
    16 
    -target = Path(sys.argv[2])
    17 
    -if not target.exists():
    18 
    -    target.mkdir(exist_ok=True)
    19 
    -
    20 
    -static_re = re.compile('"(?:../)*static/([^"]+)"')
    21 
    -link_re = re.compile('href="">"../([^"]+)"')
    22 
    -
    23 
    -
    24 
    -def clean_urls(match):
    25 
    -    m = re.match(r'(?:../)?([^/#]+)[/]?[#]?(.*)', match.group(1))
    26 
    -    slug = m.group(1)
    27 
    -    if m.group(2):
    28 
    -        anchor = '_' + m.group(2)
    29 
    -    else:
    30 
    -        anchor = ''
    31 
    -    return f'href="">"#{slug}{anchor}"'
    32 
    -
    33 
    -
    34 
    -remove_images = []
    35 
    -for p in (source / 'static/images').glob('**/*'):
    36 
    -    if p.is_file():
    37 
    -        rel = p.relative_to(source)
    38 
    -        remove_images.append(rel)
    39 
    -
    40 
    -htmls = source.rglob(f'{PAGE_NAME}/index.html')
    41 
    -for page in htmls:
    42 
    -    with page.open(encoding='utf8') as f:
    43 
    -        contents = f.read()
    44 
    -
    45 
    -    remove_images = list(filter(
    46 
    -        lambda im: contents.find(str(im)) == -1, remove_images))
    47 
    -
    48 
    -    contents = static_re.sub(f'"{STATIC_PATH}/\\1"', contents)
    49 
    -    contents = link_re.sub(clean_urls, contents)
    50 
    -
    51 
    -    rel = page.relative_to(source)
    52 
    -    dest_name = str(list(rel.parents)[-2].name)
    53 
    -    if dest_name == PAGE_NAME:
    54 
    -        dest_name = 'en'
    55 
    -    dest_name += '.html'
    56 
    -    with (target / dest_name).open('w', encoding='utf8') as f:
    57 
    -        f.write(contents)
    58 
    -
    59 
    -
    60 
    -def ignore_images(path, names):
    61 
    -    return [
    62 
    -        n
    63 
    -        for n in names
    64 
    -        if Path(path, n).relative_to(source) in remove_images
    65 
    -    ]
    66 
    -
    67 
    -
    68 
    -shutil.rmtree(target / 'static', ignore_errors=True)
    69 
    -shutil.copytree(source / 'static', target / 'static', ignore=ignore_images)

  • tools/update_manual.py
    ... ... @@ -10,7 +10,7 @@ from fetch_changelogs import load_token, AUTH_HEADER
    10 10
    11 11 
     GITLAB = "https://gitlab.torproject.org"
    12 12 
     API_URL = f"{GITLAB}/api/v4"
    13 
    -PROJECT_ID = 23
    13 
    +PROJECT_ID = 3354
    14 14 
     REF_NAME = "main"
    15 15
    16 16
    ... ... @@ -66,7 +66,7 @@ def update_manual(auth_token, base_path):
    66 66 
             raise RuntimeError("No usable job found")
    67 67 
         pipeline_id = int(job["pipeline"]["id"])
    68 68
    69 
    -    manual_fname = f"manual_{pipeline_id}.zip"
    69 
    +    manual_fname = f"marble-support_{pipeline_id}.zip"
    70 70 
         url = f"https://build-sources.tbb.torproject.org/{manual_fname}"
    71 71 
         r = requests.head(url)
    72 72 
         needs_upload = r.status_code != 200

    • —
    View it on GitLab.
    You're receiving this email because of your account on gitlab.torproject.org. Manage all notifications · Help Notification message regarding https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/commit/8b8abf69e7d2b575611b8fc3bb7f05aad5413aad at 1785253728 

    _______________________________________________
tor-commits mailing list -- tor-commits@xxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to tor-commits-leave@xxxxxxxxxxxxxxxxxxxx