Commits:
-
89980175
by Beatriz Rizental at 2026-11-05T08:37:09+01:00
BB 45308: Implement ./mach yeet command
./mach yeet is the Tor Project version of try -- but much simpler.
-
7bf6dbd8
by Beatriz Rizental at 2026-11-05T08:37:09+01:00
fixup! BB 45308: Implement ./mach yeet command
Bug 45308: Implement ./mach yeet auth
-
fd82e86b
by Beatriz Rizental at 2026-11-05T08:37:10+01:00
fixup! BB 45308: Implement ./mach yeet command
Bug 45308: Implement ./mach yeet's upload step
Uploads a local file to the configured build server's public_html
over scp.
-
6c312932
by Beatriz Rizental at 2026-11-05T08:37:10+01:00
fixup! BB 45308: Implement ./mach yeet command
Bug 45308: Implement ./mach yeet remote's CI trigger
-
cdd4f1f1
by Beatriz Rizental at 2026-11-05T08:37:10+01:00
fixup! BB 45308: Implement ./mach yeet command
Address review comments
-
48a32237
by Beatriz Rizental at 2026-11-05T08:37:11+01:00
fixup! BB 45308: Implement ./mach yeet command
Attend more review comments.
-
f661f0d7
by Pier Angelo Vendrame at 2026-11-05T08:37:11+01:00
fixup! TB 42669: [android] Use custom no-op app-services
-
adc63271
by Pier Angelo Vendrame at 2026-11-05T08:37:11+01:00
fixup! [android] Modify build system
-
6353e646
by Beatriz Rizental at 2026-11-05T08:37:11+01:00
fixup! TB 41089: Add tor-browser build scripts + Makefile to tor-browser
This reverts commit 5f43c88992e4b69ac03c5e40aeba08fe8d0bd56e.
-
1ce02a42
by Beatriz Rizental at 2026-11-05T08:37:12+01:00
fixup! TB 43564: Modify ./mach bootstrap for Tor Browser
Move NoScript gitignore statement to this commit
-
d95babfc
by Beatriz Rizental at 2026-11-05T08:37:12+01:00
fixup! BB 45120: Add tests for Tor Browser
Bug 44820: Upstream geckoiew's own manifest requests
ACCESS_NETWORK_STATE. Tor Browser comments that line
out so geckoview doesn't ask for network-state access,
but the test app needs it. So we add it back.
-
1aaee7aa
by Beatriz Rizental at 2026-11-05T08:37:12+01:00
fixup! TB 43817: Add tests for Tor Browser
Disable Fenix unit tests by default, since most of them
don't work in Tor Browser (some don't even compile). An
allowlist in the Fenix build.gradle controls which unit
tests are compiled and run, so they can be re-enabled as
they get fixed. When the allowlist is empty, the unit test
compile and test tasks are skipped entirely.
Also add a trivial SanityTest, enabled in the allowlist,
to make sure the unit test pipeline works.
-
ac3d0ae7
by Henry Wilkes at 2026-11-05T08:37:13+01:00
fixup! TB 41600: Add a tor circuit display panel.
TB 45398: Move tor circuit button into the identity box.
30 changed files:
Changes:
.gitignore
| ... |
... |
@@ -430,14 +430,9 @@ CLAUDE.local.md |
|
430
|
430
|
# lint-ignore-next-line: git-only
|
|
431
|
431
|
/*.json.gz
|
|
432
|
432
|
|
|
433
|
|
-# Ignore binary base of Tor Browser
|
|
434
|
|
-.binaries
|
|
|
433
|
+# GitLab project and token saved by `mach yeet auth`
|
|
|
434
|
+/.yeet.json
|
|
435
|
435
|
|
|
436
|
|
-# Other files modified at build time
|
|
437
|
|
-mobile/android/.experimenter.json
|
|
438
|
|
-
|
|
439
|
|
-# Tor libraries for local builds
|
|
440
|
|
-mobile/android/fenix/tools/nimbus-fml
|
|
441
|
|
-mobile/android/fenix/app/tor-expert-bundle.aar
|
|
|
436
|
+# Symlink to NoScript .xpi. It lives in MOZBUILD, but needs to be symlinked
|
|
|
437
|
+# to this path so it's added to the APK.
|
|
442
|
438
|
mobile/android/fenix/app/src/main/assets/extensions/{73a6fe31-595d-460b-a920-fcc0f8843232}.xpi |
|
443
|
|
- |
browser/base/content/navigator-toolbox.inc.xhtml
| ... |
... |
@@ -193,19 +193,6 @@ |
|
193
|
193
|
<image id="remote-control-icon"/>
|
|
194
|
194
|
</box>
|
|
195
|
195
|
|
|
196
|
|
- <!-- We follow the surrounding code and use a
|
|
197
|
|
- - <xul:box role="button"> rather than a more semantic
|
|
198
|
|
- - <html:button>, <xul:button> or <xul:toolbarbutton> -->
|
|
199
|
|
- <box id="tor-circuit-button"
|
|
200
|
|
- role="button"
|
|
201
|
|
- class="identity-box-button"
|
|
202
|
|
- align="center"
|
|
203
|
|
- data-l10n-id="tor-circuit-urlbar-button"
|
|
204
|
|
- urlbar-slot="site-info"
|
|
205
|
|
- hidden="true">
|
|
206
|
|
- <image id="tor-circuit-button-icon"/>
|
|
207
|
|
- </box>
|
|
208
|
|
-
|
|
209
|
196
|
<box
|
|
210
|
197
|
id="trust-icon-container"
|
|
211
|
198
|
align="center"
|
| ... |
... |
@@ -247,6 +234,17 @@ |
|
247
|
234
|
class="identity-box"
|
|
248
|
235
|
pageproxystate="invalid"
|
|
249
|
236
|
urlbar-slot="site-info">
|
|
|
237
|
+ <!-- We follow the surrounding code and use a
|
|
|
238
|
+ - <xul:box role="button"> rather than a more semantic
|
|
|
239
|
+ - <html:button>, <xul:button> or <xul:toolbarbutton> -->
|
|
|
240
|
+ <box id="tor-circuit-button"
|
|
|
241
|
+ role="button"
|
|
|
242
|
+ class="identity-box-button"
|
|
|
243
|
+ align="center"
|
|
|
244
|
+ data-l10n-id="tor-circuit-urlbar-button"
|
|
|
245
|
+ hidden="true">
|
|
|
246
|
+ <image id="tor-circuit-button-icon"/>
|
|
|
247
|
+ </box>
|
|
250
|
248
|
<!-- Shown when trustPanel.featureGate is off, or for
|
|
251
|
249
|
chromeUI/extensionPage where tracker blocking never runs, so we
|
|
252
|
250
|
never update `data-l10n-args`. -->
|
browser/themes/shared/identity-block/identity-block.css
| ... |
... |
@@ -7,7 +7,6 @@ |
|
7
|
7
|
#tracking-protection-icon-container,
|
|
8
|
8
|
#identity-box,
|
|
9
|
9
|
#trust-icon-container,
|
|
10
|
|
-#tor-circuit-button,
|
|
11
|
10
|
#urlbar-label-box {
|
|
12
|
11
|
height: var(--urlbar-basic-component-size);
|
|
13
|
12
|
}
|
| ... |
... |
@@ -33,7 +32,9 @@ |
|
33
|
32
|
|
|
34
|
33
|
/* And don't show the identity icon box in this case,
|
|
35
|
34
|
just permissions indication */
|
|
36
|
|
- > #identity-icon-box {
|
|
|
35
|
+ /* Hide the tor circuit when the identity box is also hidden.
|
|
|
36
|
+ * tor-browser#45398. */
|
|
|
37
|
+ > :is(#identity-icon-box, #tor-circuit-button) {
|
|
37
|
38
|
display: none;
|
|
38
|
39
|
}
|
|
39
|
40
|
}
|
mobile/android/fenix/app/build.gradle
| ... |
... |
@@ -717,6 +717,39 @@ protobuf { |
|
717
|
717
|
}
|
|
718
|
718
|
}
|
|
719
|
719
|
|
|
|
720
|
+// Unit tests are disabled by default because most of them don't work in Tor Browser.
|
|
|
721
|
+// Add source path patterns here (e.g. 'org/mozilla/fenix/home/**') to re-enable them
|
|
|
722
|
+// as they get fixed. Only the listed tests (plus the shared helpers) are compiled and run.
|
|
|
723
|
+// When the list is empty, unit tests are neither compiled nor run.
|
|
|
724
|
+def enabledUnitTests = [
|
|
|
725
|
+ 'org/mozilla/fenix/torbrowser/**',
|
|
|
726
|
+]
|
|
|
727
|
+def unitTestHelpers = [
|
|
|
728
|
+ 'org/mozilla/fenix/helpers/**',
|
|
|
729
|
+]
|
|
|
730
|
+
|
|
|
731
|
+tasks.withType(org.jetbrains.kotlin.gradle.tasks.KotlinCompile).configureEach {
|
|
|
732
|
+ if (name.contains("UnitTest")) {
|
|
|
733
|
+ if (enabledUnitTests) {
|
|
|
734
|
+ // The task already has default include patterns, so we can only narrow it down with excludes.
|
|
|
735
|
+ def allowed = new org.gradle.api.tasks.util.PatternSet()
|
|
|
736
|
+ .include((enabledUnitTests + unitTestHelpers).collect { "**/$it" })
|
|
|
737
|
+ .asSpec
|
|
|
738
|
+ exclude { !it.directory && !allowed.isSatisfiedBy(it) }
|
|
|
739
|
+ } else {
|
|
|
740
|
+ enabled = false
|
|
|
741
|
+ }
|
|
|
742
|
+ }
|
|
|
743
|
+}
|
|
|
744
|
+
|
|
|
745
|
+tasks.withType(Test).configureEach {
|
|
|
746
|
+ if (enabledUnitTests) {
|
|
|
747
|
+ include enabledUnitTests
|
|
|
748
|
+ } else {
|
|
|
749
|
+ enabled = false
|
|
|
750
|
+ }
|
|
|
751
|
+}
|
|
|
752
|
+
|
|
720
|
753
|
if (project.hasProperty("coverage")) {
|
|
721
|
754
|
tasks.withType(Test).configureEach {
|
|
722
|
755
|
jacoco.includeNoLocationClasses = true
|
mobile/android/fenix/app/src/test/java/org/mozilla/fenix/torbrowser/SanityTest.kt
|
|
1
|
+/* This Source Code Form is subject to the terms of the Mozilla Public
|
|
|
2
|
+ * License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
|
3
|
+ * file, You can obtain one at http://mozilla.org/MPL/2.0/. */
|
|
|
4
|
+
|
|
|
5
|
+package org.mozilla.fenix.torbrowser
|
|
|
6
|
+
|
|
|
7
|
+import org.junit.Assert.assertEquals
|
|
|
8
|
+import org.junit.Test
|
|
|
9
|
+
|
|
|
10
|
+class SanityTest {
|
|
|
11
|
+ @Test
|
|
|
12
|
+ fun `one plus one is two`() {
|
|
|
13
|
+ assertEquals(2, 1 + 1)
|
|
|
14
|
+ }
|
|
|
15
|
+} |
mobile/android/geckoview/src/androidTest/AndroidManifest.xml
| ... |
... |
@@ -4,6 +4,7 @@ |
|
4
|
4
|
<uses-permission android:name="android.permission.ACCESS_COARSE_LOCATION"/>
|
|
5
|
5
|
<uses-permission android:name="android.permission.ACCESS_FINE_LOCATION"/>
|
|
6
|
6
|
<uses-permission android:name="android.permission.ACCESS_MOCK_LOCATION"/>
|
|
|
7
|
+ <uses-permission android:name="android.permission.ACCESS_NETWORK_STATE"/>
|
|
7
|
8
|
<uses-permission android:name="android.permission.CAMERA"/>
|
|
8
|
9
|
<uses-permission android:name="android.permission.FOREGROUND_SERVICE"/>
|
|
9
|
10
|
<uses-permission android:name="android.permission.READ_EXTERNAL_STORAGE"
|
python/mach/mach/command_util.py
| ... |
... |
@@ -296,6 +296,7 @@ MACH_COMMANDS = { |
|
296
|
296
|
"xpcshell-test": MachCommandReference(
|
|
297
|
297
|
"testing/xpcshell/mach_commands.py", ["test"]
|
|
298
|
298
|
),
|
|
|
299
|
+ "yeet": MachCommandReference("tools/yeet/mach_commands.py"),
|
|
299
|
300
|
}
|
|
300
|
301
|
|
|
301
|
302
|
|
tools/moz.build
| ... |
... |
@@ -75,4 +75,5 @@ PYTHON_UNITTEST_MANIFESTS += [ |
|
75
|
75
|
"lint/test/python.toml",
|
|
76
|
76
|
"tryselect/test/python.toml",
|
|
77
|
77
|
"update-packaging/test/python.toml",
|
|
|
78
|
+ "yeet/tests/python/python.toml",
|
|
78
|
79
|
] |
tools/torbrowser/Makefile
deleted
|
1
|
|
-.DEFAULT_GOAL := all
|
|
2
|
|
-
|
|
3
|
|
-# https://stackoverflow.com/questions/18136918/how-to-get-current-relative-directory-of-your-makefile
|
|
4
|
|
-mkfile_path := "$(shell dirname $(realpath $(firstword $(MAKEFILE_LIST))))"
|
|
5
|
|
-
|
|
6
|
|
-DEV_ROOT = "$(mkfile_path)/../.."
|
|
7
|
|
-BINARIES = "$(DEV_ROOT)/.binaries"
|
|
8
|
|
-ARCHITECTURE = "$(shell uname -m)"
|
|
9
|
|
-
|
|
10
|
|
-# Correct the architecture naming for ARM to match what mozilla has
|
|
11
|
|
-ifeq ($(ARCHITECTURE), "arm64")
|
|
12
|
|
- ARCHITECTURE = "aarch64"
|
|
13
|
|
-endif
|
|
14
|
|
-
|
|
15
|
|
-ifeq ($(ARCHITECTURE), "aarch64")
|
|
16
|
|
- LINUX_VENDOR = "unknown"
|
|
17
|
|
-else
|
|
18
|
|
- LINUX_VENDOR = "pc"
|
|
19
|
|
-endif
|
|
20
|
|
-
|
|
21
|
|
-# Define build output path based on the platform.
|
|
22
|
|
-ifeq ("$(shell uname)", "Darwin")
|
|
23
|
|
- BUILD_OUTPUT = "$(DEV_ROOT)/obj-$(ARCHITECTURE)-apple-darwin$(shell uname -r)"
|
|
24
|
|
-else
|
|
25
|
|
- BUILD_OUTPUT = "$(DEV_ROOT)/obj-$(ARCHITECTURE)-$(LINUX_VENDOR)-linux-gnu"
|
|
26
|
|
-endif
|
|
27
|
|
-
|
|
28
|
|
-# Define the run command based on the platform.
|
|
29
|
|
-ifeq ("$(shell uname)", "Darwin")
|
|
30
|
|
- RUN_CMD := cd "$(BINARIES)/Tor Browser.app/Contents/MacOS/" && ./firefox --purgecaches
|
|
31
|
|
-else
|
|
32
|
|
- RUN_CMD := "$(BINARIES)/dev/Browser/start-tor-browser" -v --purgecaches $(ARGS)
|
|
33
|
|
-endif
|
|
34
|
|
-
|
|
35
|
|
-config:
|
|
36
|
|
- ./config.sh $(DEV_ROOT)
|
|
37
|
|
-
|
|
38
|
|
-ide-vscode:
|
|
39
|
|
- ./ide.sh vscode $(DEV_ROOT)
|
|
40
|
|
-
|
|
41
|
|
-ide-eclipse:
|
|
42
|
|
- ./ide.sh eclipse $(DEV_ROOT)
|
|
43
|
|
-
|
|
44
|
|
-ide-visualstudio:
|
|
45
|
|
- ./ide.sh visualstudio $(DEV_ROOT)
|
|
46
|
|
-
|
|
47
|
|
-fetch:
|
|
48
|
|
- ./fetch.sh $(BINARIES)
|
|
49
|
|
-
|
|
50
|
|
-build:
|
|
51
|
|
- ./build.sh $(DEV_ROOT)
|
|
52
|
|
-
|
|
53
|
|
-deploy:
|
|
54
|
|
- ./deploy.sh $(BINARIES) $(BUILD_OUTPUT)
|
|
55
|
|
-
|
|
56
|
|
-prepare-tests:
|
|
57
|
|
- ./prepare-tests.sh $(BINARIES) $(BUILD_OUTPUT)
|
|
58
|
|
-
|
|
59
|
|
-all: build deploy
|
|
60
|
|
-
|
|
61
|
|
-run:
|
|
62
|
|
- $(RUN_CMD)
|
|
63
|
|
-
|
|
64
|
|
-jslint:
|
|
65
|
|
- ./jslint.sh $(DEV_ROOT) $(JS)
|
|
66
|
|
-
|
|
67
|
|
-clobber:
|
|
68
|
|
- ./clobber.sh $(DEV_ROOT)
|
|
69
|
|
-
|
|
70
|
|
-clean:
|
|
71
|
|
- rm -rf $(BUILD_OUTPUT)
|
|
72
|
|
- |
tools/torbrowser/browser-self-sign-macos.sh
deleted
|
1
|
|
-#!/bin/bash
|
|
2
|
|
-
|
|
3
|
|
-CERTNAME=my-codesign-cert-tor
|
|
4
|
|
-BROWSERPATH=.
|
|
5
|
|
-
|
|
6
|
|
-if [ $# -ge 1 ]
|
|
7
|
|
-then
|
|
8
|
|
- BROWSERPATH=$1
|
|
9
|
|
-fi
|
|
10
|
|
-
|
|
11
|
|
-
|
|
12
|
|
-security find-certificate -c $CERTNAME > /dev/null
|
|
13
|
|
-
|
|
14
|
|
-if [ $? -ne 0 ]
|
|
15
|
|
-then
|
|
16
|
|
- echo ""
|
|
17
|
|
- echo "ERROR: Self Signing Certificate not found, please create:"
|
|
18
|
|
- echo " 1. In the Keychain Access app on your Mac, choose Keychain Access > Certificate Assistant > Create a Certificate."
|
|
19
|
|
- echo " 2. Enter the name '$CERTNAME' for the certificate"
|
|
20
|
|
- echo " 3. Choose an identity type: Self Signed Root"
|
|
21
|
|
- echo " 4. Certificate Type > Code Signing"
|
|
22
|
|
- echo " 5. Check 'Let me override defaults' & click Continue."
|
|
23
|
|
- echo " 6. Enter a unique Serial Number. (123 is fine)"
|
|
24
|
|
- echo " 7. Enter a big Validity Period (days), like 3560 & click Continue."
|
|
25
|
|
- echo " 8. Fill in your personal information & click Continue."
|
|
26
|
|
- echo " 9. Accept defaults for the rest of the dialog boxes. (Continue several times)"
|
|
27
|
|
- echo " 10. Certificate Created! Click Done."
|
|
28
|
|
- echo ""
|
|
29
|
|
- echo "For additional help see:"
|
|
30
|
|
- echo " https://support.apple.com/en-ca/guide/keychain-access/kyca8916/mac"
|
|
31
|
|
- echo " https://stackoverflow.com/questions/58356844/what-are-the-ways-or-technologies-to-sign-an-executable-application-file-in-mac"
|
|
32
|
|
-
|
|
33
|
|
- echo ""
|
|
34
|
|
- read -n 1 -r -s -p $'Press enter to launch "Keychain Access"...\n'
|
|
35
|
|
- open /System/Applications/Utilities/Keychain\ Access.app
|
|
36
|
|
-
|
|
37
|
|
- exit -1
|
|
38
|
|
-fi
|
|
39
|
|
-
|
|
40
|
|
-echo "Found $CERTNAME, looking for browser to sign..."
|
|
41
|
|
-
|
|
42
|
|
-if [ ! -f "$BROWSERPATH/XUL" ]
|
|
43
|
|
-then
|
|
44
|
|
- TESTPATH="$BROWSERPATH/Contents/MacOS"
|
|
45
|
|
- if [ -f "$TESTPATH/XUL" ]
|
|
46
|
|
- then
|
|
47
|
|
- BROWSERPATH=$TESTPATH
|
|
48
|
|
- else
|
|
49
|
|
- echo "Error: browser files not detected in $BROWSERPATH!"
|
|
50
|
|
- echo " This script needs to be run in the 'Contents/MacOS' directory of a SomeBrowser.app directory"
|
|
51
|
|
- exit -1
|
|
52
|
|
- fi
|
|
53
|
|
-fi
|
|
54
|
|
-
|
|
55
|
|
-echo "Mozilla based browser found, signing..."
|
|
56
|
|
-echo ' Will be asked for password to certificate for all the things that need to be signed. Click "Always Allow" to automate'
|
|
57
|
|
-
|
|
58
|
|
-cd "$BROWSERPATH"
|
|
59
|
|
-
|
|
60
|
|
-codesign -s $CERTNAME *.dylib
|
|
61
|
|
-codesign -s $CERTNAME plugin-container.app
|
|
62
|
|
-
|
|
63
|
|
-if [ -d Tor ]
|
|
64
|
|
-then
|
|
65
|
|
- codesign -s $CERTNAME Tor/PluggableTransports/*
|
|
66
|
|
- codesign -s $CERTNAME Tor/libevent-2.1.7.dylib
|
|
67
|
|
- if [ -f Tor/tor.real ]
|
|
68
|
|
- then
|
|
69
|
|
- codesign -s $CERTNAME Tor/tor.real
|
|
70
|
|
- fi
|
|
71
|
|
- if [ -f Tor/tor ]
|
|
72
|
|
- then
|
|
73
|
|
- codesign -s $CERTNAME Tor/tor
|
|
74
|
|
- fi
|
|
75
|
|
-fi
|
|
76
|
|
-
|
|
77
|
|
-codesign -s $CERTNAME XUL
|
|
78
|
|
-
|
|
79
|
|
-if [ -d updater.app ]
|
|
80
|
|
-then
|
|
81
|
|
- codesign -s $CERTNAME updater.app
|
|
82
|
|
-fi
|
|
83
|
|
-
|
|
84
|
|
-# mullvadbrowser
|
|
85
|
|
-if [ -f mullvadbrowser ]
|
|
86
|
|
-then
|
|
87
|
|
- codesign -s $CERTNAME mullvadbrowser
|
|
88
|
|
-fi
|
|
89
|
|
-
|
|
90
|
|
-# BB or TB
|
|
91
|
|
-if [ -f firefox ]
|
|
92
|
|
-then
|
|
93
|
|
- codesign -s $CERTNAME firefox
|
|
94
|
|
-fi
|
|
95
|
|
-
|
|
96
|
|
-echo ""
|
|
97
|
|
-echo "Browser signing step done!"
|
|
98
|
|
-echo ""
|
|
99
|
|
-
|
|
100
|
|
-echo "App still needs one more override to be easily opened with double click in Finder"
|
|
101
|
|
-echo "Alternatively you can right click it, select 'Open' and then select 'Open' from the override popup"
|
|
102
|
|
-echo "Or to enable it to be double clicked to open perform the following"
|
|
103
|
|
-echo ""
|
|
104
|
|
-echo "Double click the app and select either 'Ok' or 'Cancel' from the warning popup depending on which you get (Do Not 'Move to Trash')"
|
|
105
|
|
-echo 'Go to Preferences -> Security & Privacy and click on padlock to allow changes. '
|
|
106
|
|
-echo ' Then in "Allow appications downloaded from" select either:'
|
|
107
|
|
-echo ' - App Store and identified developers'
|
|
108
|
|
-echo ' - Anywhere'
|
|
109
|
|
-echo ' Below that may be a notice about your specific app saying it was blocked because it was not from an identified developer. Click "Open Anyways" and "Open"'
|
|
110
|
|
- |
tools/torbrowser/build.sh
deleted
|
1
|
|
-#!/bin/bash
|
|
2
|
|
-set -e
|
|
3
|
|
-DEV_ROOT=$1
|
|
4
|
|
-
|
|
5
|
|
-cd $DEV_ROOT
|
|
6
|
|
-./mach build
|
|
7
|
|
-
|
|
8
|
|
-if [ -z "$LOCALES" ]; then
|
|
9
|
|
- ./mach build stage-package
|
|
10
|
|
-else
|
|
11
|
|
- export MOZ_CHROME_MULTILOCALE=$LOCALES
|
|
12
|
|
- # No quotes on purpose
|
|
13
|
|
- ./mach package-multi-locale --locales en-US $MOZ_CHROME_MULTILOCALE
|
|
14
|
|
- AB_CD=multi ./mach build stage-package
|
|
15
|
|
-fi |
tools/torbrowser/clobber.sh
deleted
|
1
|
|
-#!/bin/bash
|
|
2
|
|
-set -e
|
|
3
|
|
-DEV_ROOT=$1
|
|
4
|
|
-
|
|
5
|
|
-cd $DEV_ROOT
|
|
6
|
|
-./mach clobber |
tools/torbrowser/config.sh
deleted
|
1
|
|
-#!/bin/bash
|
|
2
|
|
-set -e
|
|
3
|
|
-DEV_ROOT=$1
|
|
4
|
|
-
|
|
5
|
|
-cd $DEV_ROOT
|
|
6
|
|
-./mach configure |
tools/torbrowser/deploy.sh
deleted
|
1
|
|
-#!/bin/bash
|
|
2
|
|
-set -e
|
|
3
|
|
-
|
|
4
|
|
-BINARIES="$1"
|
|
5
|
|
-BUILD_OUTPUT="$2"
|
|
6
|
|
-SCRIPT_DIR="$(realpath "$(dirname "$0")")"
|
|
7
|
|
-
|
|
8
|
|
-RESDIR="$BUILD_OUTPUT/dist/firefox"
|
|
9
|
|
-if [ "$(uname)" = "Darwin" ]; then
|
|
10
|
|
- RESDIR="$RESDIR/Tor Browser.app/Contents/Resources"
|
|
11
|
|
-fi
|
|
12
|
|
-
|
|
13
|
|
-# Repackage the manual
|
|
14
|
|
-# rm -rf $BUILD_OUTPUT/_omni
|
|
15
|
|
-# mkdir $BUILD_OUTPUT/_omni
|
|
16
|
|
-# unzip $BINARIES/dev/Browser/browser/omni.ja -d $BUILD_OUTPUT/_omni
|
|
17
|
|
-# cd $BUILD_OUTPUT/_omni && zip -Xmr $RESDIR/browser/omni.ja chrome/browser/content/browser/manual
|
|
18
|
|
-# rm -rf $BUILD_OUTPUT/_omni
|
|
19
|
|
-
|
|
20
|
|
-if [ "$(uname)" = "Darwin" ]; then
|
|
21
|
|
-
|
|
22
|
|
- # copy binaries
|
|
23
|
|
- cp -r "$BUILD_OUTPUT/dist/firefox/"*.app/Contents/* "$BINARIES/Tor Browser.app/Contents/"
|
|
24
|
|
- rm -rf "$BINARIES/TorBrowser-Data/Browser/Caches/*.default/startupCache"
|
|
25
|
|
-
|
|
26
|
|
- # Self sign the Binaries
|
|
27
|
|
- cd "$BINARIES/Tor Browser.app/Contents/MacOS"
|
|
28
|
|
- "$SCRIPT_DIR/browser-self-sign-macos.sh"
|
|
29
|
|
-
|
|
30
|
|
-else
|
|
31
|
|
-
|
|
32
|
|
- # backup the startup script
|
|
33
|
|
- mv "$BINARIES/dev/Browser/firefox" "$BINARIES/dev/Browser/firefox.bak"
|
|
34
|
|
-
|
|
35
|
|
- # copy binaries
|
|
36
|
|
- cp -r "$RESDIR/"* "$BINARIES/dev/Browser"
|
|
37
|
|
- rm -rf "$BINARIES/dev/Browser/TorBrowser/Data/Browser/profile.default/startupCache"
|
|
38
|
|
-
|
|
39
|
|
- # shuffle firefox bin around and restore script to match a real deployment
|
|
40
|
|
- mv "$BINARIES/dev/Browser/firefox" "$BINARIES/dev/Browser/firefox.real"
|
|
41
|
|
- mv "$BINARIES/dev/Browser/firefox.bak" "$BINARIES/dev/Browser/firefox"
|
|
42
|
|
-
|
|
43
|
|
-fi |
tools/torbrowser/fetch.sh
deleted
|
1
|
|
-#!/bin/sh
|
|
2
|
|
-set -e
|
|
3
|
|
-
|
|
4
|
|
-BINARIES_DIR="$1"
|
|
5
|
|
-
|
|
6
|
|
-# download the current downloads.json
|
|
7
|
|
-wget https://aus1.torproject.org/torbrowser/update_3/alpha/downloads.json
|
|
8
|
|
-# get url for latest alpha linux package
|
|
9
|
|
-TOR_BROWSER_VERSION=$(grep -Eo "\"version\"\s*:\s*\"[0-9.a]+\"" downloads.json | grep -Eo "[0-9.a]+")
|
|
10
|
|
-if [ "$(uname)" = "Darwin" ]; then
|
|
11
|
|
- TOR_BROWSER_PACKAGE="tor-browser-macos-${TOR_BROWSER_VERSION}.dmg"
|
|
12
|
|
- else
|
|
13
|
|
- TOR_BROWSER_PACKAGE="tor-browser-linux-$(uname -m)-${TOR_BROWSER_VERSION}.tar.xz"
|
|
14
|
|
-fi
|
|
15
|
|
-TOR_BROWSER_PACKAGE_URL="https://dist.torproject.org/torbrowser/${TOR_BROWSER_VERSION}/${TOR_BROWSER_PACKAGE}"
|
|
16
|
|
-
|
|
17
|
|
-# remove download manifest
|
|
18
|
|
-rm downloads.json
|
|
19
|
|
-
|
|
20
|
|
-# clear out previous tor-browser and previous package
|
|
21
|
|
-rm -rf "${BINARIES_DIR}"
|
|
22
|
|
-rm -f "${TOR_BROWSER_PACKAGE}"
|
|
23
|
|
-
|
|
24
|
|
-# download
|
|
25
|
|
-wget "${TOR_BROWSER_PACKAGE_URL}"
|
|
26
|
|
-mkdir -p "${BINARIES_DIR}"
|
|
27
|
|
-
|
|
28
|
|
-# and extract
|
|
29
|
|
-if [ "$(uname)" = "Darwin" ]
|
|
30
|
|
- then
|
|
31
|
|
- hdiutil attach "${TOR_BROWSER_PACKAGE}"
|
|
32
|
|
- cp -R "/Volumes/Tor Browser Alpha/Tor Browser Alpha.app" "${BINARIES_DIR}/Tor Browser.app"
|
|
33
|
|
- hdiutil detach "/Volumes/Tor Browser Alpha"
|
|
34
|
|
- else
|
|
35
|
|
- tar -xf "${TOR_BROWSER_PACKAGE}" -C "${BINARIES_DIR}"
|
|
36
|
|
- mv "${BINARIES_DIR}/tor-browser" "${BINARIES_DIR}/dev"
|
|
37
|
|
-fi
|
|
38
|
|
-
|
|
39
|
|
-# Final cleanup
|
|
40
|
|
-rm -f "${TOR_BROWSER_PACKAGE}" |
tools/torbrowser/ide.sh
deleted
|
1
|
|
-#!/bin/bash
|
|
2
|
|
-set -e
|
|
3
|
|
-IDE=$1
|
|
4
|
|
-DEV_ROOT=$2
|
|
5
|
|
-
|
|
6
|
|
-cd $DEV_ROOT
|
|
7
|
|
-./mach ide $IDE |
tools/torbrowser/jslint.sh
deleted
|
1
|
|
-#!/bin/bash
|
|
2
|
|
-set -e
|
|
3
|
|
-DEV_ROOT=$1
|
|
4
|
|
-JS_FILE=$2
|
|
5
|
|
-
|
|
6
|
|
-cd $DEV_ROOT
|
|
7
|
|
-./mach lint -l eslint --fix $JS_FILE |
tools/torbrowser/prepare-tests.sh
deleted
|
1
|
|
-#!/bin/sh
|
|
2
|
|
-set -e
|
|
3
|
|
-
|
|
4
|
|
-BINARIES="$1"
|
|
5
|
|
-BUILD_OUTPUT="$2"
|
|
6
|
|
-
|
|
7
|
|
-if [ ! -d "$BUILD_OUTPUT" ]; then
|
|
8
|
|
- echo "Error: $BUILD_OUTPUT directory does not exist."
|
|
9
|
|
- echo "Make sure to run `mach ./build` or `make -C tools/torbrowser build`."
|
|
10
|
|
- exit 1
|
|
11
|
|
-fi
|
|
12
|
|
-
|
|
13
|
|
-if [ ! -d "$BINARIES" ]; then
|
|
14
|
|
- echo "Error: $BINARIES directory does not exist."
|
|
15
|
|
- echo "Make sure to run `make -C tools/torbrowser fetch`."
|
|
16
|
|
- exit 1
|
|
17
|
|
-fi
|
|
18
|
|
-
|
|
19
|
|
-if [ "$(uname)" = "Darwin" ]; then
|
|
20
|
|
- cp -r "$BINARIES/Tor Browser.app/Contents/MacOS/Tor" "$BUILD_OUTPUT/dist/firefox/"*.app/Contents/MacOS
|
|
21
|
|
- cp -r "$BINARIES/Tor Browser.app/Contents/Resources/fonts" "$BUILD_OUTPUT/dist/firefox/"*.app/Contents/Resources
|
|
22
|
|
-else
|
|
23
|
|
- cp -r "$BINARIES/dev/Browser/fonts" "$BUILD_OUTPUT/dist/bin"
|
|
24
|
|
- cp -r "$BINARIES/dev/Browser/TorBrowser" "$BUILD_OUTPUT/dist/bin"
|
|
25
|
|
-fi |
tools/yeet/__init__.py
|
|
1
|
+# This Source Code Form is subject to the terms of the Mozilla Public
|
|
|
2
|
+# License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
|
3
|
+# file, You can obtain one at http://mozilla.org/MPL/2.0/. |
tools/yeet/auth.py
|
|
1
|
+# This Source Code Form is subject to the terms of the Mozilla Public
|
|
|
2
|
+# License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
|
3
|
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
|
4
|
+
|
|
|
5
|
+import json
|
|
|
6
|
+import os
|
|
|
7
|
+import subprocess
|
|
|
8
|
+import urllib.error
|
|
|
9
|
+import urllib.parse
|
|
|
10
|
+import urllib.request
|
|
|
11
|
+from pathlib import Path
|
|
|
12
|
+
|
|
|
13
|
+from yeet.common import (
|
|
|
14
|
+ CANONICAL_GITLAB_PROJECT,
|
|
|
15
|
+ CANONICAL_GITLAB_REPO_NAME,
|
|
|
16
|
+ DEFAULT_SSH_HOST,
|
|
|
17
|
+ GITLAB_API_BASE,
|
|
|
18
|
+ TERM,
|
|
|
19
|
+ YEET_CONFIG_FILENAME,
|
|
|
20
|
+ prompt,
|
|
|
21
|
+)
|
|
|
22
|
+
|
|
|
23
|
+
|
|
|
24
|
+def _resolve_project(raw_username):
|
|
|
25
|
+ """Build a fork project path out of a GitLab username. Raises ValueError
|
|
|
26
|
+ if it's empty or the canonical repo's own namespace.
|
|
|
27
|
+ """
|
|
|
28
|
+ username = raw_username.strip()
|
|
|
29
|
+ if not username:
|
|
|
30
|
+ raise ValueError("A GitLab username is required.")
|
|
|
31
|
+
|
|
|
32
|
+ project = f"{username}/{CANONICAL_GITLAB_REPO_NAME}"
|
|
|
33
|
+ if project.lower() == CANONICAL_GITLAB_PROJECT.lower():
|
|
|
34
|
+ raise ValueError(
|
|
|
35
|
+ "That's the canonical repo's namespace, not a fork. Use your "
|
|
|
36
|
+ "own GitLab username instead."
|
|
|
37
|
+ )
|
|
|
38
|
+
|
|
|
39
|
+ return project
|
|
|
40
|
+
|
|
|
41
|
+
|
|
|
42
|
+def _prompt_fork():
|
|
|
43
|
+ print()
|
|
|
44
|
+ print(
|
|
|
45
|
+ TERM.yellow(
|
|
|
46
|
+ f"""`mach yeet remote` runs against your own fork of
|
|
|
47
|
+{CANONICAL_GITLAB_PROJECT}, never against the canonical repo
|
|
|
48
|
+directly -- that keeps CI runs off the shared project.
|
|
|
49
|
+
|
|
|
50
|
+If you haven't forked it yet, fork it first at:
|
|
|
51
|
+ https://gitlab.torproject.org/{CANONICAL_GITLAB_PROJECT}/-/forks/new
|
|
|
52
|
+
|
|
|
53
|
+Keep your fork up to date too. An outdated fork's .gitlab-ci.yml may be missing
|
|
|
54
|
+pieces this command relies on, so pipelines could fail or come back empty."""
|
|
|
55
|
+ )
|
|
|
56
|
+ )
|
|
|
57
|
+ print()
|
|
|
58
|
+
|
|
|
59
|
+ while True:
|
|
|
60
|
+ try:
|
|
|
61
|
+ return _resolve_project(prompt("Your GitLab username: "))
|
|
|
62
|
+ except ValueError as e:
|
|
|
63
|
+ print(TERM.red(f"ERROR! {e}"))
|
|
|
64
|
+
|
|
|
65
|
+
|
|
|
66
|
+def _gitlab_token_instructions(project):
|
|
|
67
|
+ return f"""
|
|
|
68
|
+Now you need a pipeline trigger token for your fork:
|
|
|
69
|
+ {project}
|
|
|
70
|
+
|
|
|
71
|
+This is NOT a personal access token -- it's specific to this one project, and
|
|
|
72
|
+it's the only kind that can actually run CI jobs here (their rules require it):
|
|
|
73
|
+
|
|
|
74
|
+ 1. Go to https://gitlab.torproject.org/{project}/-/settings/ci_cd#js-pipeline-triggers
|
|
|
75
|
+ 2. Expand "Pipeline trigger tokens" and add a new one.
|
|
|
76
|
+ 3. Paste the token below.
|
|
|
77
|
+
|
|
|
78
|
+The project and token will be saved to {YEET_CONFIG_FILENAME} at the root of the repository
|
|
|
79
|
+and are not committed to it.
|
|
|
80
|
+""".strip()
|
|
|
81
|
+
|
|
|
82
|
+
|
|
|
83
|
+def _print_ssh_instructions():
|
|
|
84
|
+ print()
|
|
|
85
|
+ print(
|
|
|
86
|
+ TERM.yellow(
|
|
|
87
|
+ f"""`mach yeet` also uploads to and runs commands on a build server over SSH
|
|
|
88
|
+({DEFAULT_SSH_HOST} by default). Uploaded files are served from your
|
|
|
89
|
+account's public_html there, at:
|
|
|
90
|
+ https://<host>/~<username>
|
|
|
91
|
+
|
|
|
92
|
+Anything special about the connection -- port, jump host, a hardware key's
|
|
|
93
|
+PKCS11 provider, agent forwarding -- belongs in your ~/.ssh/config, not here:
|
|
|
94
|
+we just shell out to `ssh`. Whatever already works on your command line works
|
|
|
95
|
+for `mach yeet` too."""
|
|
|
96
|
+ )
|
|
|
97
|
+ )
|
|
|
98
|
+ print()
|
|
|
99
|
+
|
|
|
100
|
+
|
|
|
101
|
+def _prompt_ssh_host():
|
|
|
102
|
+ return prompt(f"SSH host [{DEFAULT_SSH_HOST}]: ").strip() or DEFAULT_SSH_HOST
|
|
|
103
|
+
|
|
|
104
|
+
|
|
|
105
|
+def _prompt_ssh_username(ssh_host):
|
|
|
106
|
+ while True:
|
|
|
107
|
+ username = prompt(f"Your username on {ssh_host}: ").strip()
|
|
|
108
|
+ if username:
|
|
|
109
|
+ return username
|
|
|
110
|
+ print(TERM.red("ERROR! Username cannot be empty."))
|
|
|
111
|
+
|
|
|
112
|
+
|
|
|
113
|
+def _prompt_token():
|
|
|
114
|
+ while True:
|
|
|
115
|
+ token = prompt("GitLab token: ", secret=True).strip()
|
|
|
116
|
+ if token:
|
|
|
117
|
+ return token
|
|
|
118
|
+ print(TERM.red("ERROR! Token cannot be empty."))
|
|
|
119
|
+
|
|
|
120
|
+
|
|
|
121
|
+def _verify_ssh(target):
|
|
|
122
|
+ """Try an actual SSH connection. stdin is /dev/null so ssh can't swallow
|
|
|
123
|
+ input buffered for us; its touch/PIN/passphrase prompts go through
|
|
|
124
|
+ /dev/tty and still reach the user.
|
|
|
125
|
+ """
|
|
|
126
|
+ try:
|
|
|
127
|
+ result = subprocess.run(
|
|
|
128
|
+ ["ssh", target, "true"],
|
|
|
129
|
+ stdin=subprocess.DEVNULL,
|
|
|
130
|
+ check=False,
|
|
|
131
|
+ )
|
|
|
132
|
+ except FileNotFoundError:
|
|
|
133
|
+ print()
|
|
|
134
|
+ print(TERM.red("ERROR! No `ssh` binary found on PATH."))
|
|
|
135
|
+ return False
|
|
|
136
|
+ return result.returncode == 0
|
|
|
137
|
+
|
|
|
138
|
+
|
|
|
139
|
+def _verify_auth(project, token):
|
|
|
140
|
+ """Check that `project`/`token` are accepted by GitLab, without
|
|
|
141
|
+ actually triggering a pipeline.
|
|
|
142
|
+
|
|
|
143
|
+ GitLab authenticates the trigger token before validating the rest of
|
|
|
144
|
+ the request, so posting to the trigger endpoint without the required
|
|
|
145
|
+ `ref` param never queues a pipeline: a valid token/project combination
|
|
|
146
|
+ gets rejected with 400 (missing ref) instead, while an invalid token
|
|
|
147
|
+ or unknown project is rejected with 401/404 before `ref` is even
|
|
|
148
|
+ looked at.
|
|
|
149
|
+ """
|
|
|
150
|
+ url = f"{GITLAB_API_BASE}/projects/{urllib.parse.quote(project, safe='')}/trigger/pipeline"
|
|
|
151
|
+ data = urllib.parse.urlencode({"token": token}).encode()
|
|
|
152
|
+ request = urllib.request.Request(url, data=data, method="POST")
|
|
|
153
|
+ try:
|
|
|
154
|
+ with urllib.request.urlopen(request):
|
|
|
155
|
+ pass
|
|
|
156
|
+ except urllib.error.HTTPError as e:
|
|
|
157
|
+ return e.code == 400
|
|
|
158
|
+ else:
|
|
|
159
|
+ return True
|
|
|
160
|
+
|
|
|
161
|
+
|
|
|
162
|
+def run(command_context, **kwargs):
|
|
|
163
|
+ gitlab_username = kwargs.get("gitlab_username")
|
|
|
164
|
+ if gitlab_username is not None:
|
|
|
165
|
+ try:
|
|
|
166
|
+ project = _resolve_project(gitlab_username)
|
|
|
167
|
+ except ValueError as e:
|
|
|
168
|
+ print(TERM.red(f"ERROR! --gitlab-username: {e}"))
|
|
|
169
|
+ return 1
|
|
|
170
|
+ else:
|
|
|
171
|
+ project = _prompt_fork()
|
|
|
172
|
+
|
|
|
173
|
+ gitlab_token = kwargs.get("gitlab_token")
|
|
|
174
|
+ if gitlab_token is not None:
|
|
|
175
|
+ token = gitlab_token.strip()
|
|
|
176
|
+ if not token:
|
|
|
177
|
+ print()
|
|
|
178
|
+ print(TERM.red("ERROR! --gitlab-token was empty."))
|
|
|
179
|
+ return 1
|
|
|
180
|
+ else:
|
|
|
181
|
+ print()
|
|
|
182
|
+ print(_gitlab_token_instructions(project))
|
|
|
183
|
+ print()
|
|
|
184
|
+ token = _prompt_token()
|
|
|
185
|
+
|
|
|
186
|
+ verify = kwargs.get("verify", True)
|
|
|
187
|
+ if verify:
|
|
|
188
|
+ print()
|
|
|
189
|
+ print("Verifying credentials with GitLab...")
|
|
|
190
|
+ try:
|
|
|
191
|
+ verified = _verify_auth(project, token)
|
|
|
192
|
+ except urllib.error.URLError as e:
|
|
|
193
|
+ print()
|
|
|
194
|
+ print(TERM.red(f"ERROR! Could not reach GitLab: {e}"))
|
|
|
195
|
+ return 1
|
|
|
196
|
+
|
|
|
197
|
+ if not verified:
|
|
|
198
|
+ print()
|
|
|
199
|
+ print(
|
|
|
200
|
+ TERM.red(
|
|
|
201
|
+ "ERROR! GitLab rejected that project/token combination. "
|
|
|
202
|
+ "Double-check the fork name and the trigger token."
|
|
|
203
|
+ )
|
|
|
204
|
+ )
|
|
|
205
|
+ return 1
|
|
|
206
|
+ print(TERM.green("Credentials verified."))
|
|
|
207
|
+
|
|
|
208
|
+ ssh_host = kwargs.get("ssh_host")
|
|
|
209
|
+ ssh_username = kwargs.get("ssh_username")
|
|
|
210
|
+ if ssh_host is None or ssh_username is None:
|
|
|
211
|
+ _print_ssh_instructions()
|
|
|
212
|
+
|
|
|
213
|
+ if ssh_host is not None:
|
|
|
214
|
+ ssh_host = ssh_host.strip()
|
|
|
215
|
+ if not ssh_host:
|
|
|
216
|
+ print()
|
|
|
217
|
+ print(TERM.red("ERROR! --ssh-host was empty."))
|
|
|
218
|
+ return 1
|
|
|
219
|
+ else:
|
|
|
220
|
+ ssh_host = _prompt_ssh_host()
|
|
|
221
|
+
|
|
|
222
|
+ if ssh_username is not None:
|
|
|
223
|
+ ssh_username = ssh_username.strip()
|
|
|
224
|
+ if not ssh_username:
|
|
|
225
|
+ print()
|
|
|
226
|
+ print(TERM.red("ERROR! --ssh-user was empty."))
|
|
|
227
|
+ return 1
|
|
|
228
|
+ else:
|
|
|
229
|
+ ssh_username = _prompt_ssh_username(ssh_host)
|
|
|
230
|
+
|
|
|
231
|
+ if verify:
|
|
|
232
|
+ ssh_target = f"{ssh_username}@{ssh_host}"
|
|
|
233
|
+ print()
|
|
|
234
|
+ print(f"Verifying SSH access to {ssh_target}...")
|
|
|
235
|
+ if not _verify_ssh(ssh_target):
|
|
|
236
|
+ print()
|
|
|
237
|
+ print(
|
|
|
238
|
+ TERM.red(
|
|
|
239
|
+ f"ERROR! Could not SSH into {ssh_target}. Make sure you "
|
|
|
240
|
+ "can `ssh` there yourself first."
|
|
|
241
|
+ )
|
|
|
242
|
+ )
|
|
|
243
|
+ return 1
|
|
|
244
|
+ print(TERM.green("SSH access verified."))
|
|
|
245
|
+
|
|
|
246
|
+ tokens_path = Path(command_context.topsrcdir) / YEET_CONFIG_FILENAME
|
|
|
247
|
+ file_desc = os.open(
|
|
|
248
|
+ tokens_path, flags=os.O_WRONLY | os.O_CREAT | os.O_TRUNC, mode=0o600
|
|
|
249
|
+ )
|
|
|
250
|
+ with open(file_desc, "w") as f:
|
|
|
251
|
+ json.dump(
|
|
|
252
|
+ {
|
|
|
253
|
+ "project": project,
|
|
|
254
|
+ "token": token,
|
|
|
255
|
+ "ssh_host": ssh_host,
|
|
|
256
|
+ "ssh_user": ssh_username,
|
|
|
257
|
+ },
|
|
|
258
|
+ f,
|
|
|
259
|
+ )
|
|
|
260
|
+ os.chmod(tokens_path, 0o600)
|
|
|
261
|
+
|
|
|
262
|
+ print()
|
|
|
263
|
+ print(TERM.green(f"Saved to {tokens_path}"))
|
|
|
264
|
+ return 0 |
tools/yeet/common.py
|
|
1
|
+# This Source Code Form is subject to the terms of the Mozilla Public
|
|
|
2
|
+# License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
|
3
|
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
|
4
|
+
|
|
|
5
|
+import getpass
|
|
|
6
|
+import json
|
|
|
7
|
+import sys
|
|
|
8
|
+from pathlib import Path
|
|
|
9
|
+
|
|
|
10
|
+from mozterm import Terminal
|
|
|
11
|
+
|
|
|
12
|
+TERM = Terminal()
|
|
|
13
|
+
|
|
|
14
|
+GITLAB_API_BASE = "https://gitlab.torproject.org/api/v4"
|
|
|
15
|
+
|
|
|
16
|
+# The canonical repo. `mach yeet remote` must never target this directly --
|
|
|
17
|
+# only a fork of it, so CI runs don't pile up on the shared project.
|
|
|
18
|
+CANONICAL_GITLAB_NAMESPACE = "tpo/applications"
|
|
|
19
|
+CANONICAL_GITLAB_REPO_NAME = "tor-browser-bundle-testsuite"
|
|
|
20
|
+CANONICAL_GITLAB_PROJECT = f"{CANONICAL_GITLAB_NAMESPACE}/{CANONICAL_GITLAB_REPO_NAME}"
|
|
|
21
|
+
|
|
|
22
|
+# Project and token saved by `mach yeet auth`.
|
|
|
23
|
+YEET_CONFIG_FILENAME = ".yeet.json"
|
|
|
24
|
+
|
|
|
25
|
+# The default server `mach yeet` uploads to and runs commands on over SSH.
|
|
|
26
|
+# Uploaded files are served from the user's public_html on it.
|
|
|
27
|
+DEFAULT_SSH_HOST = "tb-build-03.torproject.org"
|
|
|
28
|
+
|
|
|
29
|
+
|
|
|
30
|
+def prompt(message, secret=False):
|
|
|
31
|
+ """Read a line of user input, first discarding anything typed before the
|
|
|
32
|
+ prompt appeared so stray keystrokes aren't taken as the answer.
|
|
|
33
|
+ """
|
|
|
34
|
+ if sys.stdin.isatty():
|
|
|
35
|
+ try:
|
|
|
36
|
+ import termios
|
|
|
37
|
+
|
|
|
38
|
+ termios.tcflush(sys.stdin, termios.TCIFLUSH)
|
|
|
39
|
+ except ImportError:
|
|
|
40
|
+ pass
|
|
|
41
|
+ if secret:
|
|
|
42
|
+ return getpass.getpass(message)
|
|
|
43
|
+ return input(message)
|
|
|
44
|
+
|
|
|
45
|
+
|
|
|
46
|
+def confirm(message):
|
|
|
47
|
+ """Ask a yes/no question, defaulting to no, until a valid answer is given."""
|
|
|
48
|
+ while True:
|
|
|
49
|
+ answer = prompt(f"{message} [y/N]: ").strip().lower()
|
|
|
50
|
+ if answer in ("y", "yes"):
|
|
|
51
|
+ return True
|
|
|
52
|
+ if answer in ("", "n", "no"):
|
|
|
53
|
+ return False
|
|
|
54
|
+ print(TERM.red("ERROR! Please answer 'y' or 'n'."))
|
|
|
55
|
+
|
|
|
56
|
+
|
|
|
57
|
+def read_config(topsrcdir):
|
|
|
58
|
+ """Read the project/token/ssh_* config saved by `mach yeet auth`, or
|
|
|
59
|
+ None if it hasn't been run yet.
|
|
|
60
|
+ """
|
|
|
61
|
+ config_path = Path(topsrcdir) / YEET_CONFIG_FILENAME
|
|
|
62
|
+ if not config_path.is_file():
|
|
|
63
|
+ return None
|
|
|
64
|
+
|
|
|
65
|
+ with open(config_path) as f:
|
|
|
66
|
+ return json.load(f) |
tools/yeet/mach_commands.py
|
|
1
|
+# This Source Code Form is subject to the terms of the Mozilla Public
|
|
|
2
|
+# License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
|
3
|
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
|
4
|
+
|
|
|
5
|
+from mach.decorators import Command, CommandArgument, SubCommand # noqa: I001
|
|
|
6
|
+
|
|
|
7
|
+from yeet import remote
|
|
|
8
|
+
|
|
|
9
|
+
|
|
|
10
|
+@Command(
|
|
|
11
|
+ "yeet",
|
|
|
12
|
+ category="ci",
|
|
|
13
|
+ description="TODO: build / test remotely from the current state of the repo.",
|
|
|
14
|
+)
|
|
|
15
|
+def yeet(command_context):
|
|
|
16
|
+ # TODO: this will be the main "yeet" command -- push, build and (maybe)
|
|
|
17
|
+ # test remotely from the current state of the local repo, rather than
|
|
|
18
|
+ # requiring an already-published build like `yeet remote` does.
|
|
|
19
|
+ pass
|
|
|
20
|
+
|
|
|
21
|
+
|
|
|
22
|
+@SubCommand(
|
|
|
23
|
+ "yeet",
|
|
|
24
|
+ "remote",
|
|
|
25
|
+ description="Trigger a tor-browser-bundle-testsuite CI job against an "
|
|
|
26
|
+ "already-published build, for a single platform.",
|
|
|
27
|
+)
|
|
|
28
|
+@CommandArgument(
|
|
|
29
|
+ "--platform",
|
|
|
30
|
+ default=None,
|
|
|
31
|
+ choices=remote.PLATFORMS,
|
|
|
32
|
+ help="Platform to test. Prompted for if not provided.",
|
|
|
33
|
+)
|
|
|
34
|
+@CommandArgument(
|
|
|
35
|
+ "--channel",
|
|
|
36
|
+ default=None,
|
|
|
37
|
+ choices=remote.ANDROID_CHANNELS,
|
|
|
38
|
+ help="Android build channel/flavor to install (only used with "
|
|
|
39
|
+ "--platform android_x86_64). Prompted for if not provided.",
|
|
|
40
|
+)
|
|
|
41
|
+@CommandArgument(
|
|
|
42
|
+ "installer_url",
|
|
|
43
|
+ help="URL of the installer to test, or a local file path to upload first.",
|
|
|
44
|
+)
|
|
|
45
|
+@CommandArgument(
|
|
|
46
|
+ "--sha256sums-url", default=None, help="URL of the build's sha256sums file."
|
|
|
47
|
+)
|
|
|
48
|
+@CommandArgument(
|
|
|
49
|
+ "--artifacts-url",
|
|
|
50
|
+ default=None,
|
|
|
51
|
+ help="URL of the build's artifacts directory. Inferred from the "
|
|
|
52
|
+ "in-tree browser version and --platform if not provided.",
|
|
|
53
|
+)
|
|
|
54
|
+@CommandArgument(
|
|
|
55
|
+ "--mozharness-url",
|
|
|
56
|
+ default=None,
|
|
|
57
|
+ help="URL of the mozharness.zip to use. Inferred from the in-tree "
|
|
|
58
|
+ "browser version and --platform if not provided.",
|
|
|
59
|
+)
|
|
|
60
|
+@CommandArgument(
|
|
|
61
|
+ "--version",
|
|
|
62
|
+ default=None,
|
|
|
63
|
+ help="Browser version to use when inferring --artifacts-url and "
|
|
|
64
|
+ "--mozharness-url. Defaults to the in-tree version "
|
|
|
65
|
+ "(browser/config/version.txt).",
|
|
|
66
|
+)
|
|
|
67
|
+@CommandArgument(
|
|
|
68
|
+ "--ref",
|
|
|
69
|
+ default="main",
|
|
|
70
|
+ help="tor-browser-bundle-testsuite ref to run on. Must have a "
|
|
|
71
|
+ ".gitlab-ci.yml, which the default branch may not.",
|
|
|
72
|
+)
|
|
|
73
|
+@CommandArgument(
|
|
|
74
|
+ "--tag",
|
|
|
75
|
+ dest="tags",
|
|
|
76
|
+ action="append",
|
|
|
77
|
+ default=None,
|
|
|
78
|
+ help="Test tag to filter tests by (can be passed multiple times, e.g. "
|
|
|
79
|
+ "--tag tor --tag base-browser). Defaults to '--tag tor --tag base-browser'.",
|
|
|
80
|
+)
|
|
|
81
|
+@CommandArgument(
|
|
|
82
|
+ "--dry-run",
|
|
|
83
|
+ action="store_true",
|
|
|
84
|
+ default=False,
|
|
|
85
|
+ help="Print the curl command instead of triggering the pipeline.",
|
|
|
86
|
+)
|
|
|
87
|
+@CommandArgument(
|
|
|
88
|
+ "--yes",
|
|
|
89
|
+ "-y",
|
|
|
90
|
+ dest="assume_yes",
|
|
|
91
|
+ action="store_true",
|
|
|
92
|
+ default=False,
|
|
|
93
|
+ help="Upload a local installer without asking for confirmation.",
|
|
|
94
|
+)
|
|
|
95
|
+def yeet_remote(command_context, **kwargs):
|
|
|
96
|
+ # With action="", argparse appends to a non-None default instead of
|
|
|
97
|
+ # replacing it, so the default tags are applied here only if none were given.
|
|
|
98
|
+ if kwargs["tags"] is None:
|
|
|
99
|
+ kwargs["tags"] = ["tor", "base-browser"]
|
|
|
100
|
+ return remote.run(command_context, **kwargs)
|
|
|
101
|
+
|
|
|
102
|
+
|
|
|
103
|
+@SubCommand(
|
|
|
104
|
+ "yeet",
|
|
|
105
|
+ "auth",
|
|
|
106
|
+ description="Save a GitLab project and access token for running CI jobs.",
|
|
|
107
|
+)
|
|
|
108
|
+@CommandArgument(
|
|
|
109
|
+ "--no-verify",
|
|
|
110
|
+ dest="verify",
|
|
|
111
|
+ action="store_false",
|
|
|
112
|
+ help="Skip checking the GitLab and SSH credentials before saving them.",
|
|
|
113
|
+)
|
|
|
114
|
+@CommandArgument(
|
|
|
115
|
+ "--gitlab-username",
|
|
|
116
|
+ default=None,
|
|
|
117
|
+ help="GitLab username to use, skipping that prompt.",
|
|
|
118
|
+)
|
|
|
119
|
+@CommandArgument(
|
|
|
120
|
+ "--gitlab-token",
|
|
|
121
|
+ default=None,
|
|
|
122
|
+ help=(
|
|
|
123
|
+ "GitLab pipeline trigger token to use, skipping that prompt. Note "
|
|
|
124
|
+ "this may end up in your shell history."
|
|
|
125
|
+ ),
|
|
|
126
|
+)
|
|
|
127
|
+@CommandArgument(
|
|
|
128
|
+ "--ssh-user",
|
|
|
129
|
+ dest="ssh_username",
|
|
|
130
|
+ default=None,
|
|
|
131
|
+ help="Username on the build server, skipping that prompt.",
|
|
|
132
|
+)
|
|
|
133
|
+@CommandArgument(
|
|
|
134
|
+ "--ssh-host",
|
|
|
135
|
+ default=None,
|
|
|
136
|
+ help="Build server to upload to and run commands on over SSH, skipping that prompt.",
|
|
|
137
|
+)
|
|
|
138
|
+def yeet_auth(command_context, **kwargs):
|
|
|
139
|
+ from yeet import auth
|
|
|
140
|
+
|
|
|
141
|
+ return auth.run(command_context, **kwargs) |
tools/yeet/remote.py
|
|
1
|
+# This Source Code Form is subject to the terms of the Mozilla Public
|
|
|
2
|
+# License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
|
3
|
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
|
4
|
+
|
|
|
5
|
+import json
|
|
|
6
|
+import os
|
|
|
7
|
+import shlex
|
|
|
8
|
+import urllib.error
|
|
|
9
|
+import urllib.parse
|
|
|
10
|
+import urllib.request
|
|
|
11
|
+
|
|
|
12
|
+from yeet import upload
|
|
|
13
|
+from yeet.common import (
|
|
|
14
|
+ GITLAB_API_BASE,
|
|
|
15
|
+ TERM,
|
|
|
16
|
+ prompt,
|
|
|
17
|
+ read_config,
|
|
|
18
|
+)
|
|
|
19
|
+
|
|
|
20
|
+PLATFORM_TOOLCHAIN_DIR = {
|
|
|
21
|
+ "debian_x86_64": "linux-x86_64",
|
|
|
22
|
+ "windows_x86_64": "windows-x86_64",
|
|
|
23
|
+ "macos_x86_64": "macos-x86_64",
|
|
|
24
|
+ "android_x86_64": "android-x86_64",
|
|
|
25
|
+}
|
|
|
26
|
+
|
|
|
27
|
+PLATFORMS = list(PLATFORM_TOOLCHAIN_DIR.keys())
|
|
|
28
|
+
|
|
|
29
|
+GITLAB_REQUEST_TIMEOUT_SECONDS = 30
|
|
|
30
|
+
|
|
|
31
|
+TOOLCHAINS_URL_BASE = "https://nightlies.tbb.torproject.org/nightly-builds/toolchains"
|
|
|
32
|
+
|
|
|
33
|
+# Each Android build channel/flavor ships under its own package name, so the
|
|
|
34
|
+# pipeline needs to be told which one to install.
|
|
|
35
|
+ANDROID_CHANNEL_PACKAGE_NAMES = {
|
|
|
36
|
+ "debug": "org.torproject.torbrowser_debug",
|
|
|
37
|
+ "nightly": "org.torproject.torbrowser_nightly",
|
|
|
38
|
+ "alpha": "org.torproject.torbrowser_alpha",
|
|
|
39
|
+ "release": "org.torproject.torbrowser",
|
|
|
40
|
+}
|
|
|
41
|
+ANDROID_CHANNELS = list(ANDROID_CHANNEL_PACKAGE_NAMES)
|
|
|
42
|
+
|
|
|
43
|
+
|
|
|
44
|
+def _read_browser_version(topsrcdir):
|
|
|
45
|
+ version_path = os.path.join(topsrcdir, "browser", "config", "version.txt")
|
|
|
46
|
+ with open(version_path) as f:
|
|
|
47
|
+ return f.read().strip()
|
|
|
48
|
+
|
|
|
49
|
+
|
|
|
50
|
+def _resolve_version(topsrcdir, version):
|
|
|
51
|
+ if version:
|
|
|
52
|
+ return version
|
|
|
53
|
+
|
|
|
54
|
+ version = _read_browser_version(topsrcdir)
|
|
|
55
|
+ print(
|
|
|
56
|
+ TERM.yellow(
|
|
|
57
|
+ f"""Inferred platform version {version} from browser/config/version.txt.
|
|
|
58
|
+Pass --version if this isn't the version you want to test."""
|
|
|
59
|
+ )
|
|
|
60
|
+ )
|
|
|
61
|
+ return version
|
|
|
62
|
+
|
|
|
63
|
+
|
|
|
64
|
+def _artifacts_url(version, platform):
|
|
|
65
|
+ return f"{TOOLCHAINS_URL_BASE}/{version}/{PLATFORM_TOOLCHAIN_DIR[platform]}/"
|
|
|
66
|
+
|
|
|
67
|
+
|
|
|
68
|
+def _mozharness_url(version, platform):
|
|
|
69
|
+ return f"{_artifacts_url(version, platform)}mozharness.zip"
|
|
|
70
|
+
|
|
|
71
|
+
|
|
|
72
|
+def _curl_command(url, headers, data):
|
|
|
73
|
+ args = ["curl", "--request", "POST"]
|
|
|
74
|
+ for name, value in headers.items():
|
|
|
75
|
+ args += ["--header", f"{name}: {value}"]
|
|
|
76
|
+ args += ["--data", data.decode("utf-8"), url]
|
|
|
77
|
+ return shlex.join(args)
|
|
|
78
|
+
|
|
|
79
|
+
|
|
|
80
|
+def _resolve_installer_url(topsrcdir, installer, platform, assume_yes, dry_run):
|
|
|
81
|
+ """Accept either an installer URL directly, or a local file path to
|
|
|
82
|
+ upload first. On a dry run a local file is not uploaded, and a
|
|
|
83
|
+ placeholder URL is returned instead. Raises whatever
|
|
|
84
|
+ `upload.run` raises on failure.
|
|
|
85
|
+ """
|
|
|
86
|
+ if urllib.parse.urlparse(installer).scheme in ("http", "https"):
|
|
|
87
|
+ return installer
|
|
|
88
|
+ if dry_run:
|
|
|
89
|
+ file_path = upload.resolve_file(installer)
|
|
|
90
|
+ print(TERM.yellow(f"Dry run: not uploading {file_path}."))
|
|
|
91
|
+ return f"<URL of uploaded {file_path.name}>"
|
|
|
92
|
+ return upload.run(topsrcdir, installer, platform, assume_yes=assume_yes)
|
|
|
93
|
+
|
|
|
94
|
+
|
|
|
95
|
+def _prompt_platform():
|
|
|
96
|
+ print()
|
|
|
97
|
+ print(
|
|
|
98
|
+ TERM.bold(
|
|
|
99
|
+ "Select the platform you are testing for (if not listed, tests are not supported yet):"
|
|
|
100
|
+ )
|
|
|
101
|
+ )
|
|
|
102
|
+ for i, platform in enumerate(PLATFORMS, 1):
|
|
|
103
|
+ print(f" {TERM.cyan(str(i))}) {platform}")
|
|
|
104
|
+ print()
|
|
|
105
|
+
|
|
|
106
|
+ while True:
|
|
|
107
|
+ choice = prompt(f"Platform [1-{len(PLATFORMS)}]: ").strip()
|
|
|
108
|
+ if choice.isdigit() and 1 <= int(choice) <= len(PLATFORMS):
|
|
|
109
|
+ print()
|
|
|
110
|
+ return PLATFORMS[int(choice) - 1]
|
|
|
111
|
+ print(TERM.red(f"ERROR! Please enter a number between 1 and {len(PLATFORMS)}."))
|
|
|
112
|
+
|
|
|
113
|
+
|
|
|
114
|
+def _prompt_channel():
|
|
|
115
|
+ print()
|
|
|
116
|
+ print(TERM.bold("Select the Android build channel/flavor you are testing:"))
|
|
|
117
|
+ for i, channel in enumerate(ANDROID_CHANNELS, 1):
|
|
|
118
|
+ print(f" {TERM.cyan(str(i))}) {channel}")
|
|
|
119
|
+ print()
|
|
|
120
|
+
|
|
|
121
|
+ while True:
|
|
|
122
|
+ choice = prompt(f"Channel [1-{len(ANDROID_CHANNELS)}]: ").strip()
|
|
|
123
|
+ if choice.isdigit() and 1 <= int(choice) <= len(ANDROID_CHANNELS):
|
|
|
124
|
+ print()
|
|
|
125
|
+ return ANDROID_CHANNELS[int(choice) - 1]
|
|
|
126
|
+ print(
|
|
|
127
|
+ TERM.red(
|
|
|
128
|
+ f"ERROR! Please enter a number between 1 and {len(ANDROID_CHANNELS)}."
|
|
|
129
|
+ )
|
|
|
130
|
+ )
|
|
|
131
|
+
|
|
|
132
|
+
|
|
|
133
|
+def run(command_context, **kwargs):
|
|
|
134
|
+ # TODO(tor-browser-bundle-testsuite#40120): this runs whatever the pipeline
|
|
|
135
|
+ # runs, indiscriminately -- fine for now since marionette is the only suite
|
|
|
136
|
+ # it has, but once others (xpcshell, cppunittest, mochitest, ...) exist, add
|
|
|
137
|
+ # a --suite argument here and thread it into `inputs` so a run can be scoped
|
|
|
138
|
+ # to just one.
|
|
|
139
|
+ config = read_config(command_context.topsrcdir)
|
|
|
140
|
+ if config is None:
|
|
|
141
|
+ print(TERM.red("No GitLab fork/token found, run `mach yeet auth` first."))
|
|
|
142
|
+ return 1
|
|
|
143
|
+
|
|
|
144
|
+ installer_url = kwargs["installer_url"].strip()
|
|
|
145
|
+ if not installer_url:
|
|
|
146
|
+ print(TERM.red("ERROR! The installer URL or path was empty."))
|
|
|
147
|
+ return 1
|
|
|
148
|
+
|
|
|
149
|
+ platform = kwargs.get("platform") or _prompt_platform()
|
|
|
150
|
+ dry_run = kwargs.get("dry_run", False)
|
|
|
151
|
+
|
|
|
152
|
+ try:
|
|
|
153
|
+ installer_url = _resolve_installer_url(
|
|
|
154
|
+ command_context.topsrcdir,
|
|
|
155
|
+ installer_url,
|
|
|
156
|
+ platform,
|
|
|
157
|
+ kwargs.get("assume_yes", False),
|
|
|
158
|
+ dry_run,
|
|
|
159
|
+ )
|
|
|
160
|
+ except (FileNotFoundError, RuntimeError) as e:
|
|
|
161
|
+ print()
|
|
|
162
|
+ print(TERM.red(f"ERROR! {e}"))
|
|
|
163
|
+ return 1
|
|
|
164
|
+
|
|
|
165
|
+ package_name = None
|
|
|
166
|
+ if platform == "android_x86_64":
|
|
|
167
|
+ channel = kwargs.get("channel") or _prompt_channel()
|
|
|
168
|
+ if channel not in ANDROID_CHANNEL_PACKAGE_NAMES:
|
|
|
169
|
+ print(TERM.red(f"ERROR! Unknown channel {channel!r}."))
|
|
|
170
|
+ return 1
|
|
|
171
|
+ package_name = ANDROID_CHANNEL_PACKAGE_NAMES[channel]
|
|
|
172
|
+
|
|
|
173
|
+ artifacts_url = kwargs.get("artifacts_url")
|
|
|
174
|
+ mozharness_url = kwargs.get("mozharness_url")
|
|
|
175
|
+ if not artifacts_url or not mozharness_url:
|
|
|
176
|
+ version = _resolve_version(command_context.topsrcdir, kwargs.get("version"))
|
|
|
177
|
+ if not artifacts_url:
|
|
|
178
|
+ artifacts_url = _artifacts_url(version, platform)
|
|
|
179
|
+ if not mozharness_url:
|
|
|
180
|
+ mozharness_url = _mozharness_url(version, platform)
|
|
|
181
|
+
|
|
|
182
|
+ inputs = {
|
|
|
183
|
+ "mozharness_url": mozharness_url,
|
|
|
184
|
+ f"{platform}_installer_url": installer_url,
|
|
|
185
|
+ f"{platform}_artifacts_url": artifacts_url,
|
|
|
186
|
+ }
|
|
|
187
|
+ if package_name:
|
|
|
188
|
+ inputs[f"{platform}_package_name"] = package_name
|
|
|
189
|
+ if sha256sums_url := kwargs.get("sha256sums_url"):
|
|
|
190
|
+ inputs[f"{platform}_sha256sums_url"] = sha256sums_url
|
|
|
191
|
+ if tags := kwargs.get("tags"):
|
|
|
192
|
+ inputs["tags"] = shlex.join(arg for tag in tags for arg in ("--tag", tag))
|
|
|
193
|
+
|
|
|
194
|
+ project_id = urllib.parse.quote(config["project"], safe="")
|
|
|
195
|
+ url = f"{GITLAB_API_BASE}/projects/{project_id}/trigger/pipeline"
|
|
|
196
|
+ headers = {"Content-Type": "application/json"}
|
|
|
197
|
+ data = json.dumps({
|
|
|
198
|
+ "token": config["token"],
|
|
|
199
|
+ "ref": kwargs.get("ref", "main"),
|
|
|
200
|
+ "inputs": inputs,
|
|
|
201
|
+ }).encode("utf-8")
|
|
|
202
|
+
|
|
|
203
|
+ if dry_run:
|
|
|
204
|
+ print()
|
|
|
205
|
+ print(TERM.cyan(_curl_command(url, headers, data)))
|
|
|
206
|
+ print()
|
|
|
207
|
+ return 0
|
|
|
208
|
+
|
|
|
209
|
+ request = urllib.request.Request(url, data=data, headers=headers, method="POST")
|
|
|
210
|
+
|
|
|
211
|
+ try:
|
|
|
212
|
+ with urllib.request.urlopen(
|
|
|
213
|
+ request, timeout=GITLAB_REQUEST_TIMEOUT_SECONDS
|
|
|
214
|
+ ) as response:
|
|
|
215
|
+ result = json.load(response)
|
|
|
216
|
+ except urllib.error.HTTPError as e:
|
|
|
217
|
+ body = e.read().decode("utf-8", "replace")
|
|
|
218
|
+ print(
|
|
|
219
|
+ TERM.red(f"ERROR! Failed to trigger pipeline: {e.code} {e.reason}\n{body}")
|
|
|
220
|
+ )
|
|
|
221
|
+ return 1
|
|
|
222
|
+ except urllib.error.URLError as e:
|
|
|
223
|
+ print(TERM.red(f"ERROR! Could not reach GitLab: {e}"))
|
|
|
224
|
+ return 1
|
|
|
225
|
+
|
|
|
226
|
+ print()
|
|
|
227
|
+ print(TERM.green(f"Pipeline triggered: {result.get('web_url')}"))
|
|
|
228
|
+ return 0 |
tools/yeet/tests/python/conftest.py
|
|
1
|
+# This Source Code Form is subject to the terms of the Mozilla Public
|
|
|
2
|
+# License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
|
3
|
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
|
4
|
+
|
|
|
5
|
+import json
|
|
|
6
|
+
|
|
|
7
|
+from yeet import common # noqa: E402
|
|
|
8
|
+
|
|
|
9
|
+
|
|
|
10
|
+def write_config(tmp_path, **overrides):
|
|
|
11
|
+ """Write a `.yeet.json` under `tmp_path`, as `mach yeet auth` would,
|
|
|
12
|
+ for tests that need one already in place.
|
|
|
13
|
+ """
|
|
|
14
|
+ config = {
|
|
|
15
|
+ "project": "bea/tor-browser-bundle-testsuite",
|
|
|
16
|
+ "token": "s3cr3t",
|
|
|
17
|
+ "ssh_host": common.DEFAULT_SSH_HOST,
|
|
|
18
|
+ "ssh_user": "bea",
|
|
|
19
|
+ }
|
|
|
20
|
+ config.update(overrides)
|
|
|
21
|
+ with open(tmp_path / common.YEET_CONFIG_FILENAME, "w") as f:
|
|
|
22
|
+ json.dump(config, f)
|
|
|
23
|
+ return config |
tools/yeet/tests/python/python.toml
|
|
1
|
+[DEFAULT]
|
|
|
2
|
+subsuite = "base-browser"
|
|
|
3
|
+
|
|
|
4
|
+["test_auth.py"]
|
|
|
5
|
+
|
|
|
6
|
+["test_upload.py"]
|
|
|
7
|
+
|
|
|
8
|
+["test_remote.py"]
|
|
|
9
|
+
|
|
|
10
|
+["test_common.py"] |
tools/yeet/tests/python/test_auth.py
|
|
1
|
+# This Source Code Form is subject to the terms of the Mozilla Public
|
|
|
2
|
+# License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
|
3
|
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
|
4
|
+
|
|
|
5
|
+import json
|
|
|
6
|
+import urllib.error
|
|
|
7
|
+from types import SimpleNamespace
|
|
|
8
|
+
|
|
|
9
|
+import mozunit
|
|
|
10
|
+import pytest
|
|
|
11
|
+from yeet import auth, common
|
|
|
12
|
+
|
|
|
13
|
+
|
|
|
14
|
+def _set_inputs(monkeypatch, *values):
|
|
|
15
|
+ values = iter(values)
|
|
|
16
|
+ monkeypatch.setattr("builtins.input", lambda prompt="": next(values))
|
|
|
17
|
+
|
|
|
18
|
+
|
|
|
19
|
+def _forbid_prompts(monkeypatch):
|
|
|
20
|
+ """Make input()/getpass() blow up instead of hanging, so a test can
|
|
|
21
|
+ assert that a fully non-interactive call never prompts."""
|
|
|
22
|
+
|
|
|
23
|
+ def _fail(prompt=""):
|
|
|
24
|
+ raise AssertionError(f"unexpected prompt: {prompt!r}")
|
|
|
25
|
+
|
|
|
26
|
+ monkeypatch.setattr("builtins.input", _fail)
|
|
|
27
|
+ monkeypatch.setattr(common.getpass, "getpass", _fail)
|
|
|
28
|
+
|
|
|
29
|
+
|
|
|
30
|
+@pytest.mark.parametrize("value", ["bea", " bea ", "bea.b", "bea_b-1"])
|
|
|
31
|
+def test_resolve_project_accepts_username(value):
|
|
|
32
|
+ project = auth._resolve_project(value)
|
|
|
33
|
+
|
|
|
34
|
+ assert project == f"{value.strip()}/{common.CANONICAL_GITLAB_REPO_NAME}"
|
|
|
35
|
+
|
|
|
36
|
+
|
|
|
37
|
+@pytest.mark.parametrize(
|
|
|
38
|
+ "value",
|
|
|
39
|
+ [
|
|
|
40
|
+ common.CANONICAL_GITLAB_NAMESPACE,
|
|
|
41
|
+ common.CANONICAL_GITLAB_NAMESPACE.upper(),
|
|
|
42
|
+ f" {common.CANONICAL_GITLAB_NAMESPACE} ",
|
|
|
43
|
+ ],
|
|
|
44
|
+)
|
|
|
45
|
+def test_resolve_project_rejects_canonical_namespace(value):
|
|
|
46
|
+ with pytest.raises(ValueError, match="canonical"):
|
|
|
47
|
+ auth._resolve_project(value)
|
|
|
48
|
+
|
|
|
49
|
+
|
|
|
50
|
+def test_resolve_project_accepts_top_level_group_of_canonical_namespace():
|
|
|
51
|
+ group = common.CANONICAL_GITLAB_NAMESPACE.split("/", 1)[0]
|
|
|
52
|
+
|
|
|
53
|
+ assert auth._resolve_project(group) == (
|
|
|
54
|
+ f"{group}/{common.CANONICAL_GITLAB_REPO_NAME}"
|
|
|
55
|
+ )
|
|
|
56
|
+
|
|
|
57
|
+
|
|
|
58
|
+def test_prompt_fork_rejects_canonical_namespace(monkeypatch, capsys):
|
|
|
59
|
+ canonical_namespace = common.CANONICAL_GITLAB_NAMESPACE
|
|
|
60
|
+ _set_inputs(monkeypatch, canonical_namespace, "bea")
|
|
|
61
|
+
|
|
|
62
|
+ project = auth._prompt_fork()
|
|
|
63
|
+
|
|
|
64
|
+ assert project == f"bea/{common.CANONICAL_GITLAB_REPO_NAME}"
|
|
|
65
|
+ assert "not a" in capsys.readouterr().out.lower()
|
|
|
66
|
+
|
|
|
67
|
+
|
|
|
68
|
+@pytest.mark.parametrize("value", ["", " ", "\t\n"])
|
|
|
69
|
+def test_resolve_project_rejects_blank_username(value):
|
|
|
70
|
+ with pytest.raises(ValueError):
|
|
|
71
|
+ auth._resolve_project(value)
|
|
|
72
|
+
|
|
|
73
|
+
|
|
|
74
|
+def test_prompt_fork_rejects_empty_username(monkeypatch):
|
|
|
75
|
+ _set_inputs(monkeypatch, "", " \t ", "bea")
|
|
|
76
|
+
|
|
|
77
|
+ project = auth._prompt_fork()
|
|
|
78
|
+
|
|
|
79
|
+ assert project == f"bea/{common.CANONICAL_GITLAB_REPO_NAME}"
|
|
|
80
|
+
|
|
|
81
|
+
|
|
|
82
|
+@pytest.mark.parametrize("status", [400])
|
|
|
83
|
+def test_verify_auth_accepts_valid_token(monkeypatch, status):
|
|
|
84
|
+ def fake_urlopen(request, *args, **kwargs):
|
|
|
85
|
+ raise urllib.error.HTTPError("url", status, "msg", None, None)
|
|
|
86
|
+
|
|
|
87
|
+ monkeypatch.setattr(auth.urllib.request, "urlopen", fake_urlopen)
|
|
|
88
|
+
|
|
|
89
|
+ assert auth._verify_auth("bea/tor-browser-bundle-testsuite", "tok") is True
|
|
|
90
|
+
|
|
|
91
|
+
|
|
|
92
|
+@pytest.mark.parametrize("status", [401, 404])
|
|
|
93
|
+def test_verify_auth_rejects_bad_token_or_project(monkeypatch, status):
|
|
|
94
|
+ def fake_urlopen(request, *args, **kwargs):
|
|
|
95
|
+ raise urllib.error.HTTPError("url", status, "msg", None, None)
|
|
|
96
|
+
|
|
|
97
|
+ monkeypatch.setattr(auth.urllib.request, "urlopen", fake_urlopen)
|
|
|
98
|
+
|
|
|
99
|
+ assert auth._verify_auth("bea/tor-browser-bundle-testsuite", "tok") is False
|
|
|
100
|
+
|
|
|
101
|
+
|
|
|
102
|
+def test_verify_auth_propagates_network_errors(monkeypatch):
|
|
|
103
|
+ def fake_urlopen(request, *args, **kwargs):
|
|
|
104
|
+ raise urllib.error.URLError("unreachable")
|
|
|
105
|
+
|
|
|
106
|
+ monkeypatch.setattr(auth.urllib.request, "urlopen", fake_urlopen)
|
|
|
107
|
+
|
|
|
108
|
+ with pytest.raises(urllib.error.URLError):
|
|
|
109
|
+ auth._verify_auth("bea/tor-browser-bundle-testsuite", "tok")
|
|
|
110
|
+
|
|
|
111
|
+
|
|
|
112
|
+def test_verify_ssh_success(monkeypatch):
|
|
|
113
|
+ calls = []
|
|
|
114
|
+
|
|
|
115
|
+ def fake_run(cmd, **kwargs):
|
|
|
116
|
+ calls.append((cmd, kwargs))
|
|
|
117
|
+ return SimpleNamespace(returncode=0)
|
|
|
118
|
+
|
|
|
119
|
+ monkeypatch.setattr(auth.subprocess, "run", fake_run)
|
|
|
120
|
+
|
|
|
121
|
+ assert auth._verify_ssh("bea@xxxxxxxxxxxxxxxxxxxxxxxxxx") is True
|
|
|
122
|
+ cmd, kwargs = calls[0]
|
|
|
123
|
+ assert cmd[-2:] == ["bea@xxxxxxxxxxxxxxxxxxxxxxxxxx", "true"]
|
|
|
124
|
+ assert kwargs["stdin"] is auth.subprocess.DEVNULL
|
|
|
125
|
+
|
|
|
126
|
+
|
|
|
127
|
+def test_verify_ssh_failure(monkeypatch):
|
|
|
128
|
+ monkeypatch.setattr(
|
|
|
129
|
+ auth.subprocess,
|
|
|
130
|
+ "run",
|
|
|
131
|
+ lambda *args, **kwargs: SimpleNamespace(returncode=255),
|
|
|
132
|
+ )
|
|
|
133
|
+
|
|
|
134
|
+ assert auth._verify_ssh("bea@xxxxxxxxxxxxxxxxxxxxxxxxxx") is False
|
|
|
135
|
+
|
|
|
136
|
+
|
|
|
137
|
+def test_verify_ssh_missing_binary(monkeypatch, capsys):
|
|
|
138
|
+ def fake_run(*args, **kwargs):
|
|
|
139
|
+ raise FileNotFoundError()
|
|
|
140
|
+
|
|
|
141
|
+ monkeypatch.setattr(auth.subprocess, "run", fake_run)
|
|
|
142
|
+
|
|
|
143
|
+ assert auth._verify_ssh("bea@xxxxxxxxxxxxxxxxxxxxxxxxxx") is False
|
|
|
144
|
+ assert "no `ssh` binary" in capsys.readouterr().out.lower()
|
|
|
145
|
+
|
|
|
146
|
+
|
|
|
147
|
+def test_run_saves_config_and_skips_verification(tmp_path, monkeypatch):
|
|
|
148
|
+ _set_inputs(monkeypatch, "bea", "", "bea")
|
|
|
149
|
+ monkeypatch.setattr(common.getpass, "getpass", lambda prompt="": "s3cr3t")
|
|
|
150
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
151
|
+
|
|
|
152
|
+ result = auth.run(command_context, verify=False)
|
|
|
153
|
+
|
|
|
154
|
+ assert result == 0
|
|
|
155
|
+ config_path = tmp_path / common.YEET_CONFIG_FILENAME
|
|
|
156
|
+ with open(config_path) as f:
|
|
|
157
|
+ saved = json.load(f)
|
|
|
158
|
+ assert saved == {
|
|
|
159
|
+ "project": f"bea/{common.CANONICAL_GITLAB_REPO_NAME}",
|
|
|
160
|
+ "token": "s3cr3t",
|
|
|
161
|
+ "ssh_host": common.DEFAULT_SSH_HOST,
|
|
|
162
|
+ "ssh_user": "bea",
|
|
|
163
|
+ }
|
|
|
164
|
+ assert (config_path.stat().st_mode & 0o777) == 0o600
|
|
|
165
|
+
|
|
|
166
|
+
|
|
|
167
|
+def test_prompt_ssh_username_reprompts_until_non_empty(monkeypatch, capsys):
|
|
|
168
|
+ _set_inputs(monkeypatch, "", " ", "bea")
|
|
|
169
|
+
|
|
|
170
|
+ assert auth._prompt_ssh_username(common.DEFAULT_SSH_HOST) == "bea"
|
|
|
171
|
+ assert capsys.readouterr().out.count("ERROR!") == 2
|
|
|
172
|
+
|
|
|
173
|
+
|
|
|
174
|
+def test_prompt_ssh_host_defaults_when_empty(monkeypatch):
|
|
|
175
|
+ _set_inputs(monkeypatch, " ")
|
|
|
176
|
+
|
|
|
177
|
+ assert auth._prompt_ssh_host() == common.DEFAULT_SSH_HOST
|
|
|
178
|
+
|
|
|
179
|
+
|
|
|
180
|
+def test_run_prompts_for_ssh_host(tmp_path, monkeypatch):
|
|
|
181
|
+ _set_inputs(monkeypatch, "build.example.com", "bea")
|
|
|
182
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
183
|
+
|
|
|
184
|
+ result = auth.run(
|
|
|
185
|
+ command_context, verify=False, gitlab_username="bea", gitlab_token="s3cr3t"
|
|
|
186
|
+ )
|
|
|
187
|
+
|
|
|
188
|
+ assert result == 0
|
|
|
189
|
+ with open(tmp_path / common.YEET_CONFIG_FILENAME) as f:
|
|
|
190
|
+ saved = json.load(f)
|
|
|
191
|
+ assert saved["ssh_host"] == "build.example.com"
|
|
|
192
|
+ assert saved["ssh_user"] == "bea"
|
|
|
193
|
+
|
|
|
194
|
+
|
|
|
195
|
+def test_prompt_token_reprompts_until_non_empty(monkeypatch, capsys):
|
|
|
196
|
+ values = iter(["", " ", "s3cr3t"])
|
|
|
197
|
+ monkeypatch.setattr(common.getpass, "getpass", lambda prompt="": next(values))
|
|
|
198
|
+
|
|
|
199
|
+ assert auth._prompt_token() == "s3cr3t"
|
|
|
200
|
+ assert capsys.readouterr().out.count("ERROR!") == 2
|
|
|
201
|
+
|
|
|
202
|
+
|
|
|
203
|
+def test_run_aborts_when_gitlab_verification_fails(tmp_path, monkeypatch, capsys):
|
|
|
204
|
+ _set_inputs(monkeypatch, "bea", "", "bea")
|
|
|
205
|
+ monkeypatch.setattr(common.getpass, "getpass", lambda prompt="": "s3cr3t")
|
|
|
206
|
+ monkeypatch.setattr(auth, "_verify_auth", lambda project, token: False)
|
|
|
207
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
208
|
+
|
|
|
209
|
+ result = auth.run(command_context, verify=True)
|
|
|
210
|
+
|
|
|
211
|
+ assert result == 1
|
|
|
212
|
+ assert not (tmp_path / common.YEET_CONFIG_FILENAME).exists()
|
|
|
213
|
+ assert (
|
|
|
214
|
+ "ERROR! GitLab rejected that project/token combination."
|
|
|
215
|
+ in capsys.readouterr().out
|
|
|
216
|
+ )
|
|
|
217
|
+
|
|
|
218
|
+
|
|
|
219
|
+def test_run_aborts_when_gitlab_unreachable(tmp_path, monkeypatch, capsys):
|
|
|
220
|
+ _set_inputs(monkeypatch, "bea", "", "bea")
|
|
|
221
|
+ monkeypatch.setattr(common.getpass, "getpass", lambda prompt="": "s3cr3t")
|
|
|
222
|
+
|
|
|
223
|
+ def raise_url_error(project, token):
|
|
|
224
|
+ raise urllib.error.URLError("unreachable")
|
|
|
225
|
+
|
|
|
226
|
+ monkeypatch.setattr(auth, "_verify_auth", raise_url_error)
|
|
|
227
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
228
|
+
|
|
|
229
|
+ result = auth.run(command_context, verify=True)
|
|
|
230
|
+
|
|
|
231
|
+ assert result == 1
|
|
|
232
|
+ assert not (tmp_path / common.YEET_CONFIG_FILENAME).exists()
|
|
|
233
|
+ assert (
|
|
|
234
|
+ "ERROR! Could not reach GitLab: <urlopen error unreachable>"
|
|
|
235
|
+ in capsys.readouterr().out
|
|
|
236
|
+ )
|
|
|
237
|
+
|
|
|
238
|
+
|
|
|
239
|
+def test_run_aborts_when_ssh_verification_fails(tmp_path, monkeypatch, capsys):
|
|
|
240
|
+ _set_inputs(monkeypatch, "bea", "", "bea")
|
|
|
241
|
+ monkeypatch.setattr(common.getpass, "getpass", lambda prompt="": "s3cr3t")
|
|
|
242
|
+ monkeypatch.setattr(auth, "_verify_auth", lambda project, token: True)
|
|
|
243
|
+ monkeypatch.setattr(auth, "_verify_ssh", lambda target: False)
|
|
|
244
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
245
|
+
|
|
|
246
|
+ result = auth.run(command_context, verify=True)
|
|
|
247
|
+
|
|
|
248
|
+ assert result == 1
|
|
|
249
|
+ assert not (tmp_path / common.YEET_CONFIG_FILENAME).exists()
|
|
|
250
|
+ assert (
|
|
|
251
|
+ f"ERROR! Could not SSH into bea@{common.DEFAULT_SSH_HOST}."
|
|
|
252
|
+ in capsys.readouterr().out
|
|
|
253
|
+ )
|
|
|
254
|
+
|
|
|
255
|
+
|
|
|
256
|
+def test_run_saves_config_when_verification_succeeds(tmp_path, monkeypatch):
|
|
|
257
|
+ _set_inputs(monkeypatch, "bea", "", "bea")
|
|
|
258
|
+ monkeypatch.setattr(common.getpass, "getpass", lambda prompt="": "s3cr3t")
|
|
|
259
|
+ monkeypatch.setattr(auth, "_verify_auth", lambda project, token: True)
|
|
|
260
|
+ monkeypatch.setattr(auth, "_verify_ssh", lambda target: True)
|
|
|
261
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
262
|
+
|
|
|
263
|
+ result = auth.run(command_context, verify=True)
|
|
|
264
|
+
|
|
|
265
|
+ assert result == 0
|
|
|
266
|
+ with open(tmp_path / common.YEET_CONFIG_FILENAME) as f:
|
|
|
267
|
+ assert json.load(f) == {
|
|
|
268
|
+ "project": f"bea/{common.CANONICAL_GITLAB_REPO_NAME}",
|
|
|
269
|
+ "token": "s3cr3t",
|
|
|
270
|
+ "ssh_host": common.DEFAULT_SSH_HOST,
|
|
|
271
|
+ "ssh_user": "bea",
|
|
|
272
|
+ }
|
|
|
273
|
+
|
|
|
274
|
+
|
|
|
275
|
+def test_run_uses_cli_gitlab_username_and_still_prompts_for_rest(tmp_path, monkeypatch):
|
|
|
276
|
+ # ssh host and username aren't given via CLI, so they're the only things
|
|
|
277
|
+ # prompted for.
|
|
|
278
|
+ _set_inputs(monkeypatch, "", "bea")
|
|
|
279
|
+ monkeypatch.setattr(common.getpass, "getpass", lambda prompt="": "s3cr3t")
|
|
|
280
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
281
|
+
|
|
|
282
|
+ result = auth.run(command_context, verify=False, gitlab_username="bea")
|
|
|
283
|
+
|
|
|
284
|
+ assert result == 0
|
|
|
285
|
+ with open(tmp_path / common.YEET_CONFIG_FILENAME) as f:
|
|
|
286
|
+ saved = json.load(f)
|
|
|
287
|
+ assert saved["project"] == f"bea/{common.CANONICAL_GITLAB_REPO_NAME}"
|
|
|
288
|
+ assert saved["ssh_user"] == "bea"
|
|
|
289
|
+
|
|
|
290
|
+
|
|
|
291
|
+def test_run_rejects_invalid_cli_gitlab_username(tmp_path, monkeypatch, capsys):
|
|
|
292
|
+ _forbid_prompts(monkeypatch)
|
|
|
293
|
+ canonical_namespace = common.CANONICAL_GITLAB_NAMESPACE
|
|
|
294
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
295
|
+
|
|
|
296
|
+ result = auth.run(
|
|
|
297
|
+ command_context, verify=False, gitlab_username=canonical_namespace
|
|
|
298
|
+ )
|
|
|
299
|
+
|
|
|
300
|
+ assert result == 1
|
|
|
301
|
+ assert not (tmp_path / common.YEET_CONFIG_FILENAME).exists()
|
|
|
302
|
+ assert "not a" in capsys.readouterr().out.lower()
|
|
|
303
|
+
|
|
|
304
|
+
|
|
|
305
|
+def test_run_fully_noninteractive_with_all_cli_args(tmp_path, monkeypatch):
|
|
|
306
|
+ _forbid_prompts(monkeypatch)
|
|
|
307
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
308
|
+
|
|
|
309
|
+ result = auth.run(
|
|
|
310
|
+ command_context,
|
|
|
311
|
+ verify=False,
|
|
|
312
|
+ gitlab_username="bea",
|
|
|
313
|
+ gitlab_token="s3cr3t",
|
|
|
314
|
+ ssh_username="bea",
|
|
|
315
|
+ ssh_host=common.DEFAULT_SSH_HOST,
|
|
|
316
|
+ )
|
|
|
317
|
+
|
|
|
318
|
+ assert result == 0
|
|
|
319
|
+ with open(tmp_path / common.YEET_CONFIG_FILENAME) as f:
|
|
|
320
|
+ saved = json.load(f)
|
|
|
321
|
+ assert saved == {
|
|
|
322
|
+ "project": f"bea/{common.CANONICAL_GITLAB_REPO_NAME}",
|
|
|
323
|
+ "token": "s3cr3t",
|
|
|
324
|
+ "ssh_host": common.DEFAULT_SSH_HOST,
|
|
|
325
|
+ "ssh_user": "bea",
|
|
|
326
|
+ }
|
|
|
327
|
+
|
|
|
328
|
+
|
|
|
329
|
+def test_run_saves_cli_ssh_host(tmp_path, monkeypatch):
|
|
|
330
|
+ _forbid_prompts(monkeypatch)
|
|
|
331
|
+ targets = []
|
|
|
332
|
+ monkeypatch.setattr(auth, "_verify_auth", lambda project, token: True)
|
|
|
333
|
+ monkeypatch.setattr(
|
|
|
334
|
+ auth, "_verify_ssh", lambda target: targets.append(target) or True
|
|
|
335
|
+ )
|
|
|
336
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
337
|
+
|
|
|
338
|
+ result = auth.run(
|
|
|
339
|
+ command_context,
|
|
|
340
|
+ verify=True,
|
|
|
341
|
+ gitlab_username="bea",
|
|
|
342
|
+ gitlab_token="s3cr3t",
|
|
|
343
|
+ ssh_username="bea",
|
|
|
344
|
+ ssh_host="build.example.com",
|
|
|
345
|
+ )
|
|
|
346
|
+
|
|
|
347
|
+ assert result == 0
|
|
|
348
|
+ assert targets == ["bea@xxxxxxxxxxxxxxxxx"]
|
|
|
349
|
+ with open(tmp_path / common.YEET_CONFIG_FILENAME) as f:
|
|
|
350
|
+ assert json.load(f)["ssh_host"] == "build.example.com"
|
|
|
351
|
+
|
|
|
352
|
+
|
|
|
353
|
+def test_run_fully_noninteractive_with_verification(tmp_path, monkeypatch):
|
|
|
354
|
+ _forbid_prompts(monkeypatch)
|
|
|
355
|
+ monkeypatch.setattr(auth, "_verify_auth", lambda project, token: True)
|
|
|
356
|
+ monkeypatch.setattr(auth, "_verify_ssh", lambda target: True)
|
|
|
357
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
358
|
+
|
|
|
359
|
+ result = auth.run(
|
|
|
360
|
+ command_context,
|
|
|
361
|
+ verify=True,
|
|
|
362
|
+ gitlab_username="bea",
|
|
|
363
|
+ gitlab_token="s3cr3t",
|
|
|
364
|
+ ssh_username="bea",
|
|
|
365
|
+ ssh_host=common.DEFAULT_SSH_HOST,
|
|
|
366
|
+ )
|
|
|
367
|
+
|
|
|
368
|
+ assert result == 0
|
|
|
369
|
+ with open(tmp_path / common.YEET_CONFIG_FILENAME) as f:
|
|
|
370
|
+ assert json.load(f) == {
|
|
|
371
|
+ "project": f"bea/{common.CANONICAL_GITLAB_REPO_NAME}",
|
|
|
372
|
+ "token": "s3cr3t",
|
|
|
373
|
+ "ssh_host": common.DEFAULT_SSH_HOST,
|
|
|
374
|
+ "ssh_user": "bea",
|
|
|
375
|
+ }
|
|
|
376
|
+
|
|
|
377
|
+
|
|
|
378
|
+_CLI_ARGS = {
|
|
|
379
|
+ "gitlab_username": "bea",
|
|
|
380
|
+ "gitlab_token": "s3cr3t",
|
|
|
381
|
+ "ssh_username": "bea",
|
|
|
382
|
+ "ssh_host": "build.example.com",
|
|
|
383
|
+}
|
|
|
384
|
+
|
|
|
385
|
+_SAVED_CONFIG = {
|
|
|
386
|
+ "project": f"bea/{common.CANONICAL_GITLAB_REPO_NAME}",
|
|
|
387
|
+ "token": "s3cr3t",
|
|
|
388
|
+ "ssh_host": "build.example.com",
|
|
|
389
|
+ "ssh_user": "bea",
|
|
|
390
|
+}
|
|
|
391
|
+
|
|
|
392
|
+
|
|
|
393
|
+@pytest.mark.parametrize("padding", [" ", "\t", "\n", " \t\n "])
|
|
|
394
|
+def test_run_trims_cli_args(tmp_path, monkeypatch, padding):
|
|
|
395
|
+ _forbid_prompts(monkeypatch)
|
|
|
396
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
397
|
+
|
|
|
398
|
+ result = auth.run(
|
|
|
399
|
+ command_context,
|
|
|
400
|
+ verify=False,
|
|
|
401
|
+ **{name: f"{padding}{value}{padding}" for name, value in _CLI_ARGS.items()},
|
|
|
402
|
+ )
|
|
|
403
|
+
|
|
|
404
|
+ assert result == 0
|
|
|
405
|
+ with open(tmp_path / common.YEET_CONFIG_FILENAME) as f:
|
|
|
406
|
+ assert json.load(f) == _SAVED_CONFIG
|
|
|
407
|
+
|
|
|
408
|
+
|
|
|
409
|
+def test_run_trims_prompted_values(tmp_path, monkeypatch):
|
|
|
410
|
+ _set_inputs(monkeypatch, " bea\t", " build.example.com\n", "\tbea ")
|
|
|
411
|
+ monkeypatch.setattr(common.getpass, "getpass", lambda prompt="": " s3cr3t\t")
|
|
|
412
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
413
|
+
|
|
|
414
|
+ result = auth.run(command_context, verify=False)
|
|
|
415
|
+
|
|
|
416
|
+ assert result == 0
|
|
|
417
|
+ with open(tmp_path / common.YEET_CONFIG_FILENAME) as f:
|
|
|
418
|
+ assert json.load(f) == _SAVED_CONFIG
|
|
|
419
|
+
|
|
|
420
|
+
|
|
|
421
|
+@pytest.mark.parametrize("arg", list(_CLI_ARGS))
|
|
|
422
|
+@pytest.mark.parametrize("blank", ["", " ", "\t\n"])
|
|
|
423
|
+def test_run_rejects_blank_cli_arg(tmp_path, monkeypatch, capsys, arg, blank):
|
|
|
424
|
+ _forbid_prompts(monkeypatch)
|
|
|
425
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
426
|
+
|
|
|
427
|
+ result = auth.run(command_context, verify=False, **{**_CLI_ARGS, arg: blank})
|
|
|
428
|
+
|
|
|
429
|
+ assert result == 1
|
|
|
430
|
+ assert not (tmp_path / common.YEET_CONFIG_FILENAME).exists()
|
|
|
431
|
+ assert "ERROR!" in capsys.readouterr().out
|
|
|
432
|
+
|
|
|
433
|
+
|
|
|
434
|
+if __name__ == "__main__":
|
|
|
435
|
+ mozunit.main() |
tools/yeet/tests/python/test_common.py
|
|
1
|
+# This Source Code Form is subject to the terms of the Mozilla Public
|
|
|
2
|
+# License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
|
3
|
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
|
4
|
+
|
|
|
5
|
+import sys
|
|
|
6
|
+from types import SimpleNamespace
|
|
|
7
|
+
|
|
|
8
|
+import mozunit
|
|
|
9
|
+import pytest
|
|
|
10
|
+from yeet import common
|
|
|
11
|
+
|
|
|
12
|
+
|
|
|
13
|
+@pytest.mark.parametrize("secret", [False, True])
|
|
|
14
|
+def test_prompt_flushes_pending_tty_input(monkeypatch, secret):
|
|
|
15
|
+ termios = pytest.importorskip("termios")
|
|
|
16
|
+ events = []
|
|
|
17
|
+ monkeypatch.setattr(sys, "stdin", SimpleNamespace(isatty=lambda: True))
|
|
|
18
|
+ monkeypatch.setattr(
|
|
|
19
|
+ termios, "tcflush", lambda fd, queue: events.append(("flush", queue))
|
|
|
20
|
+ )
|
|
|
21
|
+ monkeypatch.setattr(
|
|
|
22
|
+ "builtins.input", lambda message="": events.append("input") or "plain"
|
|
|
23
|
+ )
|
|
|
24
|
+ monkeypatch.setattr(
|
|
|
25
|
+ common.getpass,
|
|
|
26
|
+ "getpass",
|
|
|
27
|
+ lambda message="": events.append("getpass") or "hidden",
|
|
|
28
|
+ )
|
|
|
29
|
+
|
|
|
30
|
+ answer = common.prompt("Q: ", secret=secret)
|
|
|
31
|
+
|
|
|
32
|
+ assert answer == ("hidden" if secret else "plain")
|
|
|
33
|
+ assert events == [
|
|
|
34
|
+ ("flush", termios.TCIFLUSH),
|
|
|
35
|
+ "getpass" if secret else "input",
|
|
|
36
|
+ ]
|
|
|
37
|
+
|
|
|
38
|
+
|
|
|
39
|
+def test_prompt_skips_flush_when_not_a_tty(monkeypatch):
|
|
|
40
|
+ termios = pytest.importorskip("termios")
|
|
|
41
|
+ monkeypatch.setattr(sys, "stdin", SimpleNamespace(isatty=lambda: False))
|
|
|
42
|
+ monkeypatch.setattr(
|
|
|
43
|
+ termios,
|
|
|
44
|
+ "tcflush",
|
|
|
45
|
+ lambda *args: pytest.fail("tcflush called on non-tty stdin"),
|
|
|
46
|
+ )
|
|
|
47
|
+ monkeypatch.setattr("builtins.input", lambda message="": "piped")
|
|
|
48
|
+
|
|
|
49
|
+ assert common.prompt("Q: ") == "piped"
|
|
|
50
|
+
|
|
|
51
|
+
|
|
|
52
|
+@pytest.mark.parametrize(
|
|
|
53
|
+ "answers,expected",
|
|
|
54
|
+ [
|
|
|
55
|
+ (["y"], True),
|
|
|
56
|
+ (["YES"], True),
|
|
|
57
|
+ ([" yes "], True),
|
|
|
58
|
+ ([""], False),
|
|
|
59
|
+ (["n"], False),
|
|
|
60
|
+ (["No"], False),
|
|
|
61
|
+ (["maybe", "y"], True),
|
|
|
62
|
+ ],
|
|
|
63
|
+)
|
|
|
64
|
+def test_confirm(monkeypatch, answers, expected):
|
|
|
65
|
+ answers = iter(answers)
|
|
|
66
|
+ monkeypatch.setattr("builtins.input", lambda message="": next(answers))
|
|
|
67
|
+
|
|
|
68
|
+ assert common.confirm("Go?") is expected
|
|
|
69
|
+
|
|
|
70
|
+
|
|
|
71
|
+def test_confirm_reprompts_on_invalid_answer(monkeypatch, capsys):
|
|
|
72
|
+ answers = iter(["maybe", "sure", "n"])
|
|
|
73
|
+ monkeypatch.setattr("builtins.input", lambda message="": next(answers))
|
|
|
74
|
+
|
|
|
75
|
+ assert common.confirm("Go?") is False
|
|
|
76
|
+ assert capsys.readouterr().out.count("ERROR!") == 2
|
|
|
77
|
+
|
|
|
78
|
+
|
|
|
79
|
+if __name__ == "__main__":
|
|
|
80
|
+ mozunit.main() |
tools/yeet/tests/python/test_remote.py
|
|
1
|
+# This Source Code Form is subject to the terms of the Mozilla Public
|
|
|
2
|
+# License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
|
3
|
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
|
4
|
+
|
|
|
5
|
+import json
|
|
|
6
|
+import shlex
|
|
|
7
|
+import urllib.error
|
|
|
8
|
+from types import SimpleNamespace
|
|
|
9
|
+
|
|
|
10
|
+import mozunit
|
|
|
11
|
+import pytest
|
|
|
12
|
+from conftest import write_config
|
|
|
13
|
+from yeet import remote
|
|
|
14
|
+
|
|
|
15
|
+
|
|
|
16
|
+class _FakeResponse:
|
|
|
17
|
+ def __init__(self, body):
|
|
|
18
|
+ self._body = body
|
|
|
19
|
+
|
|
|
20
|
+ def __enter__(self):
|
|
|
21
|
+ return self
|
|
|
22
|
+
|
|
|
23
|
+ def __exit__(self, *args):
|
|
|
24
|
+ return False
|
|
|
25
|
+
|
|
|
26
|
+ def read(self):
|
|
|
27
|
+ return self._body
|
|
|
28
|
+
|
|
|
29
|
+
|
|
|
30
|
+def _write_version(tmp_path, version="153.2.0"):
|
|
|
31
|
+ version_dir = tmp_path / "browser" / "config"
|
|
|
32
|
+ version_dir.mkdir(parents=True)
|
|
|
33
|
+ (version_dir / "version.txt").write_text(f"{version}\n")
|
|
|
34
|
+
|
|
|
35
|
+
|
|
|
36
|
+def test_read_browser_version(tmp_path):
|
|
|
37
|
+ _write_version(tmp_path, "1.2.3")
|
|
|
38
|
+
|
|
|
39
|
+ assert remote._read_browser_version(tmp_path) == "1.2.3"
|
|
|
40
|
+
|
|
|
41
|
+
|
|
|
42
|
+def test_resolve_version_prefers_given_value(tmp_path):
|
|
|
43
|
+
|
|
|
44
|
+ assert remote._resolve_version(tmp_path, "9.9.9") == "9.9.9"
|
|
|
45
|
+
|
|
|
46
|
+
|
|
|
47
|
+def test_resolve_version_falls_back_to_in_tree_version(tmp_path, capsys):
|
|
|
48
|
+ _write_version(tmp_path, "1.2.3")
|
|
|
49
|
+
|
|
|
50
|
+ assert remote._resolve_version(tmp_path, None) == "1.2.3"
|
|
|
51
|
+ assert "1.2.3" in capsys.readouterr().out
|
|
|
52
|
+
|
|
|
53
|
+
|
|
|
54
|
+def test_artifacts_and_mozharness_urls():
|
|
|
55
|
+ artifacts = remote._artifacts_url("1.2.3", "android_x86_64")
|
|
|
56
|
+
|
|
|
57
|
+ assert artifacts == f"{remote.TOOLCHAINS_URL_BASE}/1.2.3/android-x86_64/"
|
|
|
58
|
+ assert remote._mozharness_url("1.2.3", "android_x86_64") == (
|
|
|
59
|
+ artifacts + "mozharness.zip"
|
|
|
60
|
+ )
|
|
|
61
|
+
|
|
|
62
|
+
|
|
|
63
|
+def test_curl_command_includes_header_data_and_url():
|
|
|
64
|
+ command = remote._curl_command(
|
|
|
65
|
+ "https://example.org/trigger",
|
|
|
66
|
+ {"Content-Type": "application/json"},
|
|
|
67
|
+ b'{"a": 1}',
|
|
|
68
|
+ )
|
|
|
69
|
+
|
|
|
70
|
+ assert shlex.split(command) == [
|
|
|
71
|
+ "curl",
|
|
|
72
|
+ "--request",
|
|
|
73
|
+ "POST",
|
|
|
74
|
+ "--header",
|
|
|
75
|
+ "Content-Type: application/json",
|
|
|
76
|
+ "--data",
|
|
|
77
|
+ '{"a": 1}',
|
|
|
78
|
+ "https://example.org/trigger",
|
|
|
79
|
+ ]
|
|
|
80
|
+
|
|
|
81
|
+
|
|
|
82
|
+def _failing_upload_run(topsrcdir, file, platform, assume_yes):
|
|
|
83
|
+ raise RuntimeError("boom")
|
|
|
84
|
+
|
|
|
85
|
+
|
|
|
86
|
+def test_resolve_installer_url_passes_through_a_url(tmp_path):
|
|
|
87
|
+
|
|
|
88
|
+ url = remote._resolve_installer_url(
|
|
|
89
|
+ tmp_path,
|
|
|
90
|
+ "https://example.org/app.apk",
|
|
|
91
|
+ "android_x86_64",
|
|
|
92
|
+ assume_yes=False,
|
|
|
93
|
+ dry_run=False,
|
|
|
94
|
+ )
|
|
|
95
|
+
|
|
|
96
|
+ assert url == "https://example.org/app.apk"
|
|
|
97
|
+
|
|
|
98
|
+
|
|
|
99
|
+def test_resolve_installer_url_uploads_a_local_path(tmp_path, monkeypatch):
|
|
|
100
|
+ calls = []
|
|
|
101
|
+
|
|
|
102
|
+ def fake_upload_run(topsrcdir, file, platform, assume_yes):
|
|
|
103
|
+ calls.append((topsrcdir, file, platform, assume_yes))
|
|
|
104
|
+ return "https://tb-build-03.torproject.org/~bea/yeet/ts.apk"
|
|
|
105
|
+
|
|
|
106
|
+ monkeypatch.setattr(remote.upload, "run", fake_upload_run)
|
|
|
107
|
+
|
|
|
108
|
+ url = remote._resolve_installer_url(
|
|
|
109
|
+ tmp_path,
|
|
|
110
|
+ "/tmp/app.apk",
|
|
|
111
|
+ "android_x86_64",
|
|
|
112
|
+ assume_yes=True,
|
|
|
113
|
+ dry_run=False,
|
|
|
114
|
+ )
|
|
|
115
|
+
|
|
|
116
|
+ assert url == "https://tb-build-03.torproject.org/~bea/yeet/ts.apk"
|
|
|
117
|
+ assert calls == [(tmp_path, "/tmp/app.apk", "android_x86_64", True)]
|
|
|
118
|
+
|
|
|
119
|
+
|
|
|
120
|
+def test_resolve_installer_url_propagates_upload_error(tmp_path, monkeypatch):
|
|
|
121
|
+ monkeypatch.setattr(remote.upload, "run", _failing_upload_run)
|
|
|
122
|
+
|
|
|
123
|
+ with pytest.raises(RuntimeError, match="boom"):
|
|
|
124
|
+ remote._resolve_installer_url(
|
|
|
125
|
+ tmp_path,
|
|
|
126
|
+ "/tmp/app.apk",
|
|
|
127
|
+ "android_x86_64",
|
|
|
128
|
+ assume_yes=False,
|
|
|
129
|
+ dry_run=False,
|
|
|
130
|
+ )
|
|
|
131
|
+
|
|
|
132
|
+
|
|
|
133
|
+def test_prompt_platform_rejects_invalid_choice_then_accepts(monkeypatch):
|
|
|
134
|
+ values = iter(["0", "99", "not-a-number", "2"])
|
|
|
135
|
+ monkeypatch.setattr("builtins.input", lambda prompt="": next(values))
|
|
|
136
|
+
|
|
|
137
|
+ assert remote._prompt_platform() == remote.PLATFORMS[1]
|
|
|
138
|
+
|
|
|
139
|
+
|
|
|
140
|
+def test_prompt_channel_rejects_invalid_choice_then_accepts(monkeypatch):
|
|
|
141
|
+ values = iter(["0", "99", "not-a-number", "2"])
|
|
|
142
|
+ monkeypatch.setattr("builtins.input", lambda prompt="": next(values))
|
|
|
143
|
+
|
|
|
144
|
+ assert remote._prompt_channel() == remote.ANDROID_CHANNELS[1]
|
|
|
145
|
+
|
|
|
146
|
+
|
|
|
147
|
+def _run_kwargs(**overrides):
|
|
|
148
|
+ kwargs = dict(
|
|
|
149
|
+ platform="android_x86_64",
|
|
|
150
|
+ channel="debug",
|
|
|
151
|
+ installer_url="https://example.org/app.apk",
|
|
|
152
|
+ sha256sums_url=None,
|
|
|
153
|
+ artifacts_url="https://example.org/artifacts/",
|
|
|
154
|
+ mozharness_url="https://example.org/mozharness.zip",
|
|
|
155
|
+ version=None,
|
|
|
156
|
+ ref="main",
|
|
|
157
|
+ tags=["tor"],
|
|
|
158
|
+ dry_run=False,
|
|
|
159
|
+ )
|
|
|
160
|
+ kwargs.update(overrides)
|
|
|
161
|
+ return kwargs
|
|
|
162
|
+
|
|
|
163
|
+
|
|
|
164
|
+def test_run_aborts_without_config(tmp_path, capsys):
|
|
|
165
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
166
|
+
|
|
|
167
|
+ result = remote.run(command_context, **_run_kwargs())
|
|
|
168
|
+
|
|
|
169
|
+ assert result == 1
|
|
|
170
|
+ assert "mach yeet auth" in capsys.readouterr().out
|
|
|
171
|
+
|
|
|
172
|
+
|
|
|
173
|
+def test_run_dry_run_prints_curl_and_skips_network(tmp_path, monkeypatch, capsys):
|
|
|
174
|
+ write_config(tmp_path)
|
|
|
175
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
176
|
+
|
|
|
177
|
+ def fail_urlopen(*args, **kwargs):
|
|
|
178
|
+ raise AssertionError("urlopen should not be called in dry-run mode")
|
|
|
179
|
+
|
|
|
180
|
+ monkeypatch.setattr(remote.urllib.request, "urlopen", fail_urlopen)
|
|
|
181
|
+
|
|
|
182
|
+ result = remote.run(command_context, **_run_kwargs(dry_run=True))
|
|
|
183
|
+
|
|
|
184
|
+ assert result == 0
|
|
|
185
|
+ out = capsys.readouterr().out
|
|
|
186
|
+ assert "curl --request POST" in out
|
|
|
187
|
+ assert "app.apk" in out
|
|
|
188
|
+
|
|
|
189
|
+
|
|
|
190
|
+def test_run_prompts_for_platform_when_missing(tmp_path, monkeypatch, capsys):
|
|
|
191
|
+ write_config(tmp_path)
|
|
|
192
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
193
|
+ calls = []
|
|
|
194
|
+
|
|
|
195
|
+ def fake_prompt_platform():
|
|
|
196
|
+ calls.append(True)
|
|
|
197
|
+ return "debian_x86_64"
|
|
|
198
|
+
|
|
|
199
|
+ monkeypatch.setattr(remote, "_prompt_platform", fake_prompt_platform)
|
|
|
200
|
+
|
|
|
201
|
+ result = remote.run(command_context, **_run_kwargs(platform=None, dry_run=True))
|
|
|
202
|
+
|
|
|
203
|
+ assert result == 0
|
|
|
204
|
+ assert len(calls) == 1
|
|
|
205
|
+ assert "debian_x86_64_installer_url" in capsys.readouterr().out
|
|
|
206
|
+
|
|
|
207
|
+
|
|
|
208
|
+def test_run_prompts_for_channel_when_android_and_missing(
|
|
|
209
|
+ tmp_path, monkeypatch, capsys
|
|
|
210
|
+):
|
|
|
211
|
+ write_config(tmp_path)
|
|
|
212
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
213
|
+ calls = []
|
|
|
214
|
+
|
|
|
215
|
+ def fake_prompt_channel():
|
|
|
216
|
+ calls.append(True)
|
|
|
217
|
+ return "nightly"
|
|
|
218
|
+
|
|
|
219
|
+ monkeypatch.setattr(remote, "_prompt_channel", fake_prompt_channel)
|
|
|
220
|
+
|
|
|
221
|
+ result = remote.run(command_context, **_run_kwargs(channel=None, dry_run=True))
|
|
|
222
|
+
|
|
|
223
|
+ assert result == 0
|
|
|
224
|
+ assert len(calls) == 1
|
|
|
225
|
+ assert remote.ANDROID_CHANNEL_PACKAGE_NAMES["nightly"] in capsys.readouterr().out
|
|
|
226
|
+
|
|
|
227
|
+
|
|
|
228
|
+def test_run_adds_package_name_input_for_android(tmp_path, capsys):
|
|
|
229
|
+ write_config(tmp_path)
|
|
|
230
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
231
|
+
|
|
|
232
|
+ result = remote.run(command_context, **_run_kwargs(channel="nightly", dry_run=True))
|
|
|
233
|
+
|
|
|
234
|
+ assert result == 0
|
|
|
235
|
+ out = capsys.readouterr().out
|
|
|
236
|
+ assert "android_x86_64_package_name" in out
|
|
|
237
|
+ assert remote.ANDROID_CHANNEL_PACKAGE_NAMES["nightly"] in out
|
|
|
238
|
+
|
|
|
239
|
+
|
|
|
240
|
+def test_run_skips_channel_for_non_android_platforms(tmp_path, monkeypatch, capsys):
|
|
|
241
|
+ write_config(tmp_path)
|
|
|
242
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
243
|
+ monkeypatch.setattr(
|
|
|
244
|
+ remote,
|
|
|
245
|
+ "_prompt_channel",
|
|
|
246
|
+ lambda: (_ for _ in ()).throw(AssertionError("should not prompt")),
|
|
|
247
|
+ )
|
|
|
248
|
+
|
|
|
249
|
+ result = remote.run(
|
|
|
250
|
+ command_context,
|
|
|
251
|
+ **_run_kwargs(
|
|
|
252
|
+ platform="debian_x86_64",
|
|
|
253
|
+ channel=None,
|
|
|
254
|
+ artifacts_url="https://example.org/artifacts/",
|
|
|
255
|
+ mozharness_url="https://example.org/mozharness.zip",
|
|
|
256
|
+ dry_run=True,
|
|
|
257
|
+ ),
|
|
|
258
|
+ )
|
|
|
259
|
+
|
|
|
260
|
+ assert result == 0
|
|
|
261
|
+ out = capsys.readouterr().out
|
|
|
262
|
+ assert "package_name" not in out
|
|
|
263
|
+
|
|
|
264
|
+
|
|
|
265
|
+def test_run_rejects_unknown_channel(tmp_path, capsys):
|
|
|
266
|
+ write_config(tmp_path)
|
|
|
267
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
268
|
+
|
|
|
269
|
+ result = remote.run(command_context, **_run_kwargs(channel="not-a-channel"))
|
|
|
270
|
+
|
|
|
271
|
+ assert result == 1
|
|
|
272
|
+ assert "ERROR! Unknown channel 'not-a-channel'." in capsys.readouterr().out
|
|
|
273
|
+
|
|
|
274
|
+
|
|
|
275
|
+@pytest.mark.parametrize("dry_run", [False, True])
|
|
|
276
|
+@pytest.mark.parametrize("installer_url", ["", " ", "\t\n"])
|
|
|
277
|
+def test_run_rejects_empty_installer(
|
|
|
278
|
+ tmp_path, monkeypatch, capsys, installer_url, dry_run
|
|
|
279
|
+):
|
|
|
280
|
+ write_config(tmp_path)
|
|
|
281
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
282
|
+ monkeypatch.setattr(
|
|
|
283
|
+ remote, "_prompt_platform", lambda: pytest.fail("prompted for platform")
|
|
|
284
|
+ )
|
|
|
285
|
+ monkeypatch.setattr(
|
|
|
286
|
+ remote.upload, "run", lambda *args, **kwargs: pytest.fail("uploaded")
|
|
|
287
|
+ )
|
|
|
288
|
+ monkeypatch.setattr(
|
|
|
289
|
+ remote.urllib.request,
|
|
|
290
|
+ "urlopen",
|
|
|
291
|
+ lambda *args, **kwargs: pytest.fail("pipeline triggered"),
|
|
|
292
|
+ )
|
|
|
293
|
+
|
|
|
294
|
+ result = remote.run(
|
|
|
295
|
+ command_context,
|
|
|
296
|
+ **_run_kwargs(platform=None, installer_url=installer_url, dry_run=dry_run),
|
|
|
297
|
+ )
|
|
|
298
|
+
|
|
|
299
|
+ assert result == 1
|
|
|
300
|
+ out = capsys.readouterr().out
|
|
|
301
|
+ assert "ERROR! The installer URL or path was empty." in out
|
|
|
302
|
+ assert "curl" not in out
|
|
|
303
|
+
|
|
|
304
|
+
|
|
|
305
|
+def test_run_trims_installer_url(tmp_path, capsys):
|
|
|
306
|
+ write_config(tmp_path)
|
|
|
307
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
308
|
+
|
|
|
309
|
+ result = remote.run(
|
|
|
310
|
+ command_context,
|
|
|
311
|
+ **_run_kwargs(installer_url=" https://example.org/app.apk\n", dry_run=True),
|
|
|
312
|
+ )
|
|
|
313
|
+
|
|
|
314
|
+ assert result == 0
|
|
|
315
|
+ assert '"android_x86_64_installer_url": "https://example.org/app.apk"' in (
|
|
|
316
|
+ capsys.readouterr().out
|
|
|
317
|
+ )
|
|
|
318
|
+
|
|
|
319
|
+
|
|
|
320
|
+def test_run_ignores_channel_for_non_android_platforms(tmp_path, capsys):
|
|
|
321
|
+ write_config(tmp_path)
|
|
|
322
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
323
|
+
|
|
|
324
|
+ result = remote.run(
|
|
|
325
|
+ command_context,
|
|
|
326
|
+ **_run_kwargs(platform="debian_x86_64", channel="nightly", dry_run=True),
|
|
|
327
|
+ )
|
|
|
328
|
+
|
|
|
329
|
+ assert result == 0
|
|
|
330
|
+ out = capsys.readouterr().out
|
|
|
331
|
+ assert "package_name" not in out
|
|
|
332
|
+ assert remote.ANDROID_CHANNEL_PACKAGE_NAMES["nightly"] not in out
|
|
|
333
|
+
|
|
|
334
|
+
|
|
|
335
|
+@pytest.mark.parametrize(
|
|
|
336
|
+ "platform,installer",
|
|
|
337
|
+ [
|
|
|
338
|
+ ("debian_x86_64", "tor-browser-linux-x86_64.tar.xz"),
|
|
|
339
|
+ ("windows_x86_64", "tor-browser-windows-x86_64-portable.exe"),
|
|
|
340
|
+ ("macos_x86_64", "tor-browser-macos.dmg"),
|
|
|
341
|
+ ("android_x86_64", "tor-browser-android-x86_64.apk"),
|
|
|
342
|
+ ],
|
|
|
343
|
+)
|
|
|
344
|
+def test_run_uploads_a_local_path_before_triggering(
|
|
|
345
|
+ tmp_path, monkeypatch, platform, installer
|
|
|
346
|
+):
|
|
|
347
|
+ write_config(tmp_path)
|
|
|
348
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
349
|
+ installer_path = f"/tmp/{installer}"
|
|
|
350
|
+ uploaded_url = (
|
|
|
351
|
+ f"https://tb-build-03.torproject.org/~bea/yeet/ts-{platform}/{installer}"
|
|
|
352
|
+ )
|
|
|
353
|
+ uploads = []
|
|
|
354
|
+
|
|
|
355
|
+ def fake_upload_run(topsrcdir, file, platform, assume_yes):
|
|
|
356
|
+ uploads.append((file, platform))
|
|
|
357
|
+ return uploaded_url
|
|
|
358
|
+
|
|
|
359
|
+ monkeypatch.setattr(remote.upload, "run", fake_upload_run)
|
|
|
360
|
+ requests = []
|
|
|
361
|
+
|
|
|
362
|
+ def fake_urlopen(request, timeout=None):
|
|
|
363
|
+ requests.append(request)
|
|
|
364
|
+ return _FakeResponse(json.dumps({"web_url": "https://x"}).encode())
|
|
|
365
|
+
|
|
|
366
|
+ monkeypatch.setattr(remote.urllib.request, "urlopen", fake_urlopen)
|
|
|
367
|
+
|
|
|
368
|
+ result = remote.run(
|
|
|
369
|
+ command_context,
|
|
|
370
|
+ **_run_kwargs(platform=platform, installer_url=installer_path),
|
|
|
371
|
+ )
|
|
|
372
|
+
|
|
|
373
|
+ assert result == 0
|
|
|
374
|
+ assert uploads == [(installer_path, platform)]
|
|
|
375
|
+ inputs = json.loads(requests[0].data)["inputs"]
|
|
|
376
|
+ assert inputs[f"{platform}_installer_url"] == uploaded_url
|
|
|
377
|
+ assert (f"{platform}_package_name" in inputs) == (platform == "android_x86_64")
|
|
|
378
|
+
|
|
|
379
|
+
|
|
|
380
|
+def test_run_prompts_for_platform_before_uploading(tmp_path, monkeypatch):
|
|
|
381
|
+ write_config(tmp_path)
|
|
|
382
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
383
|
+ monkeypatch.setattr(remote, "_prompt_platform", lambda: "debian_x86_64")
|
|
|
384
|
+ seen = []
|
|
|
385
|
+
|
|
|
386
|
+ def fake_upload_run(topsrcdir, file, platform, assume_yes):
|
|
|
387
|
+ seen.append(platform)
|
|
|
388
|
+ raise RuntimeError("stop here")
|
|
|
389
|
+
|
|
|
390
|
+ monkeypatch.setattr(remote.upload, "run", fake_upload_run)
|
|
|
391
|
+
|
|
|
392
|
+ remote.run(
|
|
|
393
|
+ command_context, **_run_kwargs(platform=None, installer_url="/tmp/app.tar.xz")
|
|
|
394
|
+ )
|
|
|
395
|
+
|
|
|
396
|
+ assert seen == ["debian_x86_64"]
|
|
|
397
|
+
|
|
|
398
|
+
|
|
|
399
|
+@pytest.mark.parametrize("assume_yes", [False, True])
|
|
|
400
|
+def test_run_forwards_assume_yes_to_upload(tmp_path, monkeypatch, assume_yes):
|
|
|
401
|
+ write_config(tmp_path)
|
|
|
402
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
403
|
+ seen = []
|
|
|
404
|
+
|
|
|
405
|
+ def fake_upload_run(topsrcdir, file, platform, assume_yes):
|
|
|
406
|
+ seen.append(assume_yes)
|
|
|
407
|
+ raise RuntimeError("stop here")
|
|
|
408
|
+
|
|
|
409
|
+ monkeypatch.setattr(remote.upload, "run", fake_upload_run)
|
|
|
410
|
+
|
|
|
411
|
+ remote.run(
|
|
|
412
|
+ command_context,
|
|
|
413
|
+ **_run_kwargs(installer_url="/tmp/app.apk", assume_yes=assume_yes),
|
|
|
414
|
+ )
|
|
|
415
|
+
|
|
|
416
|
+ assert seen == [assume_yes]
|
|
|
417
|
+
|
|
|
418
|
+
|
|
|
419
|
+@pytest.mark.parametrize(
|
|
|
420
|
+ "tags,expected",
|
|
|
421
|
+ [
|
|
|
422
|
+ (["tor", "base-browser"], "--tag tor --tag base-browser"),
|
|
|
423
|
+ (["a b"], "--tag 'a b'"),
|
|
|
424
|
+ (["a b", '"c"'], """--tag 'a b' --tag '"c"'"""),
|
|
|
425
|
+ ],
|
|
|
426
|
+)
|
|
|
427
|
+def test_run_formats_tags_as_flags(tmp_path, monkeypatch, tags, expected):
|
|
|
428
|
+ write_config(tmp_path)
|
|
|
429
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
430
|
+ requests = []
|
|
|
431
|
+
|
|
|
432
|
+ def fake_urlopen(request, timeout=None):
|
|
|
433
|
+ requests.append(request)
|
|
|
434
|
+ return _FakeResponse(json.dumps({"web_url": "https://x"}).encode())
|
|
|
435
|
+
|
|
|
436
|
+ monkeypatch.setattr(remote.urllib.request, "urlopen", fake_urlopen)
|
|
|
437
|
+
|
|
|
438
|
+ remote.run(command_context, **_run_kwargs(tags=tags))
|
|
|
439
|
+
|
|
|
440
|
+ assert json.loads(requests[0].data)["inputs"]["tags"] == expected
|
|
|
441
|
+
|
|
|
442
|
+
|
|
|
443
|
+def test_run_omits_tags_input_when_empty(tmp_path, monkeypatch):
|
|
|
444
|
+ write_config(tmp_path)
|
|
|
445
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
446
|
+ requests = []
|
|
|
447
|
+
|
|
|
448
|
+ def fake_urlopen(request, timeout=None):
|
|
|
449
|
+ requests.append(request)
|
|
|
450
|
+ return _FakeResponse(json.dumps({"web_url": "https://x"}).encode())
|
|
|
451
|
+
|
|
|
452
|
+ monkeypatch.setattr(remote.urllib.request, "urlopen", fake_urlopen)
|
|
|
453
|
+
|
|
|
454
|
+ remote.run(command_context, **_run_kwargs(tags=[]))
|
|
|
455
|
+
|
|
|
456
|
+ assert "tags" not in json.loads(requests[0].data)["inputs"]
|
|
|
457
|
+
|
|
|
458
|
+
|
|
|
459
|
+def _triggered_inputs(tmp_path, monkeypatch, **overrides):
|
|
|
460
|
+ write_config(tmp_path)
|
|
|
461
|
+ _write_version(tmp_path, "1.2.3")
|
|
|
462
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
463
|
+ requests = []
|
|
|
464
|
+
|
|
|
465
|
+ def fake_urlopen(request, timeout=None):
|
|
|
466
|
+ requests.append(request)
|
|
|
467
|
+ return _FakeResponse(json.dumps({"web_url": "https://x"}).encode())
|
|
|
468
|
+
|
|
|
469
|
+ monkeypatch.setattr(remote.urllib.request, "urlopen", fake_urlopen)
|
|
|
470
|
+
|
|
|
471
|
+ assert remote.run(command_context, **_run_kwargs(**overrides)) == 0
|
|
|
472
|
+ return json.loads(requests[0].data)["inputs"]
|
|
|
473
|
+
|
|
|
474
|
+
|
|
|
475
|
+@pytest.mark.parametrize("sha256sums_url", [None, "https://example.org/sha256sums.txt"])
|
|
|
476
|
+def test_run_includes_sha256sums_input_only_when_given(
|
|
|
477
|
+ tmp_path, monkeypatch, sha256sums_url
|
|
|
478
|
+):
|
|
|
479
|
+ inputs = _triggered_inputs(tmp_path, monkeypatch, sha256sums_url=sha256sums_url)
|
|
|
480
|
+
|
|
|
481
|
+ assert inputs.get("android_x86_64_sha256sums_url") == sha256sums_url
|
|
|
482
|
+
|
|
|
483
|
+
|
|
|
484
|
+@pytest.mark.parametrize("artifacts_url", [None, "https://example.org/artifacts/"])
|
|
|
485
|
+@pytest.mark.parametrize("mozharness_url", [None, "https://example.org/mozharness.zip"])
|
|
|
486
|
+def test_run_uses_given_artifacts_and_mozharness_urls_or_infers_them(
|
|
|
487
|
+ tmp_path, monkeypatch, artifacts_url, mozharness_url
|
|
|
488
|
+):
|
|
|
489
|
+ inputs = _triggered_inputs(
|
|
|
490
|
+ tmp_path,
|
|
|
491
|
+ monkeypatch,
|
|
|
492
|
+ artifacts_url=artifacts_url,
|
|
|
493
|
+ mozharness_url=mozharness_url,
|
|
|
494
|
+ )
|
|
|
495
|
+
|
|
|
496
|
+ assert inputs["android_x86_64_artifacts_url"] == (
|
|
|
497
|
+ artifacts_url or remote._artifacts_url("1.2.3", "android_x86_64")
|
|
|
498
|
+ )
|
|
|
499
|
+ assert inputs["mozharness_url"] == (
|
|
|
500
|
+ mozharness_url or remote._mozharness_url("1.2.3", "android_x86_64")
|
|
|
501
|
+ )
|
|
|
502
|
+
|
|
|
503
|
+
|
|
|
504
|
+def test_run_aborts_when_local_upload_fails(tmp_path, monkeypatch, capsys):
|
|
|
505
|
+ write_config(tmp_path)
|
|
|
506
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
507
|
+ monkeypatch.setattr(remote.upload, "run", _failing_upload_run)
|
|
|
508
|
+ monkeypatch.setattr(
|
|
|
509
|
+ remote.urllib.request,
|
|
|
510
|
+ "urlopen",
|
|
|
511
|
+ lambda *args, **kwargs: pytest.fail("pipeline triggered after failed upload"),
|
|
|
512
|
+ )
|
|
|
513
|
+
|
|
|
514
|
+ result = remote.run(command_context, **_run_kwargs(installer_url="/tmp/app.apk"))
|
|
|
515
|
+
|
|
|
516
|
+ assert result == 1
|
|
|
517
|
+ assert "ERROR! boom" in capsys.readouterr().out
|
|
|
518
|
+
|
|
|
519
|
+
|
|
|
520
|
+def test_run_dry_run_does_not_upload_a_local_path(tmp_path, monkeypatch, capsys):
|
|
|
521
|
+ write_config(tmp_path)
|
|
|
522
|
+ apk = tmp_path / "app.apk"
|
|
|
523
|
+ apk.write_bytes(b"not really an apk")
|
|
|
524
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
525
|
+ monkeypatch.setattr(
|
|
|
526
|
+ remote.upload,
|
|
|
527
|
+ "run",
|
|
|
528
|
+ lambda *args, **kwargs: pytest.fail("uploaded during a dry run"),
|
|
|
529
|
+ )
|
|
|
530
|
+
|
|
|
531
|
+ result = remote.run(
|
|
|
532
|
+ command_context, **_run_kwargs(installer_url=str(apk), dry_run=True)
|
|
|
533
|
+ )
|
|
|
534
|
+
|
|
|
535
|
+ assert result == 0
|
|
|
536
|
+ assert "<URL of uploaded app.apk>" in capsys.readouterr().out
|
|
|
537
|
+
|
|
|
538
|
+
|
|
|
539
|
+def test_run_dry_run_rejects_missing_local_path(tmp_path, capsys):
|
|
|
540
|
+ write_config(tmp_path)
|
|
|
541
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
542
|
+ missing = tmp_path / "missing.apk"
|
|
|
543
|
+
|
|
|
544
|
+ result = remote.run(
|
|
|
545
|
+ command_context, **_run_kwargs(installer_url=str(missing), dry_run=True)
|
|
|
546
|
+ )
|
|
|
547
|
+
|
|
|
548
|
+ assert result == 1
|
|
|
549
|
+ out = capsys.readouterr().out
|
|
|
550
|
+ assert f"ERROR! No such file: {missing}" in out
|
|
|
551
|
+ assert "curl" not in out
|
|
|
552
|
+
|
|
|
553
|
+
|
|
|
554
|
+def test_run_infers_version_artifacts_and_mozharness(tmp_path, capsys):
|
|
|
555
|
+ write_config(tmp_path)
|
|
|
556
|
+ _write_version(tmp_path, "1.2.3")
|
|
|
557
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
558
|
+
|
|
|
559
|
+ result = remote.run(
|
|
|
560
|
+ command_context,
|
|
|
561
|
+ **_run_kwargs(artifacts_url=None, mozharness_url=None, dry_run=True),
|
|
|
562
|
+ )
|
|
|
563
|
+
|
|
|
564
|
+ assert result == 0
|
|
|
565
|
+ out = capsys.readouterr().out
|
|
|
566
|
+ assert "1.2.3" in out
|
|
|
567
|
+ assert "android-x86_64" in out
|
|
|
568
|
+
|
|
|
569
|
+
|
|
|
570
|
+def test_run_triggers_pipeline_and_reports_url(tmp_path, monkeypatch, capsys):
|
|
|
571
|
+ config = write_config(tmp_path)
|
|
|
572
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
573
|
+
|
|
|
574
|
+ requests = []
|
|
|
575
|
+
|
|
|
576
|
+ def fake_urlopen(request, timeout=None):
|
|
|
577
|
+ requests.append(request)
|
|
|
578
|
+ body = json.dumps({"web_url": "https://gitlab.example/pipelines/1"}).encode()
|
|
|
579
|
+ return _FakeResponse(body)
|
|
|
580
|
+
|
|
|
581
|
+ monkeypatch.setattr(remote.urllib.request, "urlopen", fake_urlopen)
|
|
|
582
|
+
|
|
|
583
|
+ result = remote.run(command_context, **_run_kwargs())
|
|
|
584
|
+
|
|
|
585
|
+ assert result == 0
|
|
|
586
|
+ assert "https://gitlab.example/pipelines/1" in capsys.readouterr().out
|
|
|
587
|
+ assert requests[0].full_url == (
|
|
|
588
|
+ f"{remote.GITLAB_API_BASE}/projects/"
|
|
|
589
|
+ f"{config['project'].replace('/', '%2F')}/trigger/pipeline"
|
|
|
590
|
+ )
|
|
|
591
|
+ payload = json.loads(requests[0].data)
|
|
|
592
|
+ assert payload["token"] == config["token"]
|
|
|
593
|
+ assert payload["ref"] == "main"
|
|
|
594
|
+ assert payload["inputs"]["tags"] == "--tag tor"
|
|
|
595
|
+
|
|
|
596
|
+
|
|
|
597
|
+def test_run_reports_http_error(tmp_path, monkeypatch, capsys):
|
|
|
598
|
+ write_config(tmp_path)
|
|
|
599
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
600
|
+
|
|
|
601
|
+ def fake_urlopen(request, timeout=None):
|
|
|
602
|
+ raise urllib.error.HTTPError(
|
|
|
603
|
+ request.full_url,
|
|
|
604
|
+ 401,
|
|
|
605
|
+ "Unauthorized",
|
|
|
606
|
+ None,
|
|
|
607
|
+ SimpleNamespace(read=lambda: b"bad token", close=lambda: None),
|
|
|
608
|
+ )
|
|
|
609
|
+
|
|
|
610
|
+ monkeypatch.setattr(remote.urllib.request, "urlopen", fake_urlopen)
|
|
|
611
|
+
|
|
|
612
|
+ result = remote.run(command_context, **_run_kwargs())
|
|
|
613
|
+
|
|
|
614
|
+ assert result == 1
|
|
|
615
|
+ out = capsys.readouterr().out
|
|
|
616
|
+ assert "401" in out
|
|
|
617
|
+ assert "bad token" in out
|
|
|
618
|
+
|
|
|
619
|
+
|
|
|
620
|
+def test_run_reports_url_error(tmp_path, monkeypatch, capsys):
|
|
|
621
|
+ write_config(tmp_path)
|
|
|
622
|
+ command_context = SimpleNamespace(topsrcdir=str(tmp_path))
|
|
|
623
|
+
|
|
|
624
|
+ def fake_urlopen(request, timeout=None):
|
|
|
625
|
+ raise urllib.error.URLError("unreachable")
|
|
|
626
|
+
|
|
|
627
|
+ monkeypatch.setattr(remote.urllib.request, "urlopen", fake_urlopen)
|
|
|
628
|
+
|
|
|
629
|
+ result = remote.run(command_context, **_run_kwargs())
|
|
|
630
|
+
|
|
|
631
|
+ assert result == 1
|
|
|
632
|
+ assert "unreachable" in capsys.readouterr().out
|
|
|
633
|
+
|
|
|
634
|
+
|
|
|
635
|
+if __name__ == "__main__":
|
|
|
636
|
+ mozunit.main() |
tools/yeet/tests/python/test_upload.py
|
|
1
|
+# This Source Code Form is subject to the terms of the Mozilla Public
|
|
|
2
|
+# License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
|
3
|
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
|
4
|
+
|
|
|
5
|
+from datetime import datetime, timedelta, timezone
|
|
|
6
|
+from types import SimpleNamespace
|
|
|
7
|
+
|
|
|
8
|
+import mozunit
|
|
|
9
|
+import pytest
|
|
|
10
|
+from conftest import write_config
|
|
|
11
|
+from yeet import upload
|
|
|
12
|
+
|
|
|
13
|
+
|
|
|
14
|
+def _freeze_time(monkeypatch, fixed):
|
|
|
15
|
+ class _FrozenDatetime(datetime):
|
|
|
16
|
+ @classmethod
|
|
|
17
|
+ def now(cls, tz=None):
|
|
|
18
|
+ return fixed.astimezone(tz)
|
|
|
19
|
+
|
|
|
20
|
+ monkeypatch.setattr(upload, "datetime", _FrozenDatetime)
|
|
|
21
|
+
|
|
|
22
|
+
|
|
|
23
|
+def test_resolve_file_missing():
|
|
|
24
|
+ with pytest.raises(FileNotFoundError, match="No such file"):
|
|
|
25
|
+ upload.resolve_file("/no/such/file/here")
|
|
|
26
|
+
|
|
|
27
|
+
|
|
|
28
|
+def test_resolve_file_existing(tmp_path):
|
|
|
29
|
+ apk = tmp_path / "fenix-x86_64-debug.apk"
|
|
|
30
|
+ apk.write_bytes(b"not really an apk")
|
|
|
31
|
+
|
|
|
32
|
+ assert upload.resolve_file(str(apk)) == apk.resolve()
|
|
|
33
|
+
|
|
|
34
|
+
|
|
|
35
|
+@pytest.mark.parametrize(
|
|
|
36
|
+ "fixed,expected",
|
|
|
37
|
+ [
|
|
|
38
|
+ (datetime(2026, 9, 14, 12, 34, 56, tzinfo=timezone.utc), "20260914-123456"),
|
|
|
39
|
+ (
|
|
|
40
|
+ datetime(
|
|
|
41
|
+ 2026, 9, 14, 18, 4, 56, tzinfo=timezone(timedelta(hours=5, minutes=30))
|
|
|
42
|
+ ),
|
|
|
43
|
+ "20260914-123456",
|
|
|
44
|
+ ),
|
|
|
45
|
+ (
|
|
|
46
|
+ datetime(2026, 9, 14, 21, 0, 0, tzinfo=timezone(timedelta(hours=-5))),
|
|
|
47
|
+ "20260915-020000",
|
|
|
48
|
+ ),
|
|
|
49
|
+ ],
|
|
|
50
|
+)
|
|
|
51
|
+def test_remote_dir_name_is_current_utc_time_and_platform(monkeypatch, fixed, expected):
|
|
|
52
|
+ _freeze_time(monkeypatch, fixed)
|
|
|
53
|
+
|
|
|
54
|
+ assert upload._remote_dir_name("android_x86_64") == f"{expected}-android_x86_64"
|
|
|
55
|
+
|
|
|
56
|
+
|
|
|
57
|
+def test_upload_success(monkeypatch):
|
|
|
58
|
+ calls = []
|
|
|
59
|
+
|
|
|
60
|
+ def fake_run(cmd, **kwargs):
|
|
|
61
|
+ calls.append(cmd)
|
|
|
62
|
+ assert kwargs["stdin"] is upload.subprocess.DEVNULL
|
|
|
63
|
+ return SimpleNamespace(returncode=0)
|
|
|
64
|
+
|
|
|
65
|
+ monkeypatch.setattr(upload.subprocess, "run", fake_run)
|
|
|
66
|
+
|
|
|
67
|
+ url = upload._upload(upload.Path("/tmp/app.apk"), "bea", "example.org", "ts-os")
|
|
|
68
|
+
|
|
|
69
|
+ assert url == "https://example.org/~bea/yeet/ts-os/app.apk"
|
|
|
70
|
+ assert calls[0][0] == "ssh"
|
|
|
71
|
+ assert calls[0][-3:] == ["mkdir", "-p", "~/public_html/yeet/ts-os"]
|
|
|
72
|
+ assert calls[1][0] == "scp"
|
|
|
73
|
+ assert calls[1][-2:] == [
|
|
|
74
|
+ "/tmp/app.apk",
|
|
|
75
|
+ "bea@xxxxxxxxxxx:~/public_html/yeet/ts-os/app.apk",
|
|
|
76
|
+ ]
|
|
|
77
|
+
|
|
|
78
|
+
|
|
|
79
|
+def test_upload_replaces_unsafe_filename_characters(monkeypatch):
|
|
|
80
|
+ calls = []
|
|
|
81
|
+
|
|
|
82
|
+ def fake_run(cmd, **kwargs):
|
|
|
83
|
+ calls.append(cmd)
|
|
|
84
|
+ return SimpleNamespace(returncode=0)
|
|
|
85
|
+
|
|
|
86
|
+ monkeypatch.setattr(upload.subprocess, "run", fake_run)
|
|
|
87
|
+
|
|
|
88
|
+ url = upload._upload(
|
|
|
89
|
+ upload.Path("/tmp/app (1)#%?.apk"), "bea", "example.org", "ts-os"
|
|
|
90
|
+ )
|
|
|
91
|
+
|
|
|
92
|
+ assert url == "https://example.org/~bea/yeet/ts-os/app__1____.apk"
|
|
|
93
|
+ assert calls[1][-2:] == [
|
|
|
94
|
+ "/tmp/app (1)#%?.apk",
|
|
|
95
|
+ "bea@xxxxxxxxxxx:~/public_html/yeet/ts-os/app__1____.apk",
|
|
|
96
|
+ ]
|
|
|
97
|
+
|
|
|
98
|
+
|
|
|
99
|
+def test_upload_stops_if_mkdir_fails(monkeypatch):
|
|
|
100
|
+ calls = []
|
|
|
101
|
+
|
|
|
102
|
+ def fake_run(cmd, **kwargs):
|
|
|
103
|
+ calls.append(cmd)
|
|
|
104
|
+ return SimpleNamespace(returncode=1)
|
|
|
105
|
+
|
|
|
106
|
+ monkeypatch.setattr(upload.subprocess, "run", fake_run)
|
|
|
107
|
+
|
|
|
108
|
+ with pytest.raises(RuntimeError):
|
|
|
109
|
+ upload._upload(upload.Path("/tmp/app.apk"), "bea", "example.org", "ts-os")
|
|
|
110
|
+
|
|
|
111
|
+ assert len(calls) == 1
|
|
|
112
|
+
|
|
|
113
|
+
|
|
|
114
|
+def test_upload_fails_if_scp_fails(monkeypatch):
|
|
|
115
|
+ def fake_run(cmd, **kwargs):
|
|
|
116
|
+ return SimpleNamespace(returncode=0 if cmd[0] == "ssh" else 1)
|
|
|
117
|
+
|
|
|
118
|
+ monkeypatch.setattr(upload.subprocess, "run", fake_run)
|
|
|
119
|
+
|
|
|
120
|
+ with pytest.raises(RuntimeError):
|
|
|
121
|
+ upload._upload(upload.Path("/tmp/app.apk"), "bea", "example.org", "ts-os")
|
|
|
122
|
+
|
|
|
123
|
+
|
|
|
124
|
+def test_run_aborts_without_config(tmp_path):
|
|
|
125
|
+ apk = tmp_path / "app.apk"
|
|
|
126
|
+ apk.touch()
|
|
|
127
|
+
|
|
|
128
|
+ with pytest.raises(RuntimeError, match="mach yeet auth"):
|
|
|
129
|
+ upload.run(tmp_path, str(apk), "android_x86_64")
|
|
|
130
|
+
|
|
|
131
|
+
|
|
|
132
|
+def test_run_aborts_when_file_missing(tmp_path):
|
|
|
133
|
+ write_config(tmp_path)
|
|
|
134
|
+
|
|
|
135
|
+ with pytest.raises(FileNotFoundError):
|
|
|
136
|
+ upload.run(tmp_path, str(tmp_path / "missing.apk"), "android_x86_64")
|
|
|
137
|
+
|
|
|
138
|
+
|
|
|
139
|
+def _write_apk(tmp_path):
|
|
|
140
|
+ apk = tmp_path / "app.apk"
|
|
|
141
|
+ apk.write_bytes(b"not really an apk")
|
|
|
142
|
+ return apk
|
|
|
143
|
+
|
|
|
144
|
+
|
|
|
145
|
+def _set_confirm_answers(monkeypatch, *answers):
|
|
|
146
|
+ answers = iter(answers)
|
|
|
147
|
+ monkeypatch.setattr("builtins.input", lambda prompt="": next(answers))
|
|
|
148
|
+
|
|
|
149
|
+
|
|
|
150
|
+def test_run_uploads_into_timestamped_dir_and_prints_url(tmp_path, monkeypatch, capsys):
|
|
|
151
|
+ config = write_config(tmp_path)
|
|
|
152
|
+ apk = tmp_path / "app.apk"
|
|
|
153
|
+ apk.write_bytes(b"not really an apk")
|
|
|
154
|
+ _freeze_time(monkeypatch, datetime(2026, 9, 14, 12, 34, 56, tzinfo=timezone.utc))
|
|
|
155
|
+
|
|
|
156
|
+ calls = []
|
|
|
157
|
+
|
|
|
158
|
+ def fake_upload(file_path, ssh_user, ssh_host, remote_dir):
|
|
|
159
|
+ calls.append((file_path, ssh_user, ssh_host, remote_dir))
|
|
|
160
|
+ return f"https://{ssh_host}/~{ssh_user}/yeet/{remote_dir}/{file_path.name}"
|
|
|
161
|
+
|
|
|
162
|
+ monkeypatch.setattr(upload, "_upload", fake_upload)
|
|
|
163
|
+
|
|
|
164
|
+ url = upload.run(tmp_path, str(apk), "android_x86_64", assume_yes=True)
|
|
|
165
|
+
|
|
|
166
|
+ assert url == (
|
|
|
167
|
+ f"https://{config['ssh_host']}/~{config['ssh_user']}/yeet/20260914-123456-android_x86_64/app.apk"
|
|
|
168
|
+ )
|
|
|
169
|
+ assert calls == [
|
|
|
170
|
+ (
|
|
|
171
|
+ apk.resolve(),
|
|
|
172
|
+ config["ssh_user"],
|
|
|
173
|
+ config["ssh_host"],
|
|
|
174
|
+ "20260914-123456-android_x86_64",
|
|
|
175
|
+ )
|
|
|
176
|
+ ]
|
|
|
177
|
+ assert url in capsys.readouterr().out
|
|
|
178
|
+
|
|
|
179
|
+
|
|
|
180
|
+def test_run_reports_upload_failure(tmp_path, monkeypatch):
|
|
|
181
|
+ write_config(tmp_path)
|
|
|
182
|
+ apk = tmp_path / "app.apk"
|
|
|
183
|
+ apk.write_bytes(b"not really an apk")
|
|
|
184
|
+
|
|
|
185
|
+ def fake_upload(file_path, ssh_user, ssh_host, remote_dir):
|
|
|
186
|
+ raise RuntimeError("boom")
|
|
|
187
|
+
|
|
|
188
|
+ monkeypatch.setattr(upload, "_upload", fake_upload)
|
|
|
189
|
+
|
|
|
190
|
+ with pytest.raises(RuntimeError, match="boom"):
|
|
|
191
|
+ upload.run(tmp_path, str(apk), "android_x86_64", assume_yes=True)
|
|
|
192
|
+
|
|
|
193
|
+
|
|
|
194
|
+@pytest.mark.parametrize("answer", ["y", "yes", "Y"])
|
|
|
195
|
+def test_run_uploads_after_confirmation(tmp_path, monkeypatch, answer):
|
|
|
196
|
+ write_config(tmp_path)
|
|
|
197
|
+ apk = _write_apk(tmp_path)
|
|
|
198
|
+ _set_confirm_answers(monkeypatch, answer)
|
|
|
199
|
+ monkeypatch.setattr(
|
|
|
200
|
+ upload,
|
|
|
201
|
+ "_upload",
|
|
|
202
|
+ lambda file_path, ssh_user, ssh_host, remote_dir: "https://x/y",
|
|
|
203
|
+ )
|
|
|
204
|
+
|
|
|
205
|
+ assert upload.run(tmp_path, str(apk), "android_x86_64") == "https://x/y"
|
|
|
206
|
+
|
|
|
207
|
+
|
|
|
208
|
+@pytest.mark.parametrize("answers", [("",), ("n",), ("no",), ("maybe", "n")])
|
|
|
209
|
+def test_run_does_not_upload_when_declined(tmp_path, monkeypatch, answers):
|
|
|
210
|
+ write_config(tmp_path)
|
|
|
211
|
+ apk = _write_apk(tmp_path)
|
|
|
212
|
+ _set_confirm_answers(monkeypatch, *answers)
|
|
|
213
|
+ monkeypatch.setattr(
|
|
|
214
|
+ upload,
|
|
|
215
|
+ "_upload",
|
|
|
216
|
+ lambda *args: pytest.fail("_upload called after the user declined"),
|
|
|
217
|
+ )
|
|
|
218
|
+
|
|
|
219
|
+ with pytest.raises(RuntimeError, match="cancelled"):
|
|
|
220
|
+ upload.run(tmp_path, str(apk), "android_x86_64")
|
|
|
221
|
+
|
|
|
222
|
+
|
|
|
223
|
+def test_run_skips_confirmation_with_assume_yes(tmp_path, monkeypatch):
|
|
|
224
|
+ write_config(tmp_path)
|
|
|
225
|
+ apk = _write_apk(tmp_path)
|
|
|
226
|
+ monkeypatch.setattr(
|
|
|
227
|
+ "builtins.input", lambda prompt="": pytest.fail("unexpected prompt")
|
|
|
228
|
+ )
|
|
|
229
|
+ monkeypatch.setattr(
|
|
|
230
|
+ upload,
|
|
|
231
|
+ "_upload",
|
|
|
232
|
+ lambda file_path, ssh_user, ssh_host, remote_dir: "https://x/y",
|
|
|
233
|
+ )
|
|
|
234
|
+
|
|
|
235
|
+ assert (
|
|
|
236
|
+ upload.run(tmp_path, str(apk), "android_x86_64", assume_yes=True)
|
|
|
237
|
+ == "https://x/y"
|
|
|
238
|
+ )
|
|
|
239
|
+
|
|
|
240
|
+
|
|
|
241
|
+if __name__ == "__main__":
|
|
|
242
|
+ mozunit.main() |
tools/yeet/upload.py
|
|
1
|
+# This Source Code Form is subject to the terms of the Mozilla Public
|
|
|
2
|
+# License, v. 2.0. If a copy of the MPL was not distributed with this
|
|
|
3
|
+# file, You can obtain one at http://mozilla.org/MPL/2.0/.
|
|
|
4
|
+
|
|
|
5
|
+import re
|
|
|
6
|
+import subprocess
|
|
|
7
|
+from datetime import datetime, timezone
|
|
|
8
|
+from pathlib import Path
|
|
|
9
|
+
|
|
|
10
|
+from yeet.common import TERM, YEET_CONFIG_FILENAME, confirm, read_config
|
|
|
11
|
+
|
|
|
12
|
+
|
|
|
13
|
+def resolve_file(file_arg):
|
|
|
14
|
+ path = Path(file_arg).resolve()
|
|
|
15
|
+ if not path.is_file():
|
|
|
16
|
+ raise FileNotFoundError(f"No such file: {path}")
|
|
|
17
|
+ return path
|
|
|
18
|
+
|
|
|
19
|
+
|
|
|
20
|
+def _remote_dir_name(platform):
|
|
|
21
|
+ """Name the upload's directory after the current time and `platform`,
|
|
|
22
|
+ so repeated uploads never collide or overwrite each other.
|
|
|
23
|
+ """
|
|
|
24
|
+ timestamp = datetime.now(timezone.utc).strftime("%Y%m%d-%H%M%S")
|
|
|
25
|
+ return f"{timestamp}-{platform}"
|
|
|
26
|
+
|
|
|
27
|
+
|
|
|
28
|
+def _upload(file_path, ssh_user, ssh_host, remote_dir):
|
|
|
29
|
+ """scp `file_path` into ~/public_html/yeet/`remote_dir` on
|
|
|
30
|
+ ssh_user@ssh_host, keeping its name (with characters unsafe in a URL or
|
|
|
31
|
+ shell replaced by `_`) and creating that directory first if needed. Both
|
|
|
32
|
+ commands get /dev/null as stdin so they can't swallow input buffered for
|
|
|
33
|
+ us; their touch/PIN/passphrase prompts go through /dev/tty and still reach
|
|
|
34
|
+ the user.
|
|
|
35
|
+
|
|
|
36
|
+ Returns the file's final https:// URL. Raises RuntimeError on failure.
|
|
|
37
|
+ """
|
|
|
38
|
+ ssh_target = f"{ssh_user}@{ssh_host}"
|
|
|
39
|
+ remote_name = re.sub(r"[^A-Za-z0-9._-]", "_", file_path.name)
|
|
|
40
|
+ dest_dir = f"~/public_html/yeet/{remote_dir}"
|
|
|
41
|
+ dest_path = f"{dest_dir}/{remote_name}"
|
|
|
42
|
+ html_path = f"~{ssh_user}/yeet/{remote_dir}/{remote_name}"
|
|
|
43
|
+
|
|
|
44
|
+ mkdir = subprocess.run(
|
|
|
45
|
+ ["ssh", ssh_target, "mkdir", "-p", dest_dir],
|
|
|
46
|
+ stdin=subprocess.DEVNULL,
|
|
|
47
|
+ check=False,
|
|
|
48
|
+ )
|
|
|
49
|
+ if mkdir.returncode != 0:
|
|
|
50
|
+ raise RuntimeError(f"Could not create {dest_dir} on {ssh_target}.")
|
|
|
51
|
+
|
|
|
52
|
+ scp = subprocess.run(
|
|
|
53
|
+ ["scp", str(file_path), f"{ssh_target}:{dest_path}"],
|
|
|
54
|
+ stdin=subprocess.DEVNULL,
|
|
|
55
|
+ check=False,
|
|
|
56
|
+ )
|
|
|
57
|
+ if scp.returncode != 0:
|
|
|
58
|
+ raise RuntimeError(f"Upload to {ssh_target} failed.")
|
|
|
59
|
+
|
|
|
60
|
+ return f"https://{ssh_host}/{html_path}"
|
|
|
61
|
+
|
|
|
62
|
+
|
|
|
63
|
+def run(topsrcdir, file, platform, assume_yes=False):
|
|
|
64
|
+ """Upload `file` for `platform` to the configured server, asking for
|
|
|
65
|
+ confirmation first unless `assume_yes`. Returns the file's final https://
|
|
|
66
|
+ URL. Raises FileNotFoundError if `file` or the config is missing, and
|
|
|
67
|
+ RuntimeError if the upload fails or is cancelled.
|
|
|
68
|
+ """
|
|
|
69
|
+ config = read_config(topsrcdir)
|
|
|
70
|
+ if config is None:
|
|
|
71
|
+ raise RuntimeError(
|
|
|
72
|
+ f"No {YEET_CONFIG_FILENAME} found. Run `mach yeet auth` first."
|
|
|
73
|
+ )
|
|
|
74
|
+
|
|
|
75
|
+ file_path = resolve_file(file)
|
|
|
76
|
+
|
|
|
77
|
+ remote_dir = _remote_dir_name(platform)
|
|
|
78
|
+
|
|
|
79
|
+ if not assume_yes:
|
|
|
80
|
+ print()
|
|
|
81
|
+ print(
|
|
|
82
|
+ TERM.yellow(
|
|
|
83
|
+ f"{file_path} will be uploaded to {config['ssh_host']} and "
|
|
|
84
|
+ "be publicly accessible."
|
|
|
85
|
+ )
|
|
|
86
|
+ )
|
|
|
87
|
+ if not confirm("Upload it?"):
|
|
|
88
|
+ raise RuntimeError("Upload cancelled.")
|
|
|
89
|
+
|
|
|
90
|
+ print()
|
|
|
91
|
+ print(f"Uploading {file_path.name} to {remote_dir}/...")
|
|
|
92
|
+ url = _upload(file_path, config["ssh_user"], config["ssh_host"], remote_dir)
|
|
|
93
|
+
|
|
|
94
|
+ print()
|
|
|
95
|
+ print(TERM.green(f"Uploaded: {url}"))
|
|
|
96
|
+ return url |
|