[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [tor-bugs] #23756 [Core Tor/Tor]: tor's .gitlab-ci.yml is doing mirroring? why?
#23756: tor's .gitlab-ci.yml is doing mirroring? why?
--------------------------+------------------------------------
Reporter: isis | Owner: (none)
Type: defect | Status: new
Priority: Medium | Milestone: Tor: 0.3.1.x-final
Component: Core Tor/Tor | Version: Tor: 0.3.1.3-alpha
Severity: Normal | Resolution:
Keywords: tor-ci | Actual Points:
Parent ID: | Points:
Reviewer: | Sponsor:
--------------------------+------------------------------------
Description changed by isis:
Old description:
> Currently in master we have the following stanza in our .gitlab-ci.yml
> (from #22891):
>
> {{{
> update:
> script:
> - "apt-get install -y --fix-missing git openssh-client"
>
> # Run ssh-agent (inside the build environment)
> - eval $(ssh-agent -s)
>
> # Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent
> store
> - ssh-add <("$DEPLOY_KEY")
>
> # For Docker builds disable host key checking. Be aware that by
> adding that
> # you are suspectible to man-in-the-middle attacks.
> # WARNING: Use this only with the Docker executor, if you use it with
> shell
> # you will overwrite your user's SSH config.
> - mkdir -p ~/.ssh
> - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking
> no\n\n" > ~/.ssh/config'
> # In order to properly check the server's host key, assuming you
> created the
> # SSH_SERVER_HOSTKEYS variable previously, uncomment the following
> two lines
> # instead.
> - mkdir -p ~/.ssh
> - '[[ -f /.dockerenv ]] && echo "$SSH_SERVER_HOSTKEYS" >
> ~/.ssh/known_hosts'
> - echo "merging from torgit"
> - git config --global user.email "labadmin@xxxxxxxxxxx"
> - git config --global user.name "gitadmin"
> - "mkdir tor"
> - "cd tor"
> - git clone --bare https://git.torproject.org/tor.git
> - git push --mirror git@xxxxxxxxxxx:network/tor.git
> }}}
>
> Why are we doing this? Can we put a cronjob on the oniongit.eu server
> instead? It's pretty weird and frankly unexpected that my personal fork
> of tor at https://gitlab.com/isis/tor is cloning the official tor repo
> and then trying to mirror it to oniongit.eu. It also has a bunch of other
> problems:
>
> * The `ssh-add` line [https://gitlab.com/isis/tor/-/jobs/34990901 is
> broken, causing CI to fail because it sits there forever waiting for a
> passphrase].
>
> I was originally going to patch the `ssh-add` line to instead be `[[ -n
> "${DEPLOY_KEY}" -a -r "$DEPLOY_KEY" ]] && ssh-add "$DEPLOY_KEY" <<<""`
> but if I fix that, then all the rest of this script would run, so I'm
> rather glad it's failing on a more innocuous command.
>
> * Even if the `ssh-add` line weren't broken, this whole thing fails
> unless it's being run from a fork on oniongit.eu.
> * Why is it disabling SSH hostkey checking?!
> * Why is it making the `~/.ssh` directory twice?
> * Why is it assuming that environment variables are set? e.g. `$FOO`
> versus `${FOO}` or better `test -n ${FOO}`
> * Why is it unconditionally setting (global!) git config options? (I
> assume to disable the warning that git spits out when you don't have
> `$GIT_{AUTHOR,COMMITTER}_{NAME,EMAIL}` set, but why would a CI config set
> them globally instead of just setting the correct environment variables?)
> * Why are the mirror URLs hardcoded?
> * Why is the git username and email hardcoded?
> * Why is any of this even running when I push to
> https://gitlab.com/isis/tor?
> * Why is it unconditionally starting an ssh-agent?
>
> I'm sorry if this is all necessary and I'm just not understanding the
> setup, but it's all just extremely unexpected behaviour from what is
> supposed to be a CI config file. Further, it's not even doing the same
> testing as our .travis.yml, but I'll make another ticket for that issue.
New description:
Currently in master we have the following stanza in our .gitlab-ci.yml
(from #22891):
{{{
update:
script:
- "apt-get install -y --fix-missing git openssh-client"
# Run ssh-agent (inside the build environment)
- eval $(ssh-agent -s)
# Add the SSH key stored in SSH_PRIVATE_KEY variable to the agent
store
- ssh-add <("$DEPLOY_KEY")
# For Docker builds disable host key checking. Be aware that by adding
that
# you are suspectible to man-in-the-middle attacks.
# WARNING: Use this only with the Docker executor, if you use it with
shell
# you will overwrite your user's SSH config.
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking
no\n\n" > ~/.ssh/config'
# In order to properly check the server's host key, assuming you
created the
# SSH_SERVER_HOSTKEYS variable previously, uncomment the following two
lines
# instead.
- mkdir -p ~/.ssh
- '[[ -f /.dockerenv ]] && echo "$SSH_SERVER_HOSTKEYS" >
~/.ssh/known_hosts'
- echo "merging from torgit"
- git config --global user.email "labadmin@xxxxxxxxxxx"
- git config --global user.name "gitadmin"
- "mkdir tor"
- "cd tor"
- git clone --bare https://git.torproject.org/tor.git
- git push --mirror git@xxxxxxxxxxx:network/tor.git
}}}
Why are we doing this? Can we put a cronjob on the oniongit.eu server
instead? It's pretty weird and frankly unexpected that my personal fork of
tor at https://gitlab.com/isis/tor is cloning the official tor repo and
then trying to mirror it to oniongit.eu. It also has a bunch of other
problems:
* The `ssh-add` line [https://gitlab.com/isis/tor/-/jobs/34990901 is
broken, causing CI to fail because it sits there forever waiting for a
passphrase].
I was originally going to patch the `ssh-add` line to instead be `[[ -n
"${DEPLOY_KEY}" -a -r "$DEPLOY_KEY" ]] && ssh-add "$DEPLOY_KEY" <<<""` but
if I fix that, then all the rest of this script would run, so I'm rather
glad it's failing on a more innocuous command.
* Even if the `ssh-add` line weren't broken, this whole thing fails unless
it's being run from a fork on oniongit.eu.
* Why is it disabling SSH hostkey checking?!
* Why is it making the `~/.ssh` directory twice?
* Why is it assuming that environment variables are set? e.g. `$FOO`
versus `${FOO}` or better `test -n ${FOO}`
* Why is it unconditionally setting (global!) git config options? (I
assume to disable the warning that git spits out when you don't have
`$GIT_{AUTHOR,COMMITTER}_{NAME,EMAIL}` set, but why would a CI config set
them globally instead of just setting the correct environment variables?)
* Why are the mirror URLs hardcoded?
* Why is the git username and email hardcoded?
* Why is any of this even running when I push to
https://gitlab.com/isis/tor?
* Why is any of this even running when I push anywhere?
* Why is it unconditionally starting an ssh-agent?
* Why is using the existence of a
([https://superuser.com/questions/1021834/what-are-dockerenv-and-
dockerinit#1021925 deprecated!]) `/.dockerenv` file to determine if we're
in a docker container?
* Why is it assuming we're in the ''correct'' docker container, when lots
of things, especially lots of CI systems, use docker?
I'm sorry if this is all necessary and I'm just not understanding the
setup, but it's all just extremely unexpected behaviour from what is
supposed to be a CI config file. Further, it's not even doing the same
testing as our .travis.yml, but I'll make another ticket for that issue.
--
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/23756#comment:1>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs