[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [tor-bugs] #30020 [Internal Services/Tor Sysadmin Team]: switch from our custom YAML implementation to Hiera
#30020: switch from our custom YAML implementation to Hiera
-------------------------------------------------+-------------------------
Reporter: anarcat | Owner: anarcat
Type: project | Status:
| accepted
Priority: Medium | Milestone:
Component: Internal Services/Tor Sysadmin Team | Version:
Severity: Normal | Resolution:
Keywords: | Actual Points:
Parent ID: #29387 | Points:
Reviewer: | Sponsor:
-------------------------------------------------+-------------------------
Description changed by anarcat:
Old description:
> We currently use a custom-made YAML database for assigning roles to
> servers and other metadata. I started using Hiera for some hosts and it
> seems to be working well.
>
> Hiera is officially supported in Puppet and shipped by default in Puppet
> 5 and later. It's the standard way of specifying metadata and class
> parameters for hosts. I suspect it covers most of our needs in terms of
> metadata and should cover most if not all of what we're currently doing
> with the YAML stuff in Puppet.
>
> We should therefore switch to using Hiera instead of our homegrown
> solution.
>
> This involves converting:
>
> * `if has_role('foo') { include foo }` into `classes: [ 'foo' ]` in
> hiera
> * hardcoded macros in the ferm module's `me.conf.erb` into exported
> resources
> * templates looping over allnodeinfo into exported resources
> * the `$roles` array into Hiera
> * the `$localinfo` into Hiera (assuming all the data is there)
> * the `$nodeinfo` and `$allnodeinfo` arrays into Hiera (assuming we can
> switch from LDAP for host inventory)
> * basically any other stuff of the kind, including those files:
> {{{
> ./modules/torproject_org/misc/hoster.yaml
> ./modules/torproject_org/misc/local.yaml
> ./modules/ipsec/misc/config.yaml
> ./modules/roles/misc/static-components.yaml
> ./modules/roles/files/spec/spec-redirects.yaml
> }}}
>
> Ideally, all YAML data should end up in the hiera/ directory somehow.
> This is the first step in making our repository public (#29387) but also
> using Hiera as a more elaborate inventory system (#30273).
>
> The idea of switching from LDAP to Hiera for host inventory will
> definitely need to be evaluated more thoroughly before going ahead with
> that part of the conversion, but YAML stuff in Puppet should definitely
> be converted.
>
> The general goal of this is both to allow for a better inventory system
> but also make it easier for people to get onboarded with Puppet. By using
> community standards like Hiera, we make it easier for new people to get
> familiar with the puppet infrastructures and do things meaningfully.
New description:
We currently use a custom-made YAML database for assigning roles to
servers and other metadata. I started using Hiera for some hosts and it
seems to be working well.
Hiera is officially supported in Puppet and shipped by default in Puppet 5
and later. It's the standard way of specifying metadata and class
parameters for hosts. I suspect it covers most of our needs in terms of
metadata and should cover most if not all of what we're currently doing
with the YAML stuff in Puppet.
We should therefore switch to using Hiera instead of our homegrown
solution.
This involves converting:
* `if has_role('foo') { include foo }` into `classes: [ 'foo' ]` in hiera
(DONE!)
* hardcoded macros in the ferm module's `me.conf.erb` into exported
resources (DONE, except for HOST_TPO)
* templates looping over allnodeinfo into exported resources
* the `$roles` array into Hiera (DONE!)
* the `$localinfo` into Hiera (assuming all the data is there) (DONE!)
* the `$nodeinfo` and `$allnodeinfo` arrays into Hiera (assuming we can
switch from LDAP for host inventory)
* basically any other stuff of the kind, including those files:
{{{
./modules/torproject_org/misc/hoster.yaml
./modules/torproject_org/misc/local.yaml <- DONE!
./modules/ipsec/misc/config.yaml
./modules/roles/misc/static-components.yaml
./modules/roles/files/spec/spec-redirects.yaml
}}}
Ideally, all YAML data should end up in the hiera/ directory somehow. This
is the first step in making our repository public (#29387) but also using
Hiera as a more elaborate inventory system (#30273).
The idea of switching from LDAP to Hiera for host inventory will
definitely need to be evaluated more thoroughly before going ahead with
that part of the conversion, but YAML stuff in Puppet should definitely be
converted.
The general goal of this is both to allow for a better inventory system
but also make it easier for people to get onboarded with Puppet. By using
community standards like Hiera, we make it easier for new people to get
familiar with the puppet infrastructures and do things meaningfully.
Update: `get_roles()`, `has_role()`, `yamlinfo()` and `local.yaml` are
*all* gone! The main chunks remaining are now `nodeinfo()`,
`allnodeinfo()`, `$nodeinfo` and `hoster.yaml`. A plan has been laid out
for that replacement below. Obviously, the ipsec, static components and
redirects YAML files could use a transition into Hiera as well, but those
are lower priority.
--
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/30020#comment:11>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs