[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
Re: [tor-bugs] #30020 [Internal Services/Tor Sysadmin Team]: switch from our custom YAML implementation to Hiera
#30020: switch from our custom YAML implementation to Hiera
-------------------------------------------------+-------------------------
Reporter: anarcat | Owner: anarcat
Type: project | Status:
| accepted
Priority: Medium | Milestone:
Component: Internal Services/Tor Sysadmin Team | Version:
Severity: Normal | Resolution:
Keywords: | Actual Points:
Parent ID: #29387 | Points:
Reviewer: | Sponsor:
-------------------------------------------------+-------------------------
Comment (by anarcat):
grand milestone today: `local.yaml` was removed from the repository, along
with `get_role` and `yamlinfo`, which are all now useless.
WHOOHOO!
== Next step: hoster.yaml
the next chunk we need to convert would be, i think,
`./modules/torproject_org/misc/hoster.yaml`, which specifies those things:
* `netrange`: used to create the `TPO_NET` macro in ferm (unused?) and
determine in which `hoster` a given host is (through `whohosts.rb`, which
does IP range calculations from the host's IP as seen from LDAP)
* `mirror-debian`: used in `torproject_org` class to define the APT mirror
for this host
* `mirror-debian-security`: unused?
* `nameservers`: used to configured upstream forwarders in unbound on each
host
* `nameservers_break_dnssec` : used to disable unbound forwarding in case
of broken upstream DNS, unused
* `allow_dns_query`: used to tell unbound to allow other network ranges
(ie. generally on this site) to use *this* node as recursive DNS server
(if `misc.resolver-recursive` is true, which is the case when the LDAP ip
of the host is listed in the hoster's `nameservers` list)
Debian.org has hooked hosters.yaml into hiera, and the way they did it is
to have one .yaml file per hoster, for example:
https://salsa.debian.org/dsa-team/mirror/dsa-
puppet/blob/master/hieradata/bytemark.yaml
Unfortunately, the `hoster.yaml` is still present on d.o:
https://salsa.debian.org/dsa-team/mirror/dsa-
puppet/blob/master/modules/puppetmaster/lib/puppet/parser/functions/whohosts.rb
there we have the same code as on tpo:
{{{
yamlfile =
Puppet::Parser::Files.find_file('debian_org/misc/hoster.yaml',
compiler.environment)
}}}
Here the file, which contains more than just ip ranges:
https://salsa.debian.org/dsa-team/mirror/dsa-
puppet/blob/master/modules/debian_org/files/misc/hoster.yaml
So their transition isn't complete, but it matches some of the ideas I had
(namely to have one YAML file per hoster).
So what I would suggest we do to get rid of the hoster.yaml file is this:
1. convert all the aforementioned variables used in `hoster.yaml` into
class variables, defaulting to the values loaded from `hoster.yaml` (ie.
`$nodeinfo`)
2. test the variables by overriding them in (e.g.)
`hiera/nodes/foo.example.com.yaml`
3. break up the `hoster.yaml` file into multiple smaller files in
`hiera/hoster/%{hoster}.yaml`
4. add that path to `hiera.yaml`
5. test that host can load its variables from the `hoster` search path by
hardcoding a value by hand in facter
6. create a new YAML variable that gives us a IP range -> hoster mapping
7. create a function that looks through those to guess the hoster for a
given IP address
8. use that function to create a fact (through a template, but with a
variable defined in the base class) that defines the `$hoster` variable
that hiera will use to load the right YAML
9. remove `hoster.yaml`
That's a first step. At that stage, `hoster.yaml` is gone, but `$nodeinfo`
remains and still might contain host-specific configuration. Those should
be extract *out* of the `$nodeinfo` construct and into manifest business
logic. And *then* the `nodeinfo.rb` code can be ripped out.
There might be a better way to define a hoster per node than guess it with
its IP address and drop it as a fact, but I can't think of anything right
now.
--
Ticket URL: <https://trac.torproject.org/projects/tor/ticket/30020#comment:10>
Tor Bug Tracker & Wiki <https://trac.torproject.org/>
The Tor Project: anonymity online
_______________________________________________
tor-bugs mailing list
tor-bugs@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-bugs