[tor-talk] Tor 0.4.3.1-alpha is released!

[Nick Mathewson <nickm@xxxxxxxxxxxxxx>]

Hi, all!

There's a new alpha Tor release! Because it's an alpha, you should
only run it if you're ready to find more bugs than usual, and report
them on trac.torproject.org.

The source code is available from the usual place on
https://www.torproject.org/download/tor/ ; if you build Tor from
source, why not give it a try? And if you don't build Tor from source,
packages should be ready over the coming days, with a Tor Browser
alpha release likely by mid-February.

Here's what's new:

Changes in version 0.4.3.1-alpha - 2020-01-22
  This is the first alpha release in the 0.4.3.x series. It includes
  improved support for application integration of onion services, support
  for building in a client-only mode, and newly improved internal
  documentation (online at https://src-ref.docs.torproject.org/tor/). It
  also has numerous other small bugfixes and features, as well as
  improvements to our code's internal organization that should help us
  write better code in the future.

  o New system requirements:
    - When building Tor, you now need to have Python 3 in order to run
      the integration tests. (Python 2 is officially unsupported
      upstream, as of 1 Jan 2020.) Closes ticket 32608.

  o Major features (build system):
    - The relay code can now be disabled using the --disable-module-relay
      configure option. When this option is set, we also disable the
      dirauth module. Closes ticket 32123.
    - When Tor is compiled --disable-module-relay, we also omit the code
      used to act as a directory cache. Closes ticket 32487.

  o Major features (directory authority, ed25519):
    - Add support for banning a relay's ed25519 keys in the approved-
      routers file. This will help us migrate away from RSA keys in the
      future. Previously, only RSA keys could be banned in approved-
      routers. Resolves ticket 22029. Patch by Neel Chauhan.

  o Major features (onion service, controller):
    - New control port commands to manage client-side onion service
      authorization credentials. The ONION_CLIENT_AUTH_ADD command adds
      a credential, ONION_CLIENT_AUTH_REMOVE deletes a credential, and
      ONION_CLIENT_AUTH_VIEW lists the credentials. Closes ticket 30381.

  o Major features (onion service, SOCKS5):
    - Introduce a new SocksPort flag, ExtendedErrors, to support more
      detailed error codes in information for applications that support
      them. Closes ticket 30382; implements proposal 304.

  o Major features (proxy):
    - In addition to its current supported proxy types (HTTP CONNECT,
      SOCKS4, and SOCKS5), Tor can now make its OR connections through a
      HAProxy server. A new torrc option was added to specify the
      address/port of the server: TCPProxy <protocol> <host>:<port>.
      Currently the only supported protocol for the option is haproxy.
      Closes ticket 31518. Patch done by Suphanat Chunhapanya (haxxpop).

  o Major bugfixes (linux seccomp sandbox):
    - Correct how we use libseccomp. Particularly, stop assuming that
      rules are applied in a particular order or that more rules are
      processed after the first match. Neither is the case! In
      libseccomp <2.4.0 this lead to some rules having no effect.
      libseccomp 2.4.0 changed how rules are generated, leading to a
      different ordering, which in turn led to a fatal crash during
      startup. Fixes bug 29819; bugfix on 0.2.5.1-alpha. Patch by
      Peter Gerber.
    - Fix crash when reloading logging configuration while the
      experimental sandbox is enabled. Fixes bug 32841; bugfix on
      0.4.1.7. Patch by Peter Gerber.

  o Major bugfixes (networking):
    - Correctly handle IPv6 addresses in SOCKS5 RESOLVE_PTR requests,
      and accept strings as well as binary addresses. Fixes bug 32315;
      bugfix on 0.3.5.1-alpha.

  o Major bugfixes (onion service):
    - Report HS circuit failure back into the HS subsystem so we take
      appropriate action with regards to the client introduction point
      failure cache. This improves reachability of onion services, since
      now clients notice failing introduction circuits properly. Fixes
      bug 32020; bugfix on 0.3.2.1-alpha.

  o Minor feature (configure, build system):
    - Output a list of enabled/disabled features at the end of the
      configure process in a pleasing way. Closes ticket 31373.

  o Minor feature (heartbeat, onion service):
    - Add the DoS INTRODUCE2 defenses counter to the heartbeat DoS
      message. Closes ticket 31371.

  o Minor features (configuration validation):
    - Configuration validation can now be done by per-module callbacks,
      rather than a global validation function. This will let us reduce
      the size of config.c and some of its more cumbersome functions.
      Closes ticket 31241.

  o Minor features (configuration):
    - If a configured hardware crypto accelerator in AccelName is
      prefixed with "!", Tor now exits when it cannot be found. Closes
      ticket 32406.
    - We now use flag-driven logic to warn about obsolete configuration
      fields, so that we can include their names. In 0.4.2, we used a
      special type, which prevented us from generating good warnings.
      Implements ticket 32404.

  o Minor features (controller):
    - Add stream isolation data to STREAM event. Closes ticket 19859.
    - Implement a new GETINFO command to fetch microdescriptor
      consensus. Closes ticket 31684.

  o Minor features (debugging, directory system):
    - Don't crash when we find a non-guard with a guard-fraction value
      set. Instead, log a bug warning, in an attempt to figure out how
      this happened. Diagnostic for ticket 32868.

  o Minor features (defense in depth):
    - Add additional checks around tor_vasprintf() usage, in case the
      function returns an error. Patch by Tobias Stoeckmann. Fixes
      ticket 31147.

  o Minor features (developer tooling):
    - Remove the 0.2.9.x series branches from git scripts (git-merge-
      forward.sh, git-pull-all.sh, git-push-all.sh, git-setup-dirs.sh).
      Closes ticket 32772.

  o Minor features (developer tools):
    - Add a check_cocci_parse.sh script that checks that new code is
      parseable by Coccinelle. Add an exceptions file for unparseable
      files, and run the script from travis CI. Closes ticket 31919.
    - Call the check_cocci_parse.sh script from a 'check-cocci' Makefile
      target. Closes ticket 31919.
    - Add a rename_c_identifiers.py tool to rename a bunch of C
      identifiers at once, and generate a well-formed commit message
      describing the change. This should help with refactoring. Closes
      ticket 32237.
    - Add some scripts in "scripts/coccinelle" to invoke the Coccinelle
      semantic patching tool with the correct flags. These flags are
      fairly easy to forget, and these scripts should help us use
      Coccinelle more effectively in the future. Closes ticket 31705.

  o Minor features (Doxygen):
    - Update Doxygen configuration file to a more recent template (from
      1.8.15). Closes ticket 32110.
    - "make doxygen" now works with out-of-tree builds. Closes
      ticket 32113.
    - Make sure that doxygen outputs documentation for all of our C
      files. Previously, some were missing @file declarations, causing
      them to be ignored. Closes ticket 32307.
    - Our "make doxygen" target now respects --enable-fatal-warnings by
      default, and does not warn about items that are missing
      documentation. To warn about missing documentation, run configure
      with the "--enable-missing-doc-warnings" flag: doing so suspends
      fatal warnings for doxygen. Closes ticket 32385.

  o Minor features (git scripts):
    - Add TOR_EXTRA_CLONE_ARGS to git-setup-dirs.sh for git clone
      customisation. Closes ticket 32347.
    - Add git-setup-dirs.sh, which sets up an upstream git repository
      and worktrees for tor maintainers. Closes ticket 29603.
    - Add TOR_EXTRA_REMOTE_* to git-setup-dirs.sh for a custom extra
      remote. Closes ticket 32347.
    - Call the check_cocci_parse.sh script from the git commit and push
      hooks. Closes ticket 31919.
    - Make git-push-all.sh skip unchanged branches when pushing to
      upstream. The script already skipped unchanged test branches.
      Closes ticket 32216.
    - Make git-setup-dirs.sh create a master symlink in the worktree
      directory. Closes ticket 32347.
    - Skip unmodified source files when doing some existing git hook
      checks. Related to ticket 31919.

  o Minor features (IPv6, client):
    - Make Tor clients tell dual-stack exits that they prefer IPv6
      connections. This change is equivalent to setting the PreferIPv6
      flag on SOCKSPorts (and most other listener ports). Tor Browser
      has been setting this flag for some time, and we want to remove a
      client distinguisher at exits. Closes ticket 32637.

  o Minor features (portability, android):
    - When building for Android, disable some tests that depend on $HOME
      and/or pwdb, which Android doesn't have. Closes ticket 32825.
      Patch from Hans-Christoph Steiner.

  o Minor features (relay modularity):
    - Split the relay and server pluggable transport config code into
      separate files in the relay module. Disable this code when the
      relay module is disabled. Closes part of ticket 32213.
    - When the relay module is disabled, reject attempts to set the
      ORPort, DirPort, DirCache, BridgeRelay, ExtORPort, or
      ServerTransport* options, rather than ignoring the values of these
      options. Closes part of ticket 32213.

  o Minor features (relay):
    - When the relay module is disabled, change the default config so
      that DirCache is 0, and ClientOnly is 1. Closes ticket 32410.

  o Minor features (release tools):
    - Port our ChangeLog formatting and sorting tools to Python 3.
      Closes ticket 32704.

  o Minor features (testing):
    - Detect some common failure cases for test_parseconf.sh in
      src/test/conf_failures. Closes ticket 32451.
    - Allow test_parseconf.sh to test expected log outputs for successful
      configs, as well as failed configs. Closes ticket 32451.
    - The test_parseconf.sh script now supports result variants for any
      combination of the optional libraries lzma, nss, and zstd. Closes
      ticket 32397.

  o Minor features (tests, Android):
    - When running the unit tests on Android, create temporary files in
      a subdirectory of /data/local/tmp. Closes ticket 32172. Based on a
      patch from Hans-Christoph Steiner.

  o Minor bugfixes (bridges):
    - Lowercase the configured value of BridgeDistribution before adding
      it to the descriptor. Fixes bug 32753; bugfix on 0.3.2.3-alpha.

  o Minor bugfixes (build system):
    - Fix "make autostyle" for out-of-tree builds. Fixes bug 32370;
      bugfix on 0.4.1.2-alpha.

  o Minor bugfixes (configuration handling):
    - Make control_event_conf_changed() take in a config_line_t instead
      of a smartlist of alternating key/value entries. Fixes bug 31531;
      bugfix on 0.2.3.3-alpha. Patch by Neel Chauhan.

  o Minor bugfixes (configuration):
    - Check for multiplication overflow when parsing memory units inside
      configuration. Fixes bug 30920; bugfix on 0.0.9rc1.
    - When dumping the configuration, stop adding a trailing space after
      the option name when there is no option value. This issue only
      affects options that accept an empty value or list. (Most options
      reject empty values, or delete the entire line from the dumped
      options.) Fixes bug 32352; bugfix on 0.0.9pre6.
    - Avoid changing the user's value of HardwareAccel as stored by
      SAVECONF, when AccelName is set but HardwareAccel is not. Fixes
      bug 32382; bugfix on 0.2.2.1-alpha.
    - When creating a KeyDirectory with the same location as the
      DataDirectory (not recommended), respect the DataDirectory's
      group-readable setting if one has not been set for the
      KeyDirectory. Fixes bug 27992; bugfix on 0.3.3.1-alpha.

  o Minor bugfixes (controller):
    - In routerstatus_has_changed(), check all the fields that are
      output over the control port. Fixes bug 20218; bugfix
      on 0.1.1.11-alpha

  o Minor bugfixes (correctness checks):
    - Use GCC/Clang's printf-checking feature to make sure that
      tor_assertf() arguments are correctly typed. Fixes bug 32765;
      bugfix on 0.4.1.1-alpha.

  o Minor bugfixes (developer tools):
    - Allow paths starting with ./ in scripts/add_c_file.py. Fixes bug
      31336; bugfix on 0.4.1.2-alpha.

  o Minor bugfixes (dirauth module):
    - Split the dirauth config code into a separate file in the dirauth
      module. Disable this code when the dirauth module is disabled.
      Closes ticket 32213.
    - When the dirauth module is disabled, reject attempts to set the
      AuthoritativeDir option, rather than ignoring the value of the
      option. Fixes bug 32213; bugfix on 0.3.4.1-alpha.

  o Minor bugfixes (embedded Tor):
    - When starting Tor any time after the first time in a process,
      register the thread in which it is running as the main thread.
      Previously, we only did this on Windows, which could lead to bugs
      like 23081 on non-Windows platforms. Fixes bug 32884; bugfix
      on 0.3.3.1-alpha.

  o Minor bugfixes (git scripts):
    - Avoid sleeping before the last push in git-push-all.sh. Closes
      ticket 32216.
    - Forward all unrecognised arguments in git-push-all.sh to git push.
      Closes ticket 32216.

  o Minor bugfixes (hidden service v3):
    - Do not rely on a "circuit established" flag for intro circuits but
      instead always query the HS circuit map. This is to avoid sync
      issue with that flag and the map. Fixes bug 32094; bugfix
      on 0.3.2.1-alpha.

  o Minor bugfixes (logging, crash):
    - Avoid a possible crash when trying to log a (fatal) assertion
      failure about mismatched magic numbers in configuration objects.
      Fixes bug 32771; bugfix on 0.4.2.1-alpha.

  o Minor bugfixes (onion service v2):
    - When sending the INTRO cell for a v2 Onion Service, look at the
      failure cache alongside timeout values to check if the intro point
      is marked as failed. Previously, we only looked at the relay
      timeout values. Fixes bug 25568; bugfix on 0.2.7.3-rc. Patch by
      Neel Chauhan.

  o Minor bugfixes (onion services v3, client):
    - Properly handle the client rendezvous circuit timeout. Previously
      Tor would sometimes timeout a rendezvous circuit awaiting the
      introduction ACK, and find itself unable to re-establish all
      circuits because the rendezvous circuit timed out too early. Fixes
      bug 32021; bugfix on 0.3.2.1-alpha.

  o Minor bugfixes (onion services):
    - In cancel_descriptor_fetches(), use
      connection_list_by_type_purpose() instead of
      connection_list_by_type_state(). Fixes bug 32639; bugfix on
      0.3.2.1-alpha. Patch by Neel Chauhan.

  o Minor bugfixes (scripts):
    - Fix update_versions.py for out-of-tree builds. Fixes bug 32371;
      bugfix on 0.4.0.1-alpha.

  o Minor bugfixes (test):
    - Use the same code to find the tor binary in all of our test
      scripts. This change makes sure we are always using the coverage
      binary when coverage is enabled. Fixes bug 32368; bugfix
      on 0.2.7.3-rc.

  o Minor bugfixes (testing):
    - Stop ignoring "tor --dump-config" errors in test_parseconf.sh.
      Fixes bug 32468; bugfix on 0.4.2.1-alpha.
    - When TOR_DISABLE_PRACTRACKER is set, do not apply it to the
      test_practracker.sh script. Doing so caused a test failure. Fixes
      bug 32705; bugfix on 0.4.2.1-alpha.
    - When TOR_DISABLE_PRACTRACKER is set, log a notice to stderr when
      skipping practracker checks. Fixes bug 32705; bugfix
      on 0.4.2.1-alpha.

  o Minor bugfixes (tests):
    - Our option-validation tests no longer depend on specially
      configured non-default, non-passing sets of options. Previously,
      the tests had been written to assume that options would _not_ be
      set to their defaults, which led to needless complexity and
      verbosity. Fixes bug 32175; bugfix on 0.2.8.1-alpha.

  o Minor bugfixes (windows service):
    - Initialize the publish/subscribe system when running as a windows
      service. Fixes bug 32778; bugfix on 0.4.1.1-alpha.

  o Deprecated features:
    - Deprecate the ClientAutoIPv6ORPort option. This option was not
      true "Happy Eyeballs", and often failed on connections that
      weren't reliably dual-stack. Closes ticket 32942. Patch by
      Neel Chauhan.

  o Documentation:
    - Provide a quickstart guide for a Circuit Padding Framework, and
      documentation for researchers to implement and study circuit
      padding machines. Closes ticket 28804.
    - Add documentation in 'HelpfulTools.md' to describe how to build a
      tag file. Closes ticket 32779.
    - Create a high-level description of the long-term software
      architecture goals. Closes ticket 32206.
    - Describe the --dump-config command in the manual page. Closes
      ticket 32467.
    - Unite coding advice from this_not_that.md in torguts repo into our
      coding standards document. Resolves ticket 31853.

  o Removed features:
    - Our Doxygen configuration no longer generates LaTeX output. The
      reference manual produced by doing this was over 4000 pages long,
      and generally unusable. Closes ticket 32099.
    - The option "TestingEstimatedDescriptorPropagationTime" is now
      marked as obsolete. It has had no effect since 0.3.0.7, when
      clients stopped rejecting consensuses "from the future". Closes
      ticket 32807.
    - We no longer support consensus methods before method 28; these
      methods were only used by authorities running versions of Tor that
      are now at end-of-life. In effect, this means that clients,
      relays, and authorities now assume that authorities will be
      running version 0.3.5.x or later. Closes ticket 32695.

  o Testing:
    - Add more test cases for tor's UTF-8 validation function. Also,
      check the arguments passed to the function for consistency. Closes
      ticket 32845.
    - Improve test coverage for relay and dirauth config code, focusing
      on option validation and normalization. Closes ticket 32213.
    - Improve the consistency of test_parseconf.sh output, and run all
      the tests, even if one fails. Closes ticket 32213.
    - Re-enable the Travis CI macOS Chutney build, but don't let it
      prevent the Travis job from finishing. (The Travis macOS jobs are
      slow, so we don't want to have it delay the whole CI process.)
      Closes ticket 32629.
    - Run the practracker unit tests in the pre-commit git hook. Closes
      ticket 32609.
    - Turn off Tor's Sandbox in Chutney jobs, and run those jobs on
      Ubuntu Bionic. Turning off the Sandbox is a work-around, until we
      fix the sandbox errors in 32722. Closes ticket 32240.

  o Code simplification and refactoring (channel):
    - Channel layer had a variable length cell handler that was not used
      and thus removed. Closes ticket 32892.

  o Code simplification and refactoring (configuration):
    - Immutability is now implemented as a flag on individual
      configuration options rather than as part of the option-transition
      checking code. Closes ticket 32344.
    - Instead of keeping a list of configuration options to check for
      relative paths, check all the options whose type is "FILENAME".
      Solves part of ticket 32339.
    - Our default log (which ordinarily sends NOTICE-level messages to
      standard output) is now handled in a more logical manner.
      Previously, we replaced the configured log options if they were
      empty. Now, we interpret an empty set of log options as meaning
      "use the default log". Closes ticket 31999.
    - Remove some unused arguments from the options_validate() function,
      to simplify our code and tests. Closes ticket 32187.
    - Simplify the options_validate() code so that it looks at the
      default options directly, rather than taking default options as an
      argument. This change lets us simplify its interface. Closes
      ticket 32185.
    - Use our new configuration architecture to move most authority-
      related options to the directory authority module. Closes
      ticket 32806.
    - When parsing the command line, handle options that determine our
      "quiet level" and our mode of operation (e.g., --dump-config and
      so on) all in one table. Closes ticket 32003.

  o Code simplification and refactoring (controller):
    - Create a new abstraction for formatting control protocol reply
      lines based on key-value pairs. Refactor some existing control
      protocol code to take advantage of this. Closes ticket 30984.
    - Create a helper function that can fetch network status or
      microdesc consensuses. Closes ticket 31684.

  o Code simplification and refactoring (dirauth modularization):
    - Remove the last remaining HAVE_MODULE_DIRAUTH inside a function.
      Closes ticket 32163.
    - Replace some confusing identifiers in process_descs.c. Closes
      ticket 29826.
    - Simplify some relay and dirauth config code. Closes ticket 32213.

  o Code simplification and refactoring (misc):
    - Make all the structs we declare follow the same naming convention
      of ending with "_t". Closes ticket 32415.
    - Move and rename some configuration-related code for clarity.
      Closes ticket 32304.
    - Our include.am files are now broken up by subdirectory.
      Previously, src/core/include.am covered all of the subdirectories
      in "core", "feature", and "app". Closes ticket 32137.
    - Remove underused NS*() macros from test code: they make our tests
      more confusing, especially for code-formatting tools. Closes
      ticket 32887.

  o Code simplification and refactoring (relay modularization):
    - Disable relay_periodic when the relay module is disabled. Closes
      ticket 32244.
    - Disable relay_sys when the relay module is disabled. Closes
      ticket 32245.

  o Code simplification and refactoring (tool support):
    - Add numerous missing dependencies to our include files, so that
      they can be included in different reasonable orders and still
      compile. Addresses part of ticket 32764.
    - Fix some parts of our code that were difficult for Coccinelle to
      parse. Related to ticket 31705.
    - Fix some small issues in our code that prevented automatic
      formatting tools from working. Addresses part of ticket 32764.

  o Documentation (manpage):
    - Alphabetize the Client Options section of the tor manpage. Closes
      ticket 32846.
    - Alphabetize the General Options section of the tor manpage. Closes
      ticket 32708.
    - In the tor(1) manpage, reword and improve formatting of the
      COMMAND-LINE OPTIONS and DESCRIPTION sections. Closes ticket
      32277. Based on work by Swati Thacker as part of Google Season
      of Docs.
    - In the tor(1) manpage, reword and improve formatting of the FILES,
      SEE ALSO, and BUGS sections. Closes ticket 32176. Based on work by
      Swati Thacker as part of Google Season of Docs.

  o Testing (circuit, EWMA):
    - Add unit tests for circuitmux and EWMA subsystems. Closes
      ticket 32196.

  o Testing (continuous integration):
    - Use zstd in our Travis Linux builds. Closes ticket 32242.
-- 
tor-talk mailing list - tor-talk@xxxxxxxxxxxxxxxxxxxx
To unsubscribe or change other settings go to
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk