[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

[or-cvs] r23660: {arm} Rewrite of the conf panel and several important bug fixes ch (in arm/trunk: . debian src src/interface src/interface/graphing src/util)



Author: atagar
Date: 2010-10-22 05:21:29 +0000 (Fri, 22 Oct 2010)
New Revision: 23660

Added:
   arm/trunk/src/settings.cfg
   arm/trunk/src/util/torrc.py
Removed:
   arm/trunk/armrc.sample
   arm/trunk/src/armrc.defaults
Modified:
   arm/trunk/ChangeLog
   arm/trunk/README
   arm/trunk/TODO
   arm/trunk/debian/MANIFEST
   arm/trunk/setup.py
   arm/trunk/src/interface/confPanel.py
   arm/trunk/src/interface/controller.py
   arm/trunk/src/interface/graphing/bandwidthStats.py
   arm/trunk/src/interface/graphing/graphPanel.py
   arm/trunk/src/interface/headerPanel.py
   arm/trunk/src/interface/logPanel.py
   arm/trunk/src/starter.py
   arm/trunk/src/util/conf.py
   arm/trunk/src/util/hostnames.py
   arm/trunk/src/util/log.py
   arm/trunk/src/util/panel.py
   arm/trunk/src/util/torTools.py
   arm/trunk/src/util/uiTools.py
Log:
Rewrite of the conf panel and several important bug fixes

change: full rewrite of the log panel, providing:
  - change: simplified and expanded on config validation and display (performance improvements, friendly units for torrc corrections, etc)
  - change: scrolling by displayed content rather than line numbers
  - fix: unnecessary whitespace was being stripped
  - fix: scrolling was buggy if comments were being stripped
  - fix: log panel wasn't respecting the prepopulate* log level config options
  - fix: torrc validation didn't recognize 'second' and 'byte' arguments

change: revised the arm config interface (simplified and expanded to include maps)
fix: not all worker threads were daemons, causing the process to persist in a broken state after exceptions and when quitting via ctrl+c
fix: custom armrcs resulted in the default parsing config options being unavailable
fix: rounding error in rendering the scrollbar, causing it to shrink a line when at the bottom
fix: off by one error when wrapping lines in the log panel



Modified: arm/trunk/ChangeLog
===================================================================
--- arm/trunk/ChangeLog	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/ChangeLog	2010-10-22 05:21:29 UTC (rev 23660)
@@ -1,6 +1,6 @@
 CHANGE LOG
 
-10/6/10 - version 1.3.7
+10/6/10 - version 1.3.7 (r23439)
 Numerous improvements, most notably being an expanded log panel, installer, and deb/rpm builds.
 
     * added: installation/removal scripts and man page (thanks to kaner)
@@ -50,6 +50,7 @@
     * fix: race condition between heartbeat detection and getting the first BW event
     * fix: refreshing after popups to make the interface seem more responsive
     * fix: crashing and minor display issues if orport was left unset
+    * fix (10/7/10, r23463): crashing from type issue in the graph panel (caught by tomb)
 
 6/7/10 - version 1.3.6 (r22617)
 Rewrite of the first third of the interface, providing vastly improved performance, maintainability, and a few very nice features. This improved the refresh rate (which is also related to system resource usage) from 30ms to 4ms (an 87% improvement).

Modified: arm/trunk/README
===================================================================
--- arm/trunk/README	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/README	2010-10-22 05:21:29 UTC (rev 23660)
@@ -102,10 +102,11 @@
   
   src/
     __init__.py
-    starter.py  - parses and validates commandline parameters
-    prereq.py   - checks python version and for required packages
-    version.py  - version and last modified information
-    uninstall   - removal script
+    starter.py   - parses and validates commandline parameters
+    prereq.py    - checks python version and for required packages
+    version.py   - version and last modified information
+    settings.cfg - attributes loaded for parsing tor related data
+    uninstall    - removal script
     
     interface/
       graphing/

Modified: arm/trunk/TODO
===================================================================
--- arm/trunk/TODO	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/TODO	2010-10-22 05:21:29 UTC (rev 23660)
@@ -10,33 +10,31 @@
       bugs are being fixed while refactoring.
       
       [ ] conf panel
-        - move torrc validation into util
-        - fetch text via getinfo rather than reading directly?
-           conn.get_info("config-text")
-        - improve parsing failure notice to give line number
-          just giving "[ARM-WARN] Unable to validate torrc" isn't very
-          helpful...
+        - display and validation needs to recognize config entries that span
+          multiple lines
+        - option to display the armrc configuration
+        - [validation] check if there's missing entries
+          might be able to use "GETINFO config-text" to determine entries that
+          differ from the defaults, then see if they're all in the torrc
       [ ] conn panel
         - expand client connections and note location in circuit (entry-exit)
-        - for clients list all connections to detect what's going through tor
-          and what isn't? If not then netstat calls are unnecessary.
-        - check family connections to see if they're alive (VERSION cell
+        - for clients give an option to list all connections, to tell which are
+          going through tor and which might be leaking
+        - check family members to see if they're alive (VERSION cell
           handshake?)
         - fallback when pid or connection querying via pid is unavailable
           List all connections listed both by netstat and the consensus
         - note when connection times are estimates (color?), ie connection
           was established before arm
         - connection uptime to associate inbound/outbound connections?
-        - Identify controller connections (if it's arm, vidalia, etc) with
+        - identify controller connections (if it's arm, vidalia, etc) with
           special detail page for them
-        - provide bridge / client country statistics
+        - provide bridge / client country / exiting port statistics
           Include bridge related data via GETINFO option (feature request
           by waltman and ioerror).
         - pick apart applications like iftop and pktstat to see how they get
           per-connection bandwidth usage. Forum thread discussing it:
           https://bbs.archlinux.org/viewtopic.php?pid=715906
-        - give usage stats for exit port usage (popup?)
-        - country data for client connections (requested by ioerror)
       [ ] attempt to clear controller password from memory
         - http://www.codexon.com/posts/clearing-passwords-in-memory-with-python
   * release prep
@@ -74,17 +72,6 @@
       tor's uptime - blocked on implementation of the following proposal:
       https://gitweb.torproject.org/tor.git/blob/HEAD:/doc/spec/proposals/173-getinfo-option-expansion.txt
   
-  * conf panel:
-    * torrc validation doesn't catch if parameters are missing
-    * scrolling in the torrc isn't working properly when comments are stripped
-        Current method of displaying torrc is pretty stupid (lots of repeated
-        work in display loop). When rewritten fixing this bug should be
-        trivial.
-    * "ExitPolicy" entry in torrc (without path)
-        Produces "May 26 22:11:03.484 [warn] The abbreviation 'ExitPolic' is
-        deprecated. Please use 'ExitPolicy' instead". This is an error in the
-        torrc parsing when only the key is provided.
-  
   * conn panel:
     * *never* do reverse dns lookups for first hops (could be resolving via
       tor and hence leaking to the exit)
@@ -145,13 +132,11 @@
           fetching everything at each client
         * possibly make these archives downloadable from peer relays (this is a
           no-go for clients) via torrents or some dirport like scheme
-    * look at vidalia for ideas
+    * look at Vidalia and TorK for ideas
     * need to solicit for ideas on what would be most helpful to clients
-  * general purpose method of erroring nicely
-    Some errors cause portions of the display to die, but curses limps along
-    and overwrites the stacktrace. This has been mostly solved, but all errors
-    should result in a clean death, with the stacktrace saved and a nice
-    message for the user.
+    * dialog with bridge statuses (idea by mikeperry)
+      https://trac.vidalia-project.net/ticket/570
+      https://trac.torproject.org/projects/tor/ticket/2068
   * handle mutiple tor instances
     * screen style (dialog for switching between instances)
     * extra window with whatever stats can be aggregated over all instances
@@ -179,6 +164,8 @@
         Plugin for distutils. Like most mac packaging, this can only run on a
         mac. It also requires setuptools:
         http://www.errorhelp.com/search/details/74034/importerror-no-module-named-setuptools
+  * look through vidalia's tickets for more ideas
+    https://trac.vidalia-project.net/
   * look into additions to the used apis
     * curses (python 2.6 extended?): http://docs.python.org/library/curses.html
     * new control options (like "desc-annotations/id/<OR identity>")?
@@ -216,6 +203,9 @@
   * implement control-spec proposals:
     * https://gitweb.torproject.org/tor.git/blob/HEAD:/doc/spec/proposals/172-circ-getinfo-option.txt
     * https://gitweb.torproject.org/tor.git/blob/HEAD:/doc/spec/proposals/173-getinfo-option-expansion.txt
+  * gui frontend (gtk?)
+    Look into if the arm utilities and codebase would fit nicely for a gui
+    controller like Vidalia and TorK.
   * unit tests
     Primarily for util, for instance 'addfstr' would be a good candidate.
   * python 3 compatibility

Deleted: arm/trunk/armrc.sample
===================================================================
--- arm/trunk/armrc.sample	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/armrc.sample	2010-10-22 05:21:29 UTC (rev 23660)
@@ -1 +0,0 @@
-link src/armrc.defaults
\ No newline at end of file

Modified: arm/trunk/debian/MANIFEST
===================================================================
--- arm/trunk/debian/MANIFEST	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/debian/MANIFEST	2010-10-22 05:21:29 UTC (rev 23660)
@@ -2,12 +2,13 @@
 setup.cfg
 setup.py
 arm
+armrc.sample
 debian/arm.1.gz
 src/__init__.py
 src/prereq.py
 src/starter.py
 src/version.py
-src/armrc.defaults
+src/settings.cfg
 src/TorCtl/GeoIPSupport.py
 src/TorCtl/PathSupport.py
 src/TorCtl/SQLSupport.py

Modified: arm/trunk/setup.py
===================================================================
--- arm/trunk/setup.py	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/setup.py	2010-10-22 05:21:29 UTC (rev 23660)
@@ -14,7 +14,7 @@
       packages=['arm', 'arm.interface', 'arm.interface.graphing', 'arm.util', 'arm.TorCtl'],
       package_dir={'arm': 'src'},
       data_files=[("/usr/bin", ["arm"]),
-                  ("/usr/lib/arm", ["src/armrc.defaults"]),
+                  ("/usr/lib/arm", ["src/settings.cfg"]),
                   ("/usr/share/man/man1", ["debian/arm.1.gz"])],
      )
 

Deleted: arm/trunk/src/armrc.defaults
===================================================================
--- arm/trunk/src/armrc.defaults	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/armrc.defaults	2010-10-22 05:21:29 UTC (rev 23660)
@@ -1,229 +0,0 @@
-# startup options
-startup.controlPassword
-startup.interface.ipAddress 127.0.0.1
-startup.interface.port 9051
-startup.blindModeEnabled false
-startup.events N3
-
-# Seconds between querying information
-queries.ps.rate 5
-queries.connections.minRate 5
-queries.refreshRate.rate 5
-
-# Renders the interface with color if set and the terminal supports it
-features.colorInterface true
-
-# Set this if you're running in a chroot jail or other environment where tor's
-# resources (log, state, etc) should have a prefix in their paths.
-features.pathPrefix
-
-# If set, arm appends any log messages it reports while running to the given
-# log file. This does not take filters into account or include prepopulated
-# events.
-features.logFile 
-
-# Paremters for the log panel
-# ---------------------------
-# showDateDividers
-#   show borders with dates for entries from previous days
-# showDuplicateEntries
-#   shows all log entries if true, otherwise collapses similar entries with an
-#   indicator for how much is being hidden
-# entryDuration
-#   number of days log entries are kept before being dropped (if zero then
-#   they're kept until cropped due to caching limits)
-# maxLinesPerEntry
-#   max number of lines to display for a single log entry
-# prepopulate
-#   attempts to read past events from the log file if true
-# prepopulateReadLimit
-#   maximum entries read from the log file, used to prevent huge log files from
-#   causing a slow startup time.
-# maxRefreshRate
-#   rate limiting (in milliseconds) for drawing the log if updates are made
-#   rapidly (for instance, when at the DEBUG runlevel)
-
-features.log.showDateDividers true
-features.log.showDuplicateEntries false
-features.log.entryDuration 7
-features.log.maxLinesPerEntry 4
-features.log.prepopulate true
-features.log.prepopulateReadLimit 5000
-features.log.maxRefreshRate 300
-
-# General graph parameters
-# ------------------------
-# height
-#   height of graphed stats
-# maxWidth
-#   maximum number of graphed entries
-# interval
-#   0 -> each second,   1 -> 5 seconds,     2 -> 30 seconds,  3 -> minutely,      
-#   4 -> 15 minutes,    5 -> half hour,     6 -> hourly,      7 -> daily
-# bound
-#   0 -> global maxima, 1 -> local maxima,  2 -> tight
-# type
-#   0 -> None, 1 -> Bandwidth, 2 -> Connections, 3 -> System Resources
-# showIntermediateBounds
-#   shows y-axis increments between the top/bottom bounds
-
-features.graph.height 7
-features.graph.maxWidth 150
-features.graph.interval 0
-features.graph.bound 1
-features.graph.type 1
-features.graph.showIntermediateBounds true
-
-# Parameters for graphing bandwidth stats
-# ---------------------------------------
-# prepopulate
-#   attempts to use tor's state file to prepopulate the bandwidth graph at the
-#   15-minute interval (this requires the minimum of a day's worth of uptime)
-# transferInBystes
-#   shows rate measurments in bytes if true, bits otherwise
-# accounting.show
-#   provides accounting stats if AccountingMax was set
-# accounting.rate
-#   seconds between querying accounting stats
-# accounting.isTimeLong
-#   provides verbose measurements of time if true
-
-features.graph.bw.prepopulate true
-features.graph.bw.transferInBytes false
-features.graph.bw.accounting.show true
-features.graph.bw.accounting.rate 10
-features.graph.bw.accounting.isTimeLong false
-
-# Parameters for graphing ps stats
-# --------------------------------
-# primary/secondaryStat
-#   any numeric field provided by the ps command
-# cachedOnly
-#   determines if the graph should query ps or rely on cached results (this
-#   lowers the call volume but limits the graph's granularity)
-
-features.graph.ps.primaryStat %cpu
-features.graph.ps.secondaryStat rss
-features.graph.ps.cachedOnly true
-
-# Thread pool size for hostname resolutions
-# Determines the maximum number of concurrent requests. Upping this to around
-# thirty or so seems to be problematic, causing intermittently seizing.
-
-queries.hostnames.poolSize 5
-
-# Method of resolving hostnames
-# If true, uses python's internal "socket.gethostbyaddr" to resolve addresses
-# rather than the host command. This is ignored if the system's unable to make
-# parallel requests. Resolving this way seems to be much slower than host calls
-# in practice.
-
-queries.hostnames.useSocketModule false
-
-# Caching parameters
-cache.sysCalls.size 600
-cache.hostnames.size 700000
-cache.hostnames.trimSize 200000
-cache.logPanel.size 1000
-cache.armLog.size 1000
-cache.armLog.trimSize 200
-
-# Runlevels at which arm logs its events
-log.refreshRate DEBUG
-log.configEntryNotFound NONE
-log.configEntryUndefined NOTICE
-log.configEntryTypeError NOTICE
-log.torCtlPortClosed NOTICE
-log.torGetInfo DEBUG
-log.torGetConf DEBUG
-log.torEventTypeUnrecognized NOTICE
-log.torPrefixPathInvalid NOTICE
-log.sysCallMade DEBUG
-log.sysCallCached NONE
-log.sysCallFailed INFO
-log.sysCallCacheGrowing INFO
-log.panelRecreated DEBUG
-log.graph.ps.invalidStat WARN
-log.graph.ps.abandon WARN
-log.graph.bw.prepopulateSuccess NOTICE
-log.graph.bw.prepopulateFailure NOTICE
-log.logPanel.prepopulateSuccess INFO
-log.logPanel.prepopulateFailed WARN
-log.logPanel.logFileOpened NOTICE
-log.logPanel.logFileWriteFailed ERR
-log.logPanel.forceDoubleRedraw DEBUG
-log.connLookupFailed INFO
-log.connLookupFailover NOTICE
-log.connLookupAbandon WARN
-log.connLookupRateGrowing NONE
-log.hostnameCacheTrimmed INFO
-log.cursesColorSupport INFO
-
-# Snippets from common log messages
-# These are static bits of log messages, used to determine when entries with
-# dynamic content (hostnames, numbers, etc) are the same. If this matches the
-# start of both messages then the entries are flagged as duplicates. If the
-# entry begins with an asterisk (*) then it checks if the substrings exist
-# anywhere in the messages.
-# 
-# Examples for the complete messages:
-# [BW] READ: 0, WRITTEN: 0
-# [DEBUG] connection_handle_write(): After TLS write of 512: 0 read, 586 written
-# [DEBUG] flush_chunk_tls(): flushed 512 bytes, 0 ready to flush, 0 remain.
-# [DEBUG] conn_read_callback(): socket 7 wants to read.
-# [DEBUG] conn_write_callback(): socket 51 wants to write.
-# [DEBUG] connection_remove(): removing socket -1 (type OR), n_conns now 50
-# [DEBUG] connection_or_process_cells_from_inbuf(): 7: starting, inbuf_datalen 0 (0 pending in tls object).
-# [DEBUG] connection_read_to_buf(): 38: starting, inbuf_datalen 0 (0 pending in tls object). at_most 12800.
-# [DEBUG] connection_read_to_buf(): TLS connection closed on read. Closing. (Nickname moria1, address 128.31.0.34)
-# [INFO] run_connection_housekeeping(): Expiring non-open OR connection to fd 16 (79.193.61.171:443).
-# [INFO] rep_hist_downrate_old_runs(): Discounting all old stability info by a factor of 0.950000
-# [NOTICE] We stalled too much while trying to write 150 bytes to address
-#          [scrubbed].  If this happens a lot, either something is wrong with
-#          your network connection, or something is wrong with theirs. (fd 238,
-#          type Directory, state 1, marked at main.c:702).
-# [NOTICE] I learned some more directory information, but not enough to build a
-#          circuit: We have only 469/2027 usable descriptors.
-# [NOTICE] Attempt by %s to open a stream from unknown relay. Closing.
-# [WARN] You specified a server "Amunet8" by name, but this name is not
-#        registered
-# [WARN] I have no descriptor for the router named "Amunet8" in my declared
-#        family; I'll use the nickname as is, but this   may confuse clients.
-# [WARN] Problem bootstrapping. Stuck at 80%: Connecting to the Tor network.
-#        (Network is unreachable; NOROUTE; count 47;    recommendation warn)
-# [WARN] 4 unknown, 1 missing key, 3 good, 0 bad, 1 no signature, 4 required
-# [ARM_DEBUG] refresh rate: 0.001 seconds
-# [ARM_DEBUG] system call: ps -p 2354 -o %cpu,rss,%mem,etime (runtime: 0.02)
-# [ARM_DEBUG] system call: netstat -npt | grep 2354/tor (runtime: 0.02)
-# [ARM_DEBUG] recreating panel 'graph' with the dimensions of 14/124
-# [ARM_DEBUG] redrawing the log panel with the corrected content height (estimat was off by 4)
-# [ARM_DEBUG] GETINFO accounting/bytes-left (runtime: 0.0006)
-
-msg.BW READ:
-msg.DEBUG connection_handle_write(): After TLS write of
-msg.DEBUG flush_chunk_tls(): flushed
-msg.DEBUG conn_read_callback(): socket
-msg.DEBUG conn_write_callback(): socket
-msg.DEBUG connection_remove(): removing socket
-msg.DEBUG connection_or_process_cells_from_inbuf():
-msg.DEBUG *pending in tls object). at_most
-msg.DEBUG connection_read_to_buf(): TLS connection closed on read. Closing.
-msg.INFO run_connection_housekeeping(): Expiring
-msg.INFO rep_hist_downrate_old_runs(): Discounting all old stability info by a factor of
-msg.NOTICE We stalled too much while trying to write
-msg.NOTICE I learned some more directory information, but not enough to build a circuit
-msg.NOTICE Attempt by
-msg.WARN You specified a server
-msg.WARN I have no descriptor for the router named
-msg.WARN Problem bootstrapping. Stuck at
-msg.WARN *missing key,
-msg.ARM_DEBUG refresh rate:
-msg.ARM_DEBUG system call: ps
-msg.ARM_DEBUG system call: netstat
-msg.ARM_DEBUG recreating panel '
-msg.ARM_DEBUG redrawing the log panel with the corrected content height (
-msg.ARM_DEBUG GETINFO accounting/bytes
-msg.ARM_DEBUG GETINFO accounting/bytes-left
-msg.ARM_DEBUG GETINFO accounting/interval-end
-msg.ARM_DEBUG GETINFO accounting/hibernating
-

Modified: arm/trunk/src/interface/confPanel.py
===================================================================
--- arm/trunk/src/interface/confPanel.py	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/interface/confPanel.py	2010-10-22 05:21:29 UTC (rev 23660)
@@ -1,292 +1,264 @@
-#!/usr/bin/env python
-# confPanel.py -- Presents torrc with syntax highlighting.
-# Released under the GPL v3 (http://www.gnu.org/licenses/gpl.html)
+"""
+Panel displaying the torrc and validation done against it.
+"""
 
 import math
 import curses
-import socket
+import threading
 
-import controller
-from TorCtl import TorCtl
-from util import log, panel, torTools, uiTools
+from util import log, panel, torrc, uiTools
 
-# torrc parameters that can be defined multiple times without overwriting
-# from src/or/config.c (entries with LINELIST or LINELIST_S)
-# last updated for tor version 0.2.1.19
-MULTI_LINE_PARAM = ["AlternateBridgeAuthority", "AlternateDirAuthority", "AlternateHSAuthority", "AuthDirBadDir", "AuthDirBadExit", "AuthDirInvalid", "AuthDirReject", "Bridge", "ControlListenAddress", "ControlSocket", "DirListenAddress", "DirPolicy", "DirServer", "DNSListenAddress", "ExitPolicy", "HashedControlPassword", "HiddenServiceDir", "HiddenServiceOptions", "HiddenServicePort", "HiddenServiceVersion", "HiddenServiceAuthorizeClient", "HidServAuth", "Log", "MapAddress", "NatdListenAddress", "NodeFamily", "ORListenAddress", "ReachableAddresses", "ReachableDirAddresses", "ReachableORAddresses", "RecommendedVersions", "RecommendedClientVersions", "RecommendedServerVersions", "SocksListenAddress", "SocksPolicy", "TransListenAddress", "__HashedControlSessionPassword"]
+DEFAULT_CONFIG = {"features.torrc.validate": True,
+                  "features.config.showScrollbars": True,
+                  "features.config.maxLinesPerEntry": 5,
+                  "log.confPanel.torrcReadFailed": log.WARN,
+                  "log.torrcValidation.duplicateEntries": log.NOTICE,
+                  "log.torrcValidation.torStateDiffers": log.NOTICE}
 
-# hidden service options need to be fetched with HiddenServiceOptions
-HIDDEN_SERVICE_PARAM = ["HiddenServiceDir", "HiddenServiceOptions", "HiddenServicePort", "HiddenServiceVersion", "HiddenServiceAuthorizeClient"]
-HIDDEN_SERVICE_FETCH_PARAM = "HiddenServiceOptions"
-
-# size modifiers allowed by config.c
-LABEL_KB = ["kb", "kbyte", "kbytes", "kilobyte", "kilobytes"]
-LABEL_MB = ["m", "mb", "mbyte", "mbytes", "megabyte", "megabytes"]
-LABEL_GB = ["gb", "gbyte", "gbytes", "gigabyte", "gigabytes"]
-LABEL_TB = ["tb", "terabyte", "terabytes"]
-
-# GETCONF aliases (from the _option_abbrevs struct of src/or/config.c)
-# fix for: https://trac.torproject.org/projects/tor/ticket/1798
-# TODO: this has been fixed in tor- wait for a while then retest and remove
-# TODO: the following alias entry doesn't work on Tor 0.2.1.19:
-# "HashedControlPassword": "__HashedControlSessionPassword"
-CONF_ALIASES = {"l": "Log",
-                "AllowUnverifiedNodes": "AllowInvalidNodes",
-                "AutomapHostSuffixes": "AutomapHostsSuffixes",
-                "AutomapHostOnResolve": "AutomapHostsOnResolve",
-                "BandwidthRateBytes": "BandwidthRate",
-                "BandwidthBurstBytes": "BandwidthBurst",
-                "DirFetchPostPeriod": "StatusFetchPeriod",
-                "MaxConn": "ConnLimit",
-                "ORBindAddress": "ORListenAddress",
-                "DirBindAddress": "DirListenAddress",
-                "SocksBindAddress": "SocksListenAddress",
-                "UseHelperNodes": "UseEntryGuards",
-                "NumHelperNodes": "NumEntryGuards",
-                "UseEntryNodes": "UseEntryGuards",
-                "NumEntryNodes": "NumEntryGuards",
-                "ResolvConf": "ServerDNSResolvConfFile",
-                "SearchDomains": "ServerDNSSearchDomains",
-                "ServerDNSAllowBrokenResolvConf": "ServerDNSAllowBrokenConfig",
-                "PreferTunnelledDirConns": "PreferTunneledDirConns",
-                "BridgeAuthoritativeDirectory": "BridgeAuthoritativeDir",
-                "StrictEntryNodes": "StrictNodes",
-                "StrictExitNodes": "StrictNodes"}
-
-
-# time modifiers allowed by config.c
-LABEL_MIN = ["minute", "minutes"]
-LABEL_HOUR = ["hour", "hours"]
-LABEL_DAY = ["day", "days"]
-LABEL_WEEK = ["week", "weeks"]
-
 class ConfPanel(panel.Panel):
   """
-  Presents torrc with syntax highlighting in a scroll-able area.
+  Presents torrc, armrc, or loaded settings with syntax highlighting in a
+  scrollable area.
   """
   
-  def __init__(self, stdscr, confLocation, conn):
+  def __init__(self, stdscr, config=None):
     panel.Panel.__init__(self, stdscr, "conf", 0)
-    self.confLocation = confLocation
-    self.showLineNum = True
-    self.stripComments = False
-    self.confContents = []
-    self.scroll = 0
     
-    # lines that don't matter due to duplicates
-    self.irrelevantLines = []
+    self._config = dict(DEFAULT_CONFIG)
+    if config:
+      config.update(self._config, {"features.config.maxLinesPerEntry": 1})
     
-    # used to check consistency with tor's actual values - corrections mapping
-    # is of line numbers (one-indexed) to tor's actual values
+    self.valsLock = threading.RLock()
+    self.scroll = 0
+    self.showLineNum = True
+    self.stripComments = False
+    self.confLocation = ""
+    self.confContents = None # read torrc, None if it failed to load
     self.corrections = {}
-    self.conn = conn
     
+    # height of the content when last rendered (the cached value is invalid if
+    # _lastContentHeightArgs is None or differs from the current dimensions)
+    self._lastContentHeight = 1
+    self._lastContentHeightArgs = None
+    
     self.reset()
   
-  def reset(self, logErrors=True):
+  def reset(self, logErrors = True):
     """
     Reloads torrc contents and resets scroll height. Returns True if
     successful, else false.
+    
+    Arguments:
+      logErrors - logs if unable to read the torrc or issues are found during
+                  validation
     """
     
+    self.valsLock.acquire()
+    
     try:
-      resetSuccessful = True
-      
-      confFile = open(torTools.getPathPrefix() + self.confLocation, "r")
+      self.confLocation = torrc.getConfigLocation()
+      confFile = open(self.confLocation, "r")
       self.confContents = confFile.readlines()
       confFile.close()
+      self.scroll = 0
       
-      # checks if torrc differs from get_option data
-      self.irrelevantLines = []
-      self.corrections = {}
-      parsedCommands = {}       # mapping of parsed commands to line numbers
+      # sets the content height to be something somewhat reasonable
+      self._lastContentHeight = len(self.confContents)
+      self._lastContentHeightArgs = None
+    except IOError, exc:
+      self.confContents = None
+      msg = "Unable to load torrc (%s)" % exc
+      if logErrors: log.log(self._config["log.confPanel.torrcReadFailed"], msg)
+      self.valsLock.release()
+      return False
+    
+    if self._config["features.torrc.validate"]:
+      self.corrections = torrc.validate(self.confContents)
       
-      for lineNumber in range(len(self.confContents)):
-        lineText = self.confContents[lineNumber].strip()
+      if self.corrections and logErrors:
+        # logs issues found during validation
+        irrelevantLines, mismatchLines = [], []
+        for lineNum in self.corrections:
+          problem = self.corrections[lineNum][0]
+          if problem == torrc.VAL_DUPLICATE: irrelevantLines.append(lineNum)
+          elif problem == torrc.VAL_MISMATCH: mismatchLines.append(lineNum)
         
-        if lineText and lineText[0] != "#":
-          # relevant to tor (not blank nor comment)
-          ctlEnd = lineText.find(" ")   # end of command
-          argEnd = lineText.find("#")   # end of argument (start of comment or end of line)
-          if argEnd == -1: argEnd = len(lineText)
-          command, argument = lineText[:ctlEnd], lineText[ctlEnd:argEnd].strip()
+        if irrelevantLines:
+          irrelevantLines.sort()
           
-          # replace aliases with the internal representation of the command
-          if command in CONF_ALIASES: command = CONF_ALIASES[command]
-          
-          # tor appears to replace tabs with a space, for instance:
-          # "accept\t*:563" is read back as "accept *:563"
-          argument = argument.replace("\t", " ")
-          
-          # expands value if it's a size or time
-          comp = argument.strip().lower().split(" ")
-          if len(comp) > 1:
-            size = 0
-            if comp[1] in LABEL_KB: size = int(comp[0]) * 1024
-            elif comp[1] in LABEL_MB: size = int(comp[0]) * 1048576
-            elif comp[1] in LABEL_GB: size = int(comp[0]) * 1073741824
-            elif comp[1] in LABEL_TB: size = int(comp[0]) * 1099511627776
-            elif comp[1] in LABEL_MIN: size = int(comp[0]) * 60
-            elif comp[1] in LABEL_HOUR: size = int(comp[0]) * 3600
-            elif comp[1] in LABEL_DAY: size = int(comp[0]) * 86400
-            elif comp[1] in LABEL_WEEK: size = int(comp[0]) * 604800
-            if size != 0: argument = str(size)
-              
-          # most parameters are overwritten if defined multiple times, if so
-          # it's erased from corrections and noted as duplicate instead
-          if not command in MULTI_LINE_PARAM and command in parsedCommands.keys():
-            previousLineNum = parsedCommands[command]
-            self.irrelevantLines.append(previousLineNum)
-            if previousLineNum in self.corrections.keys(): del self.corrections[previousLineNum]
-          
-          parsedCommands[command] = lineNumber + 1
-          
-          # check validity against tor's actual state
-          try:
-            actualValues = []
-            if command in HIDDEN_SERVICE_PARAM:
-              # hidden services are fetched via a special command
-              hsInfo = self.conn.get_option(HIDDEN_SERVICE_FETCH_PARAM)
-              for entry in hsInfo:
-                if entry[0] == command:
-                  actualValues.append(entry[1])
-                  break
-            else:
-              # general case - fetch all valid values
-              for key, val in self.conn.get_option(command):
-                if val == None:
-                  # TODO: investigate situations where this might occure
-                  # (happens if trying to parse HIDDEN_SERVICE_PARAM)
-                  if logErrors: log.log(log.WARN, "BUG: Failed to find torrc value for %s" % key)
-                  continue
-                
-                # TODO: check for a better way of figuring out CSV parameters
-                # (kinda doubt this is right... in config.c its listed as being
-                # a 'LINELIST') - still, good enough for common cases
-                if command in MULTI_LINE_PARAM: toAdd = val.split(",")
-                else: toAdd = [val]
-                
-                for newVal in toAdd:
-                  newVal = newVal.strip()
-                  if newVal not in actualValues: actualValues.append(newVal)
-            
-            # there might be multiple values on a single line - if so, check each
-            if command in MULTI_LINE_PARAM and "," in argument:
-              arguments = []
-              for entry in argument.split(","):
-                arguments.append(entry.strip())
-            else:
-              arguments = [argument]
-            
-            for entry in arguments:
-              if not entry in actualValues:
-                self.corrections[lineNumber + 1] = ", ".join(actualValues)
-          except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
-            if logErrors: log.log(log.WARN, "Unable to validate line %i of the torrc: %s" % (lineNumber + 1, lineText))
-      
-      # logs issues that arose
-      if self.irrelevantLines and logErrors:
-        if len(self.irrelevantLines) > 1: first, second, third = "Entries", "are", ", including lines"
-        else: first, second, third = "Entry", "is", " on line"
-        baseMsg = "%s in your torrc %s ignored due to duplication%s" % (first, second, third)
+          if len(irrelevantLines) > 1: first, second, third = "Entries", "are", ", including lines"
+          else: first, second, third = "Entry", "is", " on line"
+          msgStart = "%s in your torrc %s ignored due to duplication%s" % (first, second, third)
+          msgLines = ", ".join([str(val + 1) for val in irrelevantLines])
+          msg = "%s: %s (highlighted in blue)" % (msgStart, msgLines)
+          log.log(self._config["log.torrcValidation.duplicateEntries"], msg)
         
-        log.log(log.NOTICE, "%s: %s (highlighted in blue)" % (baseMsg, ", ".join([str(val) for val in self.irrelevantLines])))
-      
-      if self.corrections and logErrors:
-        log.log(log.WARN, "Tor's state differs from loaded torrc")
-    except IOError, exc:
-      resetSuccessful = False
-      self.confContents = ["### Unable to load torrc ###"]
-      if logErrors: log.log(log.WARN, "Unable to load torrc (%s)" % str(exc))
+        if mismatchLines:
+          mismatchLines.sort()
+          msgStart = "Tor's state differs from loaded torrc on line%s" % ("s" if len(mismatchLines) > 1 else "")
+          msgLines = ", ".join([str(val + 1) for val in mismatchLines])
+          msg = "%s: %s" % (msgStart, msgLines)
+          log.log(self._config["log.torrcValidation.torStateDiffers"], msg)
     
-    self.scroll = 0
-    return resetSuccessful
+    if self.confContents:
+      # Restricts contents to be displayable characters:
+      # - Tabs print as three spaces. Keeping them as tabs is problematic for
+      #   the layout since it's counted as a single character, but occupies
+      #   several cells.
+      # - Strips control and unprintable characters.
+      for lineNum in range(len(self.confContents)):
+        lineText = self.confContents[lineNum]
+        lineText = lineText.replace("\t", "   ")
+        lineText = "".join([char for char in lineText if curses.ascii.isprint(char)])
+        self.confContents[lineNum] = lineText
+    
+    self.redraw(True)
+    self.valsLock.release()
+    return True
   
   def handleKey(self, key):
-    if uiTools.isScrollKey(key):
+    self.valsLock.acquire()
+    if uiTools.isScrollKey(key) and self.confContents != None:
       pageHeight = self.getPreferredSize()[0] - 1
-      contentHeight = len(self.confContents)
-      self.scroll = uiTools.getScrollPosition(key, self.scroll, pageHeight, contentHeight)
-    elif key == ord('n') or key == ord('N'): self.showLineNum = not self.showLineNum
+      newScroll = uiTools.getScrollPosition(key, self.scroll, pageHeight, self._lastContentHeight)
+      
+      if self.scroll != newScroll:
+        self.scroll = newScroll
+        self.redraw(True)
+    elif key == ord('n') or key == ord('N'):
+      self.showLineNum = not self.showLineNum
+      self._lastContentHeightArgs = None
+      self.redraw(True)
     elif key == ord('s') or key == ord('S'):
       self.stripComments = not self.stripComments
       self.scroll = 0
-    self.redraw(True)
+      self._lastContentHeightArgs = None
+      self.redraw(True)
+    
+    self.valsLock.release()
   
   def draw(self, subwindow, width, height):
-    self.addstr(0, 0, "Tor Config (%s):" % self.confLocation, curses.A_STANDOUT)
+    self.valsLock.acquire()
     
-    pageHeight = height - 1
-    if self.confContents: numFieldWidth = int(math.log10(len(self.confContents))) + 1
-    else: numFieldWidth = 0 # torrc is blank
-    lineNum, displayLineNum = self.scroll + 1, 1 # lineNum corresponds to torrc, displayLineNum concerns what's presented
+    # If true, we assume that the cached value in self._lastContentHeight is
+    # still accurate, and stop drawing when there's nothing more to display.
+    # Otherwise the self._lastContentHeight is suspect, and we'll process all
+    # the content to check if it's right (and redraw again with the corrected
+    # height if not).
+    trustLastContentHeight = self._lastContentHeightArgs == (width, height)
     
-    # determine the ending line in the display (prevents us from going to the 
-    # effort of displaying lines that aren't visible - isn't really a 
-    # noticeable improvement unless the torrc is bazaarly long) 
-    if not self.stripComments:
-      endingLine = min(len(self.confContents), self.scroll + pageHeight)
-    else:
-      # checks for the last line of displayable content (ie, non-comment)
-      endingLine = self.scroll
-      displayedLines = 0        # number of lines of content
-      for i in range(self.scroll, len(self.confContents)):
-        endingLine += 1
-        lineText = self.confContents[i].strip()
-        
-        if lineText and lineText[0] != "#":
-          displayedLines += 1
-          if displayedLines == pageHeight: break
+    # draws the top label
+    locationLabel = " (%s)" % self.confLocation if self.confLocation else ""
+    self.addstr(0, 0, "Tor Config%s:" % locationLabel, curses.A_STANDOUT)
     
-    for i in range(self.scroll, endingLine):
-      lineText = self.confContents[i].strip()
-      skipLine = False # true if we're not presenting line due to stripping
+    # restricts scroll location to valid bounds
+    self.scroll = max(0, min(self.scroll, self._lastContentHeight - height + 1))
+    
+    renderedContents = self.confContents
+    if self.confContents == None:
+      renderedContents = ["### Unable to load torrc ###"]
+    elif self.stripComments:
+      renderedContents = torrc.stripComments(self.confContents)
+    
+    # offset to make room for the line numbers
+    lineNumOffset = int(math.log10(len(renderedContents))) + 2 if self.showLineNum else 0
+    
+    # draws left-hand scroll bar if content's longer than the height
+    scrollOffset = 0
+    if self._config["features.config.showScrollbars"] and self._lastContentHeight > height - 1:
+      scrollOffset = 3
+      self.addScrollBar(self.scroll, self.scroll + height - 1, self._lastContentHeight, 1)
+    
+    displayLine = -self.scroll + 1 # line we're drawing on
+    
+    for lineNumber in range(0, len(renderedContents)):
+      lineText = renderedContents[lineNumber]
+      lineText = lineText.rstrip() # remove ending whitespace
       
-      command, argument, correction, comment = "", "", "", ""
-      commandColor, argumentColor, correctionColor, commentColor = "green", "cyan", "cyan", "white"
+      # blank lines are hidden when stripping comments
+      hideLine = self.stripComments and not lineText
       
-      if not lineText:
-        # no text
-        if self.stripComments: skipLine = True
-      elif lineText[0] == "#":
-        # whole line is commented out
-        comment = lineText
-        if self.stripComments: skipLine = True
+      # splits the line into its component (msg, format) tuples
+      lineComp = {"option": ["", curses.A_BOLD | uiTools.getColor("green")],
+                  "argument": ["", curses.A_BOLD | uiTools.getColor("cyan")],
+                  "correction": ["", curses.A_BOLD | uiTools.getColor("cyan")],
+                  "comment": ["", uiTools.getColor("white")]}
+      
+      # parses the comment
+      commentIndex = lineText.find("#")
+      if commentIndex != -1:
+        lineComp["comment"][0] = lineText[commentIndex:]
+        lineText = lineText[:commentIndex]
+      
+      # splits the option and argument, preserving any whitespace around them
+      strippedLine = lineText.strip()
+      optionIndex = strippedLine.find(" ")
+      if optionIndex == -1:
+        lineComp["option"][0] = lineText # no argument provided
       else:
-        # parse out command, argument, and possible comment
-        ctlEnd = lineText.find(" ")   # end of command
-        argEnd = lineText.find("#")   # end of argument (start of comment or end of line)
-        if argEnd == -1: argEnd = len(lineText)
+        optionText = strippedLine[:optionIndex]
+        optionEnd = lineText.find(optionText) + len(optionText)
+        lineComp["option"][0] = lineText[:optionEnd]
+        lineComp["argument"][0] = lineText[optionEnd:]
+      
+      # gets the correction
+      if lineNumber in self.corrections:
+        lineIssue, lineIssueMsg = self.corrections[lineNumber]
         
-        command, argument, comment = lineText[:ctlEnd], lineText[ctlEnd:argEnd], lineText[argEnd:]
-        if self.stripComments: comment = ""
-        
-        # Tabs print as three spaces. Keeping them as tabs is problematic for
-        # the layout since it's counted as a single character, but occupies
-        # several cells.
-        argument = argument.replace("\t", "   ")
-        
-        # changes presentation if value's incorrect or irrelevant
-        if lineNum in self.corrections.keys():
-          argumentColor = "red"
-          correction = " (%s)" % self.corrections[lineNum]
-        elif lineNum in self.irrelevantLines:
-          commandColor = "blue"
-          argumentColor = "blue"
+        if lineIssue == torrc.VAL_DUPLICATE:
+          lineComp["option"][1] = curses.A_BOLD | uiTools.getColor("blue")
+          lineComp["argument"][1] = curses.A_BOLD | uiTools.getColor("blue")
+        elif lineIssue == torrc.VAL_MISMATCH:
+          lineComp["argument"][1] = curses.A_BOLD | uiTools.getColor("red")
+          lineComp["correction"][0] = " (%s)" % lineIssueMsg
       
-      if not skipLine:
-        numOffset = 0     # offset for line numbering
-        if self.showLineNum:
-          self.addstr(displayLineNum, 0, ("%%%ii" % numFieldWidth) % lineNum, curses.A_BOLD | uiTools.getColor("yellow"))
-          numOffset = numFieldWidth + 1
+      # draws the line number
+      if self.showLineNum and not hideLine and displayLine < height and displayLine >= 1:
+        lineNumStr = ("%%%ii" % (lineNumOffset - 1)) % (lineNumber + 1)
+        self.addstr(displayLine, scrollOffset, lineNumStr, curses.A_BOLD | uiTools.getColor("yellow"))
+      
+      # draws the rest of the components with line wrap
+      cursorLoc, lineOffset = lineNumOffset + scrollOffset, 0
+      maxLinesPerEntry = self._config["features.config.maxLinesPerEntry"]
+      displayQueue = [lineComp[entry] for entry in ("option", "argument", "correction", "comment")]
+      
+      while displayQueue:
+        msg, format = displayQueue.pop(0)
+        if hideLine: break
         
-        xLoc = 0
-        displayLineNum, xLoc = controller.addstr_wrap(self, displayLineNum, xLoc, command, curses.A_BOLD | uiTools.getColor(commandColor), numOffset)
-        displayLineNum, xLoc = controller.addstr_wrap(self, displayLineNum, xLoc, argument, curses.A_BOLD | uiTools.getColor(argumentColor), numOffset)
-        displayLineNum, xLoc = controller.addstr_wrap(self, displayLineNum, xLoc, correction, curses.A_BOLD | uiTools.getColor(correctionColor), numOffset)
-        displayLineNum, xLoc = controller.addstr_wrap(self, displayLineNum, xLoc, comment, uiTools.getColor(commentColor), numOffset)
+        maxMsgSize, includeBreak = width - cursorLoc, False
+        if len(msg) >= maxMsgSize:
+          # message is too long - break it up
+          includeBreak = True
+          if lineOffset == maxLinesPerEntry - 1:
+            msg = uiTools.cropStr(msg, maxMsgSize)
+          else:
+            msg, remainder = uiTools.cropStr(msg, maxMsgSize, 4, 4, uiTools.END_WITH_HYPHEN, True)
+            displayQueue.insert(0, (remainder.strip(), format))
         
-        displayLineNum += 1
+        drawLine = displayLine + lineOffset
+        if msg and drawLine < height and drawLine >= 1:
+          self.addstr(drawLine, cursorLoc, msg, format)
+        
+        cursorLoc += len(msg)
+        if includeBreak or not displayQueue:
+          lineOffset += 1
+          cursorLoc = lineNumOffset + scrollOffset
       
-      lineNum += 1
+      displayLine += lineOffset
+      
+      if trustLastContentHeight and displayLine >= height: break
+    
+    if not trustLastContentHeight:
+      self._lastContentHeightArgs = (width, height)
+      newContentHeight = displayLine + self.scroll - 1
+      
+      if self._lastContentHeight != newContentHeight:
+        self._lastContentHeight = newContentHeight
+        self.redraw(True)
+    
+    self.valsLock.release()
+  
+  def redraw(self, forceRedraw=False, block=False):
+    panel.Panel.redraw(self, forceRedraw, block)
 

Modified: arm/trunk/src/interface/controller.py
===================================================================
--- arm/trunk/src/interface/controller.py	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/interface/controller.py	2010-10-22 05:21:29 UTC (rev 23660)
@@ -344,16 +344,16 @@
   # attempts to determine tor's current pid (left as None if unresolveable, logging an error later)
   torPid = torTools.getConn().getMyPid()
   
-  try:
-    confLocation = conn.get_info("config-file")["config-file"]
-    if confLocation[0] != "/":
-      # relative path - attempt to add process pwd
-      try:
-        results = sysTools.call("pwdx %s" % torPid)
-        if len(results) == 1 and len(results[0].split()) == 2: confLocation = "%s/%s" % (results[0].split()[1], confLocation)
-      except IOError: pass # pwdx call failed
-  except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
-    confLocation = ""
+  #try:
+  #  confLocation = conn.get_info("config-file")["config-file"]
+  #  if confLocation[0] != "/":
+  #    # relative path - attempt to add process pwd
+  #    try:
+  #      results = sysTools.call("pwdx %s" % torPid)
+  #      if len(results) == 1 and len(results[0].split()) == 2: confLocation = "%s/%s" % (results[0].split()[1], confLocation)
+  #    except IOError: pass # pwdx call failed
+  #except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed):
+  #  confLocation = ""
   
   # minor refinements for connection resolver
   if not isBlindMode:
@@ -378,7 +378,7 @@
   
   panels["conn"] = connPanel.ConnPanel(stdscr, conn, isBlindMode)
   panels["control"] = ControlPanel(stdscr, isBlindMode)
-  panels["torrc"] = confPanel.ConfPanel(stdscr, confLocation, conn)
+  panels["torrc"] = confPanel.ConfPanel(stdscr, config)
   
   # provides error if pid coulnd't be determined (hopefully shouldn't happen...)
   if not torPid: log.log(log.WARN, "Unable to resolve tor pid, abandoning connection listing")
@@ -448,7 +448,7 @@
   
   # provides notice about any unused config keys
   for key in config.getUnusedKeys():
-    log.log(CONFIG["log.configEntryUndefined"], "unrecognized configuration entry: %s" % key)
+    log.log(CONFIG["log.configEntryUndefined"], "unused configuration entry: %s" % key)
   
   lastPerformanceLog = 0 # ensures we don't do performance logging too frequently
   redrawStartTime = time.time()
@@ -531,8 +531,8 @@
       for panelKey in (PAGE_S + PAGES[page]):
         # redrawing popup can result in display flicker when it should be hidden
         if panelKey != "popup":
-          if panelKey in ("header", "graph", "log"):
-            # revised panel (handles its own content refreshing)
+          if panelKey in ("header", "graph", "log", "torrc"):
+            # revised panel (manages its own content refreshing)
             panels[panelKey].redraw()
           else:
             panels[panelKey].redraw(True)

Modified: arm/trunk/src/interface/graphing/bandwidthStats.py
===================================================================
--- arm/trunk/src/interface/graphing/bandwidthStats.py	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/interface/graphing/bandwidthStats.py	2010-10-22 05:21:29 UTC (rev 23660)
@@ -37,8 +37,7 @@
     
     self._config = dict(DEFAULT_CONFIG)
     if config:
-      config.update(self._config)
-      self._config["features.graph.bw.accounting.rate"] = max(1, self._config["features.graph.bw.accounting.rate"])
+      config.update(self._config, {"features.graph.bw.accounting.rate": 1})
     
     # accounting data (set by _updateAccountingInfo method)
     self.accountingLastUpdated = 0

Modified: arm/trunk/src/interface/graphing/graphPanel.py
===================================================================
--- arm/trunk/src/interface/graphing/graphPanel.py	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/interface/graphing/graphPanel.py	2010-10-22 05:21:29 UTC (rev 23660)
@@ -48,11 +48,11 @@
           "features.graph.showIntermediateBounds": True}
 
 def loadConfig(config):
-  config.update(CONFIG)
-  CONFIG["features.graph.height"] = max(MIN_GRAPH_HEIGHT, CONFIG["features.graph.height"])
-  CONFIG["features.graph.maxWidth"] = max(1, CONFIG["features.graph.maxWidth"])
-  CONFIG["features.graph.interval"] = min(len(UPDATE_INTERVALS) - 1, max(0, CONFIG["features.graph.interval"]))
-  CONFIG["features.graph.bound"] = min(2, max(0, CONFIG["features.graph.bound"]))
+  config.update(CONFIG, {
+    "features.graph.height": MIN_GRAPH_HEIGHT,
+    "features.graph.maxWidth": 1,
+    "features.graph.interval": (0, len(UPDATE_INTERVALS) - 1),
+    "features.graph.bound": (0, 2)})
 
 class GraphStats(TorCtl.PostEventListener):
   """

Modified: arm/trunk/src/interface/headerPanel.py
===================================================================
--- arm/trunk/src/interface/headerPanel.py	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/interface/headerPanel.py	2010-10-22 05:21:29 UTC (rev 23660)
@@ -62,8 +62,7 @@
     self._config = dict(DEFAULT_CONFIG)
     
     if config:
-      config.update(self._config)
-      self._config["queries.ps.rate"] = max(self._config["queries.ps.rate"], 1)
+      config.update(self._config, {"queries.ps.rate": 1})
     
     self.vals = {}
     self.valsLock = threading.RLock()

Modified: arm/trunk/src/interface/logPanel.py
===================================================================
--- arm/trunk/src/interface/logPanel.py	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/interface/logPanel.py	2010-10-22 05:21:29 UTC (rev 23660)
@@ -168,10 +168,10 @@
   for confKey in armConf.getKeys():
     if confKey.startswith("msg."):
       eventType = confKey[4:].upper()
-      messages = armConf.get(confKey)
+      messages = armConf.get(confKey, [])
       COMMON_LOG_MESSAGES[eventType] = messages
 
-def getLogFileEntries(runlevels, readLimit = None, addLimit = None):
+def getLogFileEntries(runlevels, readLimit = None, addLimit = None, config = None):
   """
   Parses tor's log file for past events matching the given runlevels, providing
   a list of log entries (ordered newest to oldest). Limiting the number of read
@@ -182,11 +182,15 @@
     runlevels - event types (DEBUG - ERR) to be returned
     readLimit - max lines of the log file that'll be read (unlimited if None)
     addLimit  - maximum entries to provide back (unlimited if None)
+    config    - configuration parameters related to this panel, uses defaults
+                if left as None
   """
   
   startTime = time.time()
   if not runlevels: return []
   
+  if not config: config = DEFAULT_CONFIG
+  
   # checks tor's configuration for the log file's location (if any exists)
   loggingTypes, loggingLocation = None, None
   for loggingEntry in torTools.getConn().getOption("Log", [], True):
@@ -236,7 +240,7 @@
       logFile.close()
   except IOError:
     msg = "Unable to read tor's log file: %s" % loggingLocation
-    log.log(DEFAULT_CONFIG["log.logPanel.prepopulateFailed"], msg)
+    log.log(config["log.logPanel.prepopulateFailed"], msg)
   
   if not lines: return []
   
@@ -278,7 +282,7 @@
   
   if addLimit: loggedEvents = loggedEvents[:addLimit]
   msg = "Read %i entries from tor's log file: %s (read limit: %i, runtime: %0.3f)" % (len(loggedEvents), loggingLocation, readLimit, time.time() - startTime)
-  log.log(DEFAULT_CONFIG["log.logPanel.prepopulateSuccess"], msg)
+  log.log(config["log.logPanel.prepopulateSuccess"], msg)
   return loggedEvents
 
 def getDaybreaks(events, ignoreTimeForCache = False):
@@ -496,17 +500,16 @@
   def __init__(self, stdscr, loggedEvents, config=None):
     panel.Panel.__init__(self, stdscr, "log", 0)
     threading.Thread.__init__(self)
+    self.setDaemon(True)
     
     self._config = dict(DEFAULT_CONFIG)
     
     if config:
-      config.update(self._config)
-      
-      # ensures prepopulation and cache sizes are sane
-      self._config["features.log.maxLinesPerEntry"] = max(self._config["features.log.maxLinesPerEntry"], 1)
-      self._config["features.log.prepopulateReadLimit"] = max(self._config["features.log.prepopulateReadLimit"], 0)
-      self._config["features.log.maxRefreshRate"] = max(self._config["features.log.maxRefreshRate"], 10)
-      self._config["cache.logPanel.size"] = max(self._config["cache.logPanel.size"], 50)
+      config.update(self._config, {
+        "features.log.maxLinesPerEntry": 1,
+        "features.log.prepopulateReadLimit": 0,
+        "features.log.maxRefreshRate": 10,
+        "cache.logPanel.size": 50})
     
     # collapses duplicate log entries if false, showing only the most recent
     self.showDuplicates = self._config["features.log.showDuplicateEntries"]
@@ -543,7 +546,7 @@
       setRunlevels = list(set.intersection(set(self.loggedEvents), set(RUNLEVELS)))
       readLimit = self._config["features.log.prepopulateReadLimit"]
       addLimit = self._config["cache.logPanel.size"]
-      torEventBacklog = getLogFileEntries(setRunlevels, readLimit, addLimit)
+      torEventBacklog = getLogFileEntries(setRunlevels, readLimit, addLimit, self._config)
     
     # adds arm listener and fetches past events
     log.LOG_LOCK.acquire()
@@ -838,7 +841,7 @@
           if lineOffset == maxEntriesPerLine: break
           
           maxMsgSize = width - cursorLoc
-          if len(msg) >= maxMsgSize:
+          if len(msg) > maxMsgSize:
             # message is too long - break it up
             if lineOffset == maxEntriesPerLine - 1:
               msg = uiTools.cropStr(msg, maxMsgSize)
@@ -1047,6 +1050,8 @@
     - grown beyond the cache limit
     - outlived the configured log duration
     
+    Argument:
+      eventListing - listing of log entries
     """
     
     cacheSize = self._config["cache.logPanel.size"]

Copied: arm/trunk/src/settings.cfg (from rev 23436, arm/trunk/src/armrc.defaults)
===================================================================
--- arm/trunk/src/settings.cfg	                        (rev 0)
+++ arm/trunk/src/settings.cfg	2010-10-22 05:21:29 UTC (rev 23660)
@@ -0,0 +1,165 @@
+# Snippets from common log messages
+# These are static bits of log messages, used to determine when entries with
+# dynamic content (hostnames, numbers, etc) are the same. If this matches the
+# start of both messages then the entries are flagged as duplicates. If the
+# entry begins with an asterisk (*) then it checks if the substrings exist
+# anywhere in the messages.
+# 
+# Examples for the complete messages:
+# [BW] READ: 0, WRITTEN: 0
+# [DEBUG] connection_handle_write(): After TLS write of 512: 0 read, 586 written
+# [DEBUG] flush_chunk_tls(): flushed 512 bytes, 0 ready to flush, 0 remain.
+# [DEBUG] conn_read_callback(): socket 7 wants to read.
+# [DEBUG] conn_write_callback(): socket 51 wants to write.
+# [DEBUG] connection_remove(): removing socket -1 (type OR), n_conns now 50
+# [DEBUG] connection_or_process_cells_from_inbuf(): 7: starting, inbuf_datalen
+#         0 (0 pending in tls object).
+# [DEBUG] connection_read_to_buf(): 38: starting, inbuf_datalen 0 (0 pending in
+#         tls object). at_most 12800.
+# [DEBUG] connection_read_to_buf(): TLS connection closed on read. Closing.
+#         (Nickname moria1, address 128.31.0.34)
+# [INFO] run_connection_housekeeping(): Expiring non-open OR connection to fd
+#        16 (79.193.61.171:443).
+# [INFO] rep_hist_downrate_old_runs(): Discounting all old stability info by a
+#        factor of 0.950000
+# [NOTICE] We stalled too much while trying to write 150 bytes to address
+#          [scrubbed].  If this happens a lot, either something is wrong with
+#          your network connection, or something is wrong with theirs. (fd 238,
+#          type Directory, state 1, marked at main.c:702).
+# [NOTICE] I learned some more directory information, but not enough to build a
+#          circuit: We have only 469/2027 usable descriptors.
+# [NOTICE] Attempt by %s to open a stream from unknown relay. Closing.
+# [WARN] You specified a server "Amunet8" by name, but this name is not
+#        registered
+# [WARN] I have no descriptor for the router named "Amunet8" in my declared
+#        family; I'll use the nickname as is, but this   may confuse clients.
+# [WARN] Problem bootstrapping. Stuck at 80%: Connecting to the Tor network.
+#        (Network is unreachable; NOROUTE; count 47;    recommendation warn)
+# [WARN] 4 unknown, 1 missing key, 3 good, 0 bad, 1 no signature, 4 required
+# [ARM_DEBUG] refresh rate: 0.001 seconds
+# [ARM_DEBUG] system call: ps -p 2354 -o %cpu,rss,%mem,etime (runtime: 0.02)
+# [ARM_DEBUG] system call: netstat -npt | grep 2354/tor (runtime: 0.02)
+# [ARM_DEBUG] recreating panel 'graph' with the dimensions of 14/124
+# [ARM_DEBUG] redrawing the log panel with the corrected content height (estimat was off by 4)
+# [ARM_DEBUG] GETINFO accounting/bytes-left (runtime: 0.0006)
+
+msg.BW READ:
+msg.DEBUG connection_handle_write(): After TLS write of
+msg.DEBUG flush_chunk_tls(): flushed
+msg.DEBUG conn_read_callback(): socket
+msg.DEBUG conn_write_callback(): socket
+msg.DEBUG connection_remove(): removing socket
+msg.DEBUG connection_or_process_cells_from_inbuf():
+msg.DEBUG *pending in tls object). at_most
+msg.DEBUG connection_read_to_buf(): TLS connection closed on read. Closing.
+msg.INFO run_connection_housekeeping(): Expiring
+msg.INFO rep_hist_downrate_old_runs(): Discounting all old stability info by a factor of
+msg.NOTICE We stalled too much while trying to write
+msg.NOTICE I learned some more directory information, but not enough to build a circuit
+msg.NOTICE Attempt by
+msg.WARN You specified a server
+msg.WARN I have no descriptor for the router named
+msg.WARN Problem bootstrapping. Stuck at
+msg.WARN *missing key,
+msg.ARM_DEBUG refresh rate:
+msg.ARM_DEBUG system call: ps
+msg.ARM_DEBUG system call: netstat
+msg.ARM_DEBUG recreating panel '
+msg.ARM_DEBUG redrawing the log panel with the corrected content height (
+msg.ARM_DEBUG GETINFO accounting/bytes
+msg.ARM_DEBUG GETINFO accounting/bytes-left
+msg.ARM_DEBUG GETINFO accounting/interval-end
+msg.ARM_DEBUG GETINFO accounting/hibernating
+
+# some config options are fetched via special values
+torrc.map HiddenServiceDir => HiddenServiceOptions
+torrc.map HiddenServicePort => HiddenServiceOptions
+torrc.map HiddenServiceVersion => HiddenServiceOptions
+torrc.map HiddenServiceAuthorizeClient => HiddenServiceOptions
+
+# torrc parameters that can be defined multiple times without overwriting
+# from src/or/config.c (entries with LINELIST or LINELIST_S)
+# last updated for tor version 0.2.1.19
+torrc.multiline AlternateBridgeAuthority
+torrc.multiline AlternateDirAuthority
+torrc.multiline AlternateHSAuthority
+torrc.multiline AuthDirBadDir
+torrc.multiline AuthDirBadExit
+torrc.multiline AuthDirInvalid
+torrc.multiline AuthDirReject
+torrc.multiline Bridge
+torrc.multiline ControlListenAddress
+torrc.multiline ControlSocket
+torrc.multiline DirListenAddress
+torrc.multiline DirPolicy
+torrc.multiline DirServer
+torrc.multiline DNSListenAddress
+torrc.multiline ExitPolicy
+torrc.multiline HashedControlPassword
+torrc.multiline HiddenServiceDir
+torrc.multiline HiddenServiceOptions
+torrc.multiline HiddenServicePort
+torrc.multiline HiddenServiceVersion
+torrc.multiline HiddenServiceAuthorizeClient
+torrc.multiline HidServAuth
+torrc.multiline Log
+torrc.multiline MapAddress
+torrc.multiline NatdListenAddress
+torrc.multiline NodeFamily
+torrc.multiline ORListenAddress
+torrc.multiline ReachableAddresses
+torrc.multiline ReachableDirAddresses
+torrc.multiline ReachableORAddresses
+torrc.multiline RecommendedVersions
+torrc.multiline RecommendedClientVersions
+torrc.multiline RecommendedServerVersions
+torrc.multiline SocksListenAddress
+torrc.multiline SocksPolicy
+torrc.multiline TransListenAddress
+torrc.multiline __HashedControlSessionPassword
+
+# valid torrc aliases from the _option_abbrevs struct of src/or/config.c
+# These couldn't be requested via GETCONF (in 0.2.1.19), but I think this has
+# been fixed. Discussion is in:
+# https://trac.torproject.org/projects/tor/ticket/1802
+# 
+# TODO: This workaround should be dropped after a few releases.
+torrc.alias l => Log
+torrc.alias AllowUnverifiedNodes => AllowInvalidNodes
+torrc.alias AutomapHostSuffixes => AutomapHostsSuffixes
+torrc.alias AutomapHostOnResolve => AutomapHostsOnResolve
+torrc.alias BandwidthRateBytes => BandwidthRate
+torrc.alias BandwidthBurstBytes => BandwidthBurst
+torrc.alias DirFetchPostPeriod => StatusFetchPeriod
+torrc.alias MaxConn => ConnLimit
+torrc.alias ORBindAddress => ORListenAddress
+torrc.alias DirBindAddress => DirListenAddress
+torrc.alias SocksBindAddress => SocksListenAddress
+torrc.alias UseHelperNodes => UseEntryGuards
+torrc.alias NumHelperNodes => NumEntryGuards
+torrc.alias UseEntryNodes => UseEntryGuards
+torrc.alias NumEntryNodes => NumEntryGuards
+torrc.alias ResolvConf => ServerDNSResolvConfFile
+torrc.alias SearchDomains => ServerDNSSearchDomains
+torrc.alias ServerDNSAllowBrokenResolvConf => ServerDNSAllowBrokenConfig
+torrc.alias PreferTunnelledDirConns => PreferTunneledDirConns
+torrc.alias BridgeAuthoritativeDirectory => BridgeAuthoritativeDir
+torrc.alias StrictEntryNodes => StrictNodes
+torrc.alias StrictExitNodes => StrictNodes
+
+# using the following entry is problematic, despite being among the
+# __option_abbrevs mappings
+#torrc.alias HashedControlPassword => __HashedControlSessionPassword
+
+# size and time modifiers allowed by config.c
+torrc.label.size.b b, byte, bytes
+torrc.label.size.kb kb, kbyte, kbytes, kilobyte, kilobytes
+torrc.label.size.mb m, mb, mbyte, mbytes, megabyte, megabytes
+torrc.label.size.gb gb, gbyte, gbytes, gigabyte, gigabytes
+torrc.label.size.tb tb, terabyte, terabytes
+torrc.label.time.sec second, seconds
+torrc.label.time.min minute, minutes
+torrc.label.time.hour hour, hours
+torrc.label.time.day day, days
+torrc.label.time.week week, weeks
+

Modified: arm/trunk/src/starter.py
===================================================================
--- arm/trunk/src/starter.py	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/starter.py	2010-10-22 05:21:29 UTC (rev 23660)
@@ -19,6 +19,7 @@
 import util.log
 import util.panel
 import util.sysTools
+import util.torrc
 import util.torTools
 import util.uiTools
 import TorCtl.TorCtl
@@ -114,32 +115,32 @@
       print HELP_MSG
       sys.exit()
   
-  # attempts to load user's custom configuration, using defaults if not found
-  if not os.path.exists(configPath):
-    msg = "No configuration found at '%s', using defaults" % configPath
-    util.log.log(util.log.NOTICE, msg)
-    configPath = "%s/armrc.defaults" % os.path.dirname(sys.argv[0])
-  
   config = util.conf.getConfig("arm")
-  config.path = configPath
   
+  # attempts to fetch attributes for parsing tor's logs, configuration, etc
+  try: config.load("%s/settings.cfg" % os.path.dirname(sys.argv[0]))
+  except IOError, exc:
+    msg = "Failed to load the parsing configuration. This will be problematic for a few things like torrc validation and log duplication detection (%s)" % str(exc)
+    util.log.log(util.log.WARN, msg)
+  
+  # loads user's personal armrc if available
   if os.path.exists(configPath):
     try:
-      config.load()
+      config.load(configPath)
       
       # revises defaults to match user's configuration
       config.update(DEFAULTS)
       
       # loads user preferences for utilities
-      for utilModule in (util.conf, util.connections, util.hostnames, util.log, util.panel, util.sysTools, util.torTools, util.uiTools):
+      for utilModule in (util.conf, util.connections, util.hostnames, util.log, util.panel, util.sysTools, util.torrc, util.torTools, util.uiTools):
         utilModule.loadConfig(config)
     except IOError, exc:
       msg = "Failed to load configuration (using defaults): \"%s\"" % str(exc)
       util.log.log(util.log.WARN, msg)
   else:
-    # no local copy of the armrc defaults, so fall back to values in the source
-    msg = "defaults file not found, falling back (log duplicate detection will be mostly nonfunctional)"
-    util.log.log(util.log.WARN, msg)
+    # no armrc found, falling back to the defaults in the source
+    msg = "No configuration found at '%s', using defaults" % configPath
+    util.log.log(util.log.NOTICE, msg)
   
   # overwrites undefined parameters with defaults
   for key in param.keys():

Modified: arm/trunk/src/util/conf.py
===================================================================
--- arm/trunk/src/util/conf.py	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/util/conf.py	2010-10-22 05:21:29 UTC (rev 23660)
@@ -23,9 +23,6 @@
 CONFIG = {"log.configEntryNotFound": None,
           "log.configEntryTypeError": log.INFO}
 
-# key prefixes that can contain multiple values
-LIST_KEYS = ["msg."]
-
 def loadConfig(config):
   config.update(CONFIG)
 
@@ -42,21 +39,6 @@
   if not handle in CONFS: CONFS[handle] = Config()
   return CONFS[handle]
 
-def isListKey(configKey):
-  """
-  Provides true if the given configuration key can have multiple values (being
-  a list), false otherwise.
-  
-  Arguments:
-    configKey - configuration key to check
-  """
-  
-  for listKeyPrefix in LIST_KEYS:
-    if configKey.startswith(listKeyPrefix):
-      return True
-  
-  return False
-
 class Config():
   """
   Handler for easily working with custom configurations, providing persistence
@@ -73,27 +55,28 @@
     Creates a new configuration instance.
     """
     
-    self.path = None        # path to the associated configuration file
     self.contents = {}      # configuration key/value pairs
     self.contentsLock = threading.RLock()
     self.requestedKeys = set()
     self.rawContents = []   # raw contents read from configuration file
   
-  def getValue(self, key, default=None):
+  def getValue(self, key, default=None, multiple=False):
     """
-    This provides the currently value associated with a given key, and a list
-    of values if isListKey(key) is true. If no such key exists then this
-    provides the default.
+    This provides the currently value associated with a given key. If no such
+    key exists then this provides the default.
     
     Arguments:
-      key     - config setting to be fetched
-      default - value provided if no such key exists
+      key      - config setting to be fetched
+      default  - value provided if no such key exists
+      multiple - provides back a list of all values if true, otherwise this
+                 returns the last loaded configuration value
     """
     
     self.contentsLock.acquire()
     
     if key in self.contents:
       val = self.contents[key]
+      if not multiple: val = val[-1]
       self.requestedKeys.add(key)
     else:
       msg = "config entry '%s' not found, defaulting to '%s'" % (key, str(default))
@@ -104,31 +87,30 @@
     
     return val
   
-  def get(self, key, default=None, minValue=0, maxValue=None):
+  def get(self, key, default=None):
     """
     Fetches the given configuration, using the key and default value to hint
     the type it should be. Recognized types are:
+    - logging runlevel if key starts with "log."
     - boolean if default is a boolean (valid values are 'true' and 'false',
       anything else provides the default)
     - integer or float if default is a number (provides default if fails to
       cast)
-    - logging runlevel if key starts with "log."
-    - list if isListKey(key) is true
+    - list of all defined values default is a list
+    - mapping of all defined values (key/value split via "=>") if the default
+      is a dict
     
     Arguments:
       key      - config setting to be fetched
       default  - value provided if no such key exists
-      minValue - if set and default value is numeric then uses this constraint
-      maxValue - if set and default value is numeric then uses this constraint
     """
     
     callDefault = log.runlevelToStr(default) if key.startswith("log.") else default
-    val = self.getValue(key, callDefault)
+    isMultivalue = isinstance(default, list) or isinstance(default, dict)
+    val = self.getValue(key, callDefault, isMultivalue)
     if val == default: return val
     
-    if isinstance(val, list):
-      pass
-    elif key.startswith("log."):
+    if key.startswith("log."):
       if val.lower() in ("none", "debug", "info", "notice", "warn", "err"):
         val = log.strToRunlevel(val)
       else:
@@ -143,37 +125,53 @@
         log.log(CONFIG["log.configEntryTypeError"], msg)
         val = default
     elif isinstance(default, int):
-      try:
-        val = int(val)
-        if minValue: val = max(val, minValue)
-        if maxValue: val = min(val, maxValue)
+      try: val = int(val)
       except ValueError:
         msg = "config entry '%s' is expected to be an integer, defaulting to '%i'" % (key, default)
         log.log(CONFIG["log.configEntryTypeError"], msg)
         val = default
     elif isinstance(default, float):
-      try:
-        val = float(val)
-        if minValue: val = max(val, minValue)
-        if maxValue: val = min(val, maxValue)
+      try: val = float(val)
       except ValueError:
         msg = "config entry '%s' is expected to be a float, defaulting to '%f'" % (key, default)
         log.log(CONFIG["log.configEntryTypeError"], msg)
         val = default
+    elif isinstance(default, list):
+      pass # nothing special to do (already a list)
+    elif isinstance(default, dict):
+      valMap = {}
+      for entry in val:
+        if "=>" in entry:
+          entryKey, entryVal = entry.split("=>", 1)
+          valMap[entryKey.strip()] = entryVal.strip()
+        else:
+          msg = "ignoring invalid %s config entry (expected a mapping, but \"%s\" was missing \"=>\")" % (key, entry)
+          log.log(CONFIG["log.configEntryTypeError"], msg)
+      val = valMap
     
     return val
   
-  def update(self, confMappings):
+  def update(self, confMappings, limits = {}):
     """
     Revises a set of key/value mappings to reflect the current configuration.
     Undefined values are left with their current values.
     
     Arguments:
       confMappings - configuration key/value mappings to be revised
+      limits       - mappings of limits on numeric values, expected to be of
+                     the form "configKey -> min" or "configKey -> (min, max)"
     """
     
     for entry in confMappings.keys():
-      confMappings[entry] = self.get(entry, confMappings[entry])
+      val = self.get(entry, confMappings[entry])
+      
+      if entry in limits and (isinstance(val, int) or isinstance(val, float)):
+        if isinstance(limits[entry], tuple):
+          val = max(val, limits[entry][0])
+          val = min(val, limits[entry][1])
+        else: val = max(val, limits[entry])
+      
+      confMappings[entry] = val
   
   def getKeys(self):
     """
@@ -211,45 +209,38 @@
     self.contents.clear()
     self.contentsLock.release()
   
-  def load(self):
+  def load(self, path):
     """
-    Reads in the contents of the currently set configuration file (appending
-    any results to the current configuration). If the file's empty or doesn't
-    exist then this doesn't do anything.
+    Reads in the contents of the given path, adding its configuration values
+    and overwriting any that already exist. If the file's empty then this
+    doesn't do anything. Other issues (like having insufficient permissions or
+    if the file doesn't exist) result in an IOError.
     
-    Other issues (like having an unset path or insufficient permissions) result
-    in an IOError.
+    Arguments:
+      path - file path to be loaded
     """
     
-    if not self.path: raise IOError("unable to load (config path undefined)")
+    configFile = open(path, "r")
+    self.rawContents = configFile.readlines()
+    configFile.close()
     
-    if os.path.exists(self.path):
-      configFile = open(self.path, "r")
-      self.rawContents = configFile.readlines()
-      configFile.close()
+    self.contentsLock.acquire()
+    
+    for line in self.rawContents:
+      # strips any commenting or excess whitespace
+      commentStart = line.find("#")
+      if commentStart != -1: line = line[:commentStart]
+      line = line.strip()
       
-      self.contentsLock.acquire()
-      
-      for line in self.rawContents:
-        # strips any commenting or excess whitespace
-        commentStart = line.find("#")
-        if commentStart != -1: line = line[:commentStart]
-        line = line.strip()
+      # parse the key/value pair
+      if line and " " in line:
+        key, value = line.split(" ", 1)
+        value = value.strip()
         
-        # parse the key/value pair
-        if line:
-          key, value = line, ""
-          
-          # gets the key/value pair (no value was given if there isn't a space)
-          if " " in line: key, value = line.split(" ", 1)
-          
-          if isListKey(key):
-            if key in self.contents: self.contents[key].append(value)
-            else: self.contents[key] = [value]
-          else:
-            self.contents[key] = value
-      
-      self.contentsLock.release()
+        if key in self.contents: self.contents[key].append(value)
+        else: self.contents[key] = [value]
+    
+    self.contentsLock.release()
   
   def save(self, saveBackup=True):
     """

Modified: arm/trunk/src/util/hostnames.py
===================================================================
--- arm/trunk/src/util/hostnames.py	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/util/hostnames.py	2010-10-22 05:21:29 UTC (rev 23660)
@@ -47,12 +47,12 @@
           "log.hostnameCacheTrimmed": log.INFO}
 
 def loadConfig(config):
-  config.update(CONFIG)
+  config.update(CONFIG, {
+    "queries.hostnames.poolSize": 1,
+    "cache.hostnames.size": 100,
+    "cache.hostnames.trimSize": 10})
   
-  # ensures sane config values
-  CONFIG["queries.hostnames.poolSize"] = max(1, CONFIG["queries.hostnames.poolSize"])
-  CONFIG["cache.hostnames.size"] = max(100, CONFIG["cache.hostnames.size"])
-  CONFIG["cache.hostnames.trimSize"] = max(10, min(CONFIG["cache.hostnames.trimSize"], CONFIG["cache.hostnames.size"] / 2))
+  CONFIG["cache.hostnames.trimSize"] = min(CONFIG["cache.hostnames.trimSize"], CONFIG["cache.hostnames.size"] / 2)
 
 def start():
   """

Modified: arm/trunk/src/util/log.py
===================================================================
--- arm/trunk/src/util/log.py	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/util/log.py	2010-10-22 05:21:29 UTC (rev 23660)
@@ -28,11 +28,11 @@
           "cache.armLog.trimSize": 200}
 
 def loadConfig(config):
-  config.update(CONFIG)
+  config.update(CONFIG, {
+    "cache.armLog.size": 10,
+    "cache.armLog.trimSize": 5})
   
-  # ensures sane config values
-  CONFIG["cache.armLog.size"] = max(10, CONFIG["cache.armLog.size"])
-  CONFIG["cache.armLog.trimSize"] = max(5, min(CONFIG["cache.armLog.trimSize"], CONFIG["cache.armLog.size"] / 2))
+  CONFIG["cache.armLog.trimSize"] = min(CONFIG["cache.armLog.trimSize"], CONFIG["cache.armLog.size"] / 2)
 
 def strToRunlevel(runlevelStr):
   """

Modified: arm/trunk/src/util/panel.py
===================================================================
--- arm/trunk/src/util/panel.py	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/util/panel.py	2010-10-22 05:21:29 UTC (rev 23660)
@@ -2,7 +2,6 @@
 Wrapper for safely working with curses subwindows.
 """
 
-import sys
 import traceback
 import curses
 from threading import RLock
@@ -224,15 +223,6 @@
         self.win.erase() # clears any old contents
         self.draw(self.win, self.maxX - 1, self.maxY)
       self.win.refresh()
-    except:
-      # without terminating curses continues in a zombie state (requiring a
-      # kill signal to quit, and screwing up the terminal)
-      # TODO: provide a nicer, general purpose handler for unexpected exceptions
-      try:
-        tracebackFile = open("/tmp/armTraceback", "w")
-        traceback.print_exc(file=tracebackFile)
-      finally:
-        sys.exit(1)
     finally:
       CURSES_LOCK.release()
   
@@ -375,6 +365,10 @@
     if top > 0: sliderTop = max(sliderTop, 1)
     if bottom != size: sliderTop = min(sliderTop, scrollbarHeight - sliderSize - 2)
     
+    # avoids a rounding error that causes the scrollbar to be too low when at
+    # the bottom
+    if bottom == size: sliderTop = scrollbarHeight - sliderSize - 1
+    
     # draws scrollbar slider
     for i in range(scrollbarHeight):
       if i >= sliderTop and i <= sliderTop + sliderSize:

Modified: arm/trunk/src/util/torTools.py
===================================================================
--- arm/trunk/src/util/torTools.py	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/util/torTools.py	2010-10-22 05:21:29 UTC (rev 23660)
@@ -319,9 +319,6 @@
     if not suppressExc and raisedExc: raise raisedExc
     else: return result
   
-  # TODO: This could have client side caching if there were events to indicate
-  # SETCONF events. See:
-  # https://trac.torproject.org/projects/tor/ticket/1692
   def getOption(self, param, default = None, multiple = False, suppressExc = True):
     """
     Queries the control port for the given configuration option, providing the
@@ -332,24 +329,66 @@
     Arguments:
       param       - configuration option to be queried
       default     - result if the query fails and exception's suppressed
-      multiple    - provides a list of results if true, otherwise this just
-                    returns the first value
+      multiple    - provides a list with all returned values if true, otherwise
+                    this just provides the first result
       suppressExc - suppresses lookup errors (returning the default) if true,
                     otherwise this raises the original exception
     """
     
+    fetchType = "list" if multiple else "str"
+    return self._getOption(param, default, fetchType, suppressExc)
+  
+  def getOptionMap(self, param, default = None, suppressExc = True):
+    """
+    Queries the control port for the given configuration option, providing back
+    a mapping of config options to a list of the values returned.
+    
+    There's three use cases for GETCONF:
+    - a single value is provided
+    - multiple values are provided for the option queried
+    - a set of options that weren't necessarily requested are returned (for
+      instance querying HiddenServiceOptions gives HiddenServiceDir,
+      HiddenServicePort, etc)
+    
+    The vast majority of the options fall into the first two catagories, in
+    which case calling getOption is sufficient. However, for the special
+    options that give a set of values this provides back the full response. As
+    of tor version 0.2.1.25 HiddenServiceOptions was the only option like this.
+    
+    Arguments:
+      param       - configuration option to be queried
+      default     - result if the query fails and exception's suppressed
+      suppressExc - suppresses lookup errors (returning the default) if true,
+                    otherwise this raises the original exception
+    """
+    
+    return self._getOption(param, default, "map", suppressExc)
+  
+  # TODO: This could have client side caching if there were events to indicate
+  # SETCONF events. See:
+  # https://trac.torproject.org/projects/tor/ticket/1692
+  def _getOption(self, param, default, fetchType, suppressExc):
+    if not fetchType in ("str", "list", "map"):
+      msg = "BUG: unrecognized fetchType in torTools._getOption (%s)" % fetchType
+      log.log(log.ERR, msg)
+      return default
+    
     self.connLock.acquire()
     
-    startTime = time.time()
-    result, raisedExc = [], None
+    startTime, raisedExc = time.time(), None
+    result = {} if fetchType == "map" else []
     if self.isAlive():
       try:
-        if multiple:
-          for key, value in self.conn.get_option(param):
-            if value != None: result.append(value)
-        else:
+        if fetchType == "str":
           getConfVal = self.conn.get_option(param)[0][1]
           if getConfVal != None: result = getConfVal
+        else:
+          for key, value in self.conn.get_option(param):
+            if value != None:
+              if fetchType == "list": result.append(value)
+              elif fetchType == "map":
+                if key in result: result.append(value)
+                else: result[key] = [value]
       except (socket.error, TorCtl.ErrorReply, TorCtl.TorCtlClosed), exc:
         if type(exc) == TorCtl.TorCtlClosed: self.close()
         result, raisedExc = default, exc

Added: arm/trunk/src/util/torrc.py
===================================================================
--- arm/trunk/src/util/torrc.py	                        (rev 0)
+++ arm/trunk/src/util/torrc.py	2010-10-22 05:21:29 UTC (rev 23660)
@@ -0,0 +1,217 @@
+"""
+Helper functions for working with tor's configuration file.
+"""
+
+from util import sysTools, torTools, uiTools
+
+CONFIG = {"torrc.map": {},
+          "torrc.multiline": [],
+          "torrc.alias": {},
+          "torrc.label.size.b": [],
+          "torrc.label.size.kb": [],
+          "torrc.label.size.mb": [],
+          "torrc.label.size.gb": [],
+          "torrc.label.size.tb": [],
+          "torrc.label.time.sec": [],
+          "torrc.label.time.min": [],
+          "torrc.label.time.hour": [],
+          "torrc.label.time.day": [],
+          "torrc.label.time.week": []}
+
+# enums and values for numeric torrc entries
+UNRECOGNIZED, SIZE_VALUE, TIME_VALUE = range(1, 4)
+SIZE_MULT = {"b": 1, "kb": 1024, "mb": 1048576, "gb": 1073741824, "tb": 1099511627776}
+TIME_MULT = {"sec": 1, "min": 60, "hour": 3600, "day": 86400, "week": 604800}
+
+# enums for issues found during torrc validation:
+# VAL_DUPLICATE - entry is ignored due to being a duplicate
+# VAL_MISMATCH  - the value doesn't match tor's current state
+VAL_DUPLICATE, VAL_MISMATCH = range(1, 3)
+
+# cached results for the stripComments function
+STRIP_COMMENTS_ARG, STRIP_COMMENTS_RESULT = None, None
+
+def loadConfig(config):
+  CONFIG["torrc.map"] = config.get("torrc.map", {})
+  CONFIG["torrc.multiline"] = config.get("torrc.multiline", [])
+  CONFIG["torrc.alias"] = config.get("torrc.alias", {})
+  
+  # all the torrc.label.* values are comma separated lists
+  for configKey in CONFIG.keys():
+    if configKey.startswith("torrc.label."):
+      configValues = config.get(configKey, "").split(",")
+      if configValues: CONFIG[configKey] = [val.strip() for val in configValues]
+
+def getConfigLocation():
+  """
+  Provides the location of the torrc, raising an IOError with the reason if the
+  path can't be determined.
+  """
+  
+  conn = torTools.getConn()
+  configLocation = conn.getInfo("config-file")
+  if not configLocation: raise IOError("unable to query the torrc location")
+  
+  # checks if this is a relative path, needing the tor pwd to be appended
+  if configLocation[0] != "/":
+    torPid = conn.getMyPid()
+    failureMsg = "querying tor's pwd failed because %s"
+    if not torPid: raise IOError(failureMsg % "we couldn't get the pid")
+    
+    try:
+      # pwdx results are of the form:
+      # 3799: /home/atagar
+      # 5839: No such process
+      results = sysTools.call("pwdx %s" % torPid)
+      if not results:
+        raise IOError(failureMsg % "pwdx didn't return any results")
+      elif results[0].endswith("No such process"):
+        raise IOError(failureMsg % ("pwdx reported no process for pid " + torPid))
+      elif len(results) != 1 or results.count(" ") != 1:
+        raise IOError(failureMsg % "we got unexpected output from pwdx")
+      else:
+        pwdPath = results[0][results[0].find(" ") + 1:]
+        configLocation = "%s/%s" % (pwdPath, configLocation)
+    except IOError, exc:
+      raise IOError(failureMsg % ("the pwdx call failed: " + str(exc)))
+  
+  return torTools.getPathPrefix() + configLocation
+
+def stripComments(contents):
+  """
+  Provides the torrc contents back with comments and extra whitespace stripped.
+  
+  Arguments:
+    contents - torrc contents
+  """
+  
+  global STRIP_COMMENTS_ARG, STRIP_COMMENTS_RESULT
+  
+  if contents == STRIP_COMMENTS_ARG:
+    return list(STRIP_COMMENTS_RESULT)
+  
+  strippedContents = []
+  for line in contents:
+    # strips off comment
+    if line and "#" in line:
+      line = line[:line.find("#")]
+    
+    strippedContents.append(line.strip())
+  
+  STRIP_COMMENTS_ARG = list(contents)
+  STRIP_COMMENTS_RESULT = list(strippedContents)
+  return strippedContents
+
+def validate(contents):
+  """
+  Performs validation on the given torrc contents, providing back a mapping of
+  line numbers to tuples of the (issue, msg) found on them.
+  
+  Arguments:
+    contents - torrc contents
+  """
+  
+  conn = torTools.getConn()
+  contents = stripComments(contents)
+  issuesFound, seenOptions = {}, []
+  for lineNumber in range(len(contents) - 1, -1, -1):
+    lineText = contents[lineNumber]
+    if not lineText: continue
+    
+    lineComp = lineText.split(None, 1)
+    if len(lineComp) == 2: option, value = lineComp
+    else: option, value = lineText, ""
+    
+    # most parameters are overwritten if defined multiple times
+    if option in seenOptions and not option in CONFIG["torrc.multiline"]:
+      issuesFound[lineNumber] = (VAL_DUPLICATE, "")
+      continue
+    else: seenOptions.append(option)
+    
+    # replace aliases with their recognized representation
+    if option in CONFIG["torrc.alias"]:
+      option = CONFIG["torrc.alias"][option]
+    
+    # tor appears to replace tabs with a space, for instance:
+    # "accept\t*:563" is read back as "accept *:563"
+    value = value.replace("\t", " ")
+    
+    # parse value if it's a size or time, expanding the units
+    value, valueType = _parseConfValue(value)
+    
+    # issues GETCONF to get the values tor's currently configured to use
+    torValues = []
+    if option in CONFIG["torrc.map"]:
+      # special option that's fetched with special values
+      confMappings = conn.getOptionMap(CONFIG["torrc.map"][option], {})
+      if option in confMappings: torValues = confMappings[option]
+    else:
+      torValues = conn.getOption(option, [], True)
+    
+    # multiline entries can be comma separated values (for both tor and conf)
+    valueList = [value]
+    if option in CONFIG["torrc.multiline"]:
+      valueList = [val.strip() for val in value.split(",")]
+      
+      fetchedValues, torValues = torValues, []
+      for fetchedValue in fetchedValues:
+        for fetchedEntry in fetchedValue.split(","):
+          fetchedEntry = fetchedEntry.strip()
+          if not fetchedEntry in torValues:
+            torValues.append(fetchedEntry)
+    
+    for val in valueList:
+      # checks if both the argument and tor's value are empty
+      isBlankMatch = not val and not torValues
+      
+      if not isBlankMatch and not val in torValues:
+        # converts corrections to reader friedly size values
+        displayValues = torValues
+        if valueType == SIZE_VALUE:
+          displayValues = [uiTools.getSizeLabel(int(val)) for val in torValues]
+        elif valueType == TIME_VALUE:
+          displayValues = [uiTools.getTimeLabel(int(val)) for val in torValues]
+        
+        issuesFound[lineNumber] = (VAL_MISMATCH, ", ".join(displayValues))
+  
+  return issuesFound
+
+def _parseConfValue(confArg):
+  """
+  Converts size or time values to their lowest units (bytes or seconds) which
+  is what GETCONF calls provide. The returned is a tuple of the value and unit
+  type.
+  
+  Arguments:
+    confArg - torrc argument
+  """
+  
+  if confArg.count(" ") == 1:
+    val, unit = confArg.lower().split(" ", 1)
+    if not val.isdigit(): return confArg, UNRECOGNIZED
+    mult, multType = _getUnitType(unit)
+    
+    if mult != None:
+      return str(int(val) * mult), multType
+  
+  return confArg, UNRECOGNIZED
+
+def _getUnitType(unit):
+  """
+  Provides the type and multiplier for an argument's unit. The multiplier is
+  None if the unit isn't recognized.
+  
+  Arguments:
+    unit - string representation of a unit
+  """
+  
+  for label in SIZE_MULT:
+    if unit in CONFIG["torrc.label.size." + label]:
+      return SIZE_MULT[label], SIZE_VALUE
+  
+  for label in TIME_MULT:
+    if unit in CONFIG["torrc.label.time." + label]:
+      return TIME_MULT[label], TIME_VALUE
+  
+  return None, UNRECOGNIZED
+

Modified: arm/trunk/src/util/uiTools.py
===================================================================
--- arm/trunk/src/util/uiTools.py	2010-10-21 06:37:31 UTC (rev 23659)
+++ arm/trunk/src/util/uiTools.py	2010-10-22 05:21:29 UTC (rev 23660)
@@ -28,8 +28,8 @@
 SIZE_UNITS_BYTES = [(1125899906842624.0, " PB", " Petabyte"), (1099511627776.0, " TB", " Terabyte"),
                     (1073741824.0, " GB", " Gigabyte"),       (1048576.0, " MB", " Megabyte"),
                     (1024.0, " KB", " Kilobyte"),             (1.0, " B", " Byte")]
-TIME_UNITS = [(86400.0, "d", " day"),                   (3600.0, "h", " hour"),
-              (60.0, "m", " minute"),                   (1.0, "s", " second")]
+TIME_UNITS = [(86400.0, "d", " day"), (3600.0, "h", " hour"),
+              (60.0, "m", " minute"), (1.0, "s", " second")]
 
 END_WITH_ELLIPSE, END_WITH_HYPHEN = range(1, 3)
 SCROLL_KEYS = (curses.KEY_UP, curses.KEY_DOWN, curses.KEY_PPAGE, curses.KEY_NPAGE, curses.KEY_HOME, curses.KEY_END)
@@ -61,7 +61,9 @@
   Provides the msg constrained to the given length, truncating on word breaks.
   If the last words is long this truncates mid-word with an ellipse. If there
   isn't room for even a truncated single word (or one word plus the ellipse if
-  including those) then this provides an empty string. Examples:
+  including those) then this provides an empty string. If a cropped string ends
+  with a comma or period then it's stripped (unless we're providing the
+  remainder back). Examples:
   
   cropStr("This is a looooong message", 17)
   "This is a looo..."
@@ -86,26 +88,28 @@
                    cropped portion of the message
   """
   
-  if minWordLen == None: minWordLen = sys.maxint
-  minWordLen = max(0, minWordLen)
-  minCrop = max(0, minCrop)
-  
   # checks if there's room for the whole message
   if len(msg) <= size:
     if getRemainder: return (msg, "")
     else: return msg
   
+  # avoids negative input
+  size = max(0, size)
+  if minWordLen != None: minWordLen = max(0, minWordLen)
+  minCrop = max(0, minCrop)
+  
   # since we're cropping, the effective space available is less with an
   # ellipse, and cropping words requires an extra space for hyphens
   if endType == END_WITH_ELLIPSE: size -= 3
-  elif endType == END_WITH_HYPHEN: minWordLen += 1
+  elif endType == END_WITH_HYPHEN and minWordLen != None: minWordLen += 1
   
   # checks if there isn't the minimum space needed to include anything
-  if size <= minWordLen:
+  lastWordbreak = msg.rfind(" ", 0, size + 1)
+  if (minWordLen != None and size < minWordLen) or (minWordLen == None and lastWordbreak < 1):
     if getRemainder: return ("", msg)
     else: return ""
   
-  lastWordbreak = msg.rfind(" ", 0, size + 1)
+  if minWordLen == None: minWordLen = sys.maxint
   includeCrop = size - lastWordbreak - 1 >= minWordLen
   
   # if there's a max crop size then make sure we're cropping at least that many characters
@@ -122,7 +126,7 @@
   else: returnMsg, remainder = msg[:lastWordbreak], msg[lastWordbreak:]
   
   # if this is ending with a comma or period then strip it off
-  if returnMsg[-1] in (",", "."): returnMsg = returnMsg[:-1]
+  if not getRemainder and returnMsg[-1] in (",", "."): returnMsg = returnMsg[:-1]
   
   if endType == END_WITH_ELLIPSE: returnMsg += "..."