[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
[or-cvs] r11079: added replication of descriptors, cleaned up the code, last (in tor/branches/114-dist-storage: doc/spec doc/spec/proposals src/common src/or)
Author: kloesing
Date: 2007-08-11 18:30:44 -0400 (Sat, 11 Aug 2007)
New Revision: 11079
Modified:
tor/branches/114-dist-storage/doc/spec/control-spec.txt
tor/branches/114-dist-storage/doc/spec/dir-spec.txt
tor/branches/114-dist-storage/doc/spec/proposals/114-distributed-storage.txt
tor/branches/114-dist-storage/doc/spec/rend-spec.txt
tor/branches/114-dist-storage/src/common/crypto.c
tor/branches/114-dist-storage/src/or/circuitlist.c
tor/branches/114-dist-storage/src/or/circuituse.c
tor/branches/114-dist-storage/src/or/config.c
tor/branches/114-dist-storage/src/or/connection_edge.c
tor/branches/114-dist-storage/src/or/directory.c
tor/branches/114-dist-storage/src/or/dirserv.c
tor/branches/114-dist-storage/src/or/main.c
tor/branches/114-dist-storage/src/or/or.h
tor/branches/114-dist-storage/src/or/rendclient.c
tor/branches/114-dist-storage/src/or/rendcommon.c
tor/branches/114-dist-storage/src/or/rendmid.c
tor/branches/114-dist-storage/src/or/rendservice.c
tor/branches/114-dist-storage/src/or/router.c
tor/branches/114-dist-storage/src/or/routerlist.c
tor/branches/114-dist-storage/src/or/routerparse.c
tor/branches/114-dist-storage/src/or/test.c
Log:
added replication of descriptors, cleaned up the code, last merge with trunk
Modified: tor/branches/114-dist-storage/doc/spec/control-spec.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/control-spec.txt 2007-08-11 15:39:12 UTC (rev 11078)
+++ tor/branches/114-dist-storage/doc/spec/control-spec.txt 2007-08-11 22:30:44 UTC (rev 11079)
@@ -62,18 +62,26 @@
2.3. Replies from Tor to the controller
- Reply = *(MidReplyLine / DataReplyLine) EndReplyLine
+ Reply = SyncReply / AsyncReply
+ SyncReply = *(MidReplyLine / DataReplyLine) EndReplyLine
+ AsyncReply = *(MidReplyLine / DataReplyLine) EndReplyLine
- MidReplyLine = "-" ReplyLine
- DataReplyLine = "+" ReplyLine Data
- EndReplyLine = SP ReplyLine
- ReplyLine = StatusCode [ SP ReplyText ] CRLF
+ MidReplyLine = StatusCode "-" ReplyLine
+ DataReplyLine = StatusCode "+" ReplyLine Data
+ EndReplyLine = StatusCode SP ReplyLine
+ ReplyLine = [ReplyText] CRLF
ReplyText = XXXX
- StatusCode = XXXX
+ StatusCode = 3DIGIT
Specific replies are mentioned below in section 3, and described more fully
in section 4.
+ [Compatibility note: versions of Tor before 0.2.0.3-alpha sometimes
+ generate AsyncReplies of the form "*(MidReplyLine / DataReplyLine)".
+ This is incorrect, but controllers that need to work with these
+ versions of Tor should be prepared to get multi-line AsyncReplies with
+ the final line (usually "650 OK") omitted.]
+
2.4. General-use tokens
; Identifiers for servers.
@@ -217,15 +225,19 @@
"AUTHENTICATE" [ SP 1*HEXDIG / QuotedString ] CRLF
The server responds with "250 OK" on success or "515 Bad authentication" if
- the authentication cookie is incorrect.
+ the authentication cookie is incorrect. Tor closes the connection on an
+ authentication failure.
The format of the 'cookie' is implementation-dependent; see 5.1 below for
information on how the standard Tor implementation handles it.
If Tor requires authentication and the controller has not yet sent an
AUTHENTICATE message, Tor sends a "514 authentication required" reply to
- any other kind of message.
+ any other kind of message, and then closes the connection.
+ (Versions of Tor before 0.1.2.16 and 0.2.0.4-alpha did not close the
+ connection after an authentication failure.)
+
3.6. SAVECONF
Sent from the client to the server. The syntax is:
@@ -377,17 +389,24 @@
space-separated list of LongName, each preceded by a "!" if it is
believed to be not running.)
- "addr-mappings/all"
- "addr-mappings/config"
- "addr-mappings/cache"
- "addr-mappings/control" -- a \r\n-separated list of address
- mappings, each in the form of "from-address to-address".
+ "address-mappings/all"
+ "address-mappings/config"
+ "address-mappings/cache"
+ "address-mappings/control" -- a \r\n-separated list of address
+ mappings, each in the form of "from-address to-address expiry".
The 'config' key returns those address mappings set in the
configuration; the 'cache' key returns the mappings in the
client-side DNS cache; the 'control' key returns the mappings set
via the control interface; the 'all' target returns the mappings
set through any mechanism.
+ Expiry is formatted as with ADDRMAP events, except that "expiry" is
+ always a time in GMT or the string "NEVER"; see section 4.1.7.
+ First introduced in 0.2.0.3-alpha.
+ "addr-mappings/*" -- as for address-mappings/*, but without the
+ expiry portion of the value. Use of this value is deprecated
+ since 0.2.0.3-alpha; use address-mappings instead.
+
"address" -- the best guess at our external IP address. If we
have no guess, return a 551 error. (Added in 0.1.2.2-alpha)
@@ -489,6 +508,12 @@
states. See Section 4.1.10 for explanations. (Only a few of the
status events are available as getinfo's currently. Let us know if
you want more exposed.)
+ "status/version/recommended" -- List of currently recommended versions
+ "status/version/current" -- Status of the current version. One of:
+ new, old, unrecommended, recommended, new in series, obsolete.
+ "status/version/num-versioning" -- Number of versioning authorities
+ "status/version/num-concurring" -- Number of versioning authorities
+ agreeing on the status of the current version
Examples:
C: GETINFO version desc/name/moria1
@@ -904,7 +929,7 @@
If extended events are enabled (see 3.19), optional reason and
circuit counting information is provided for CLOSED and FAILED
- events.
+ events.
Reason = "MISC" / "DONE" / "CONNECTREFUSED" /
"IDENTITY" / "CONNECTRESET" / "TIMEOUT" / "NOROUTE" /
@@ -930,7 +955,7 @@
The syntax is:
"650" SP Severity SP ReplyText
or
- "650+" Severity CRLF Data
+ "650+" Severity CRLF Data 650 SP "OK" CRLF
Severity = "DEBUG" / "INFO" / "NOTICE" / "WARN"/ "ERR"
@@ -942,11 +967,18 @@
4.1.7. New Address mapping
Syntax:
- "650" SP "ADDRMAP" SP Address SP Address SP Expiry
+ "650" SP "ADDRMAP" SP Address SP Address SP Expiry SP Error SP GMTExpiry
Expiry = DQUOTE ISOTime DQUOTE / "NEVER"
- Expiry is expressed as the local time (rather than GMT).
+ Error = / "error=" ErrorCode
+ GMTExpiry = "EXPIRES=" DQUOTE IsoTime DQUOTE
+ Error and GMTExpiry are only provided if extended events are enabled.
+
+ Expiry is expressed as the local time (rather than GMT). This is a bug,
+ left in for backward compatibility; new code should look at GMTExpiry
+ instead.
+
[XXX We should rename this to ADDRESSMAP. -RD]
[Why? Surely it can't be worth the compatibility issues. -NM]
@@ -954,7 +986,7 @@
Syntax:
"650" "+" "AUTHDIR_NEWDESCS" CRLF Action CRLF Message CRLF
- Descriptor CRLF "." CRLF
+ Descriptor CRLF "." CRLF "650" SP "OK" CRLF
Action = "ACCEPTED" / "DROPPED" / "REJECTED"
Message = Text
@@ -1285,7 +1317,7 @@
4.1.12. Network status has changed
Syntax:
- "650" "+" "NS" CRLF 1*NetworkStatus "." CRLF
+ "650" "+" "NS" CRLF 1*NetworkStatus "." CRLF "650" SP "OK" CRLF
[First added in 0.1.2.3-alpha]
@@ -1336,16 +1368,16 @@
5.3. Backward compatibility with v0 control protocol.
- For backward compatibility with the "version 0" control protocol, Tor checks
- whether the third octet the first command is zero. If it is, Tor
- assumes that version 0 is in use. This feature is deprecated, and will be
- removed in the 0.1.3.x Tor development series.
+ The 'version 0' control protocol was replaced in Tor 0.1.1.x. Support was
+ removed in Tor 0.2.0.x. Every non-obsolete version of Tor now supports the
+ version 1 control protocol.
- In order to detect which version of the protocol is supported controllers
- should send the sequence [00 00 0D 0A]. This is a valid and unrecognized
- command in both protocol versions, and implementations can detect which
- error they have received.
+ For backward compatibility with the "version 0" control protocol,
+ Tor used to check whether the third octet of the first command is zero.
+ (If it was, Tor assumed that version 0 is in use.)
+ This compatibility was removed in Tor 0.1.2.16 and 0.2.0.4-alpha.
+
5.4. Options for use by controllers
Tor provides a few special configuration options for use by controllers.
Modified: tor/branches/114-dist-storage/doc/spec/dir-spec.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/dir-spec.txt 2007-08-11 15:39:12 UTC (rev 11078)
+++ tor/branches/114-dist-storage/doc/spec/dir-spec.txt 2007-08-11 22:30:44 UTC (rev 11079)
@@ -297,6 +297,29 @@
The "Digest" of a document, unless stated otherwise, is its digest *as
signed by this signature scheme*.
+1.4. Voting timeline
+
+ Every consensus document has a "valid-after" (VA) time, a "fresh-until"
+ (FU) time and a "valid-until" (VU) time. VA MUST precede FU, which MUST
+ in turn precede VU. Times are chosen so that every consensus will be
+ "fresh" until the next consensus becomes valid, and "valid" for a while
+ after. At least 2 or 3 consensuses should be valid at any given time.
+
+ The timeline for a given consensus is as follows:
+
+ VA-DistSeconds-VoteSeconds: The authorities exchange votes.
+
+ VA-DistSeconds: The authorities calculate the consensus and exchange
+ signatures.
+
+ VA: All authorities have a multiply signed consensus.
+
+ VA ... FU: Caches download the consensus.
+
+ FU: The consensus is no long the freshest consensus.
+
+ VU: The consensus is no longer valid.
+
2. Router operation and formats
ORs SHOULD generate a new router descriptor and a new extra-info
@@ -708,6 +731,14 @@
The status MUST be "vote" or "consensus", depending on the type of
the document.
+ "consensus-methods" SP IntegerList NL
+
+ [Exactly once for votes; does not occur in consensuses.]
+
+ A space-separated list of supported methods for generating
+ consensuses from votes. See section 3.4.1 for details. Method "1"
+ MUST be included.
+
"published" SP YYYY-MM-DD SP HH:MM:SS NL
[Exactly once for votes; does not occur in consensuses.]
@@ -718,25 +749,34 @@
[Exactly once.]
- The start of the Interval for this vote. XXXX
+ The start of the Interval for this vote. Before this time, the
+ consensus document produced from this vote should not be used.
+ See 1.4 for voting timeline information.
"fresh-until" SP YYYY-MM-DD SP HH:MM:SS NL
[Exactly once.]
- XXXX
+ The time at which the next consensus should be produced; before this
+ time, there is no point in downloading another consensus, since there
+ won't be a new one. See 1.4 for voting timeline information.
"valid-until" SP YYYY-MM-DD SP HH:MM:SS NL
[Exactly once.]
- The end of the Interval for this vote. XXXX
+ The end of the Interval for this vote. After this time, the
+ consensus produced by this vote should not be used. See 1.4 for
+ voting timeline information.
"voting-delay" SP VoteSeconds SP DistSeconds NL
[Exactly once.]
- XXXX
+ VoteSeconds is the number of seconds that we will allow to collect
+ votes from all authorities; DistSeconds is the number of seconds
+ we'll allow to collect signatures from all authorities. See 1.4 for
+ voting timeline information.
"client-versions" SP VersionList NL
@@ -911,19 +951,19 @@
it successfully within the last 30 minutes.
"Stable" -- A router is 'Stable' if it is active, and either its
- uptime is at least the median uptime for known active routers, or
+ uptime is at least the median uptime for known active routers or
its uptime is at least 30 days. Routers are never called stable if
they are running a version of Tor known to drop circuits stupidly.
(0.1.1.10-alpha through 0.1.1.16-rc are stupid this way.)
"Fast" -- A router is 'Fast' if it is active, and its bandwidth is
- in the top 7/8ths for known active routers.
+ either in the top 7/8ths for known active routers or at least 100KB/s.
"Guard" -- A router is a possible 'Guard' if it is 'Stable' and its
- bandwidth is above median for known active routers. If the total
- bandwidth of active non-BadExit Exit servers is less than one third
- of the total bandwidth of all active servers, no Exit is listed as
- a Guard.
+ bandwidth is either at least the median for known active routers or
+ at least 250KB/s. If the total bandwidth of active non-BadExit Exit
+ servers is less than one third of the total bandwidth of all active
+ servers, no Exit is listed as a Guard.
"Authority" -- A router is called an 'Authority' if the authority
generating the network-status document believes it is an authority.
@@ -945,7 +985,13 @@
Directory server administrators may label some servers or IPs as
blacklisted, and elect not to include them in their network-status lists.
- Thus, the network-status list includes all non-blacklisted,
+ Authorities SHOULD 'disable' any servers in excess of 3 on any single IP.
+ When there are more than 3 to choose from, authorities should first prefer
+ authorities to non-authorities, then prefer Running to non-Running, and
+ then prefer high-bandwidth to low-bandwidth. To 'disable' a server, the
+ authority *should* advertise it without the Running or Valid flag.
+
+ Thus, the network-status vote includes all non-blacklisted,
non-expired, non-superseded descriptors.
3.4. Computing a consensus from a set of votes
@@ -997,6 +1043,22 @@
The signatures at the end of a consensus document are sorted in
ascending order by identity digest.
+3.4.1. Forward compatibility
+
+ Future versions of Tor will need to include new information in the
+ consensus documents, but it is important that all authorities (or at least
+ half) generate and sign the same signed consensus.
+
+ To achieve this, authorities list in their votes their supported methods
+ for generating consensuses from votes. The method described above and
+ implemented in Tor 0.2.0.x is "1". Later methods will be assigned higher
+ numbers.
+
+ Before generating a consensus, an authority must decide which consensus
+ method to use. To do this, it looks for the highest version number
+ supported by more than 2/3 of the authorities. If it supports this
+ method, then it uses it. Otherwise, it falls back to method 1.
+
3.5. Detached signatures
Assuming full connectivity, every authority should compute and sign the
@@ -1078,17 +1140,15 @@
merged with the second-to-last period.
An authority SHOULD publish its vote immediately at the start of each voting
- period. It does this by making it available at
+ period (minus VoteSeconds+DistSeconds). It does this by making it
+ available at
http://<hostname>/tor/status-vote/current/authority.z
and sending it in an HTTP POST request to each other authority at the URL
http://<hostname>/tor/post/vote
- (Note that this requires the authority to settle upon and finalize its
- vote slightly before the start of the voting period.)
-
- If, VOTING_DELAY minutes after the voting period has begun, an authority
+ If, at the start of the voting period, minus DistSeconds, an authority
does not have a current statement from another authority, the first
- authority retrieves the other's statement.
+ authority downloads the other's statement.
Once an authority has a vote from another authority, it makes it available
at
@@ -1097,9 +1157,15 @@
The consensus status, along with as many signatures as the server
currently knows, should be available at
- http://<hostname>/tor/status-vote/current/consensus.z
+ http://<hostname>/tor/status-vote/next/consensus.z
All of the detached signatures it knows for consensus status should be
available at:
+ http://<hostname>/tor/status-vote/next/consensus-signatures.z
+
+ Once there are enough signatures, or once the voting period starts,
+ these documents are available at
+ http://<hostname>/tor/status-vote/current/consensus.z
+ and
http://<hostname>/tor/status-vote/current/consensus-signatures.z
Once an authority has computed and signed a consensus network status, it
@@ -1112,18 +1178,18 @@
[XXX possible future features include support for downloading old
consensuses.]
- [XXX Constants: VOTING_DELAY, CONSENSUS_DELAY]
-
-
4.3. Downloading consensus status documents (caches only)
- All directory servers (authorities and caches) try to keep a fresh
- set of network-status consensus documents to serve to clients. Every
- 15 minutes, or whenever the valid-until field on its most current
- consensus is about to expire
+ All directory servers (authorities and caches) try to keep a recent
+ network-status consensus document to serve to clients. A cache ALWAYS
+ downloads a network-status consensus if any of the following are true:
+ - The cache has no consensus document.
+ - The cache's consensus document is no longer valid.
+ Otherwise, the cache downloads a new consensus document at a randomly
+ chosen time after its current consensus stops being fresh. (This time is
+ chosen at random to avoid swarming the authorities at the start of each
+ period.)
-[XXXX finish this section]
-
4.4. Downloading and storing router descriptors (authorities and caches)
Periodically (currently, every 10 seconds), directory servers check
@@ -1268,14 +1334,12 @@
until it has a live network-status consensus document, and it has
descriptors for more than 1/4 of the routers that it believes are running.
- [XXXX handling clock skew at client side?]
- [XXXX fall-back to most recent?]
-
(Note: clients can and should pick caches based on the network-status
information they have: once they have first fetched network-status info
from an authority, they should not need to go to the authority directly
again.)
+
5.2. Downloading and storing router descriptors
Clients try to have the best descriptor for each router. A descriptor is
Modified: tor/branches/114-dist-storage/doc/spec/proposals/114-distributed-storage.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/proposals/114-distributed-storage.txt 2007-08-11 15:39:12 UTC (rev 11078)
+++ tor/branches/114-dist-storage/doc/spec/proposals/114-distributed-storage.txt 2007-08-11 22:30:44 UTC (rev 11079)
@@ -1,7 +1,7 @@
Filename: 114-distributed-storage.txt
Title: Distributed Storage for Tor Hidden Service Descriptors
-Version: $Revision: 10773 $
-Last-Modified: $Date: 2007-07-09 16:54:48 +0200 (Mo, 09 Jul 2007) $
+Version: $Revision: 10821 $
+Last-Modified: $Date: 2007-07-12 20:06:13 +0200 (Do, 12 Jul 2007) $
Author: Karsten Loesing
Created: 13-May-2007
Status: Open
@@ -9,17 +9,20 @@
Change history:
13-May-2007 Initial proposal
- 14-May-2007 Added changes suggested by Lasse Overlier
+ 14-May-2007 Added changes suggested by Lasse Øverlier
30-May-2007 Changed descriptor format, key length discussion, typos
09-Jul-2007 Incorporated suggestions by Roger, added status of specification
and implementation for upcoming GSoC mid-term evaluation
+ 11-Aug-2007 Updated implementation statuses, included non-consecutive
+ replication to descriptor format
Overview:
The basic idea of this proposal is to distribute the tasks of storing and
serving hidden service descriptors from currently three authoritative
- directory nodes among a large subset of all onion routers. The two reasons to
- do this are better scalability and improved security properties. Further,
+ directory nodes among a large subset of all onion routers. The three
+ reasons to do this are better robustness (availability), better
+ scalability, and improved security properties. Further,
this proposal suggests changes to the hidden service descriptor format to
prevent new security threats coming from decentralization and to gain even
better security properties.
@@ -41,7 +44,7 @@
increasing number of replicas. These scalability issues have an impact on the
current usage of hidden services and put an even higher burden on the
development of new kinds of applications for hidden services that might
- require storing even bigger numbers of descriptors.
+ require storing even more descriptors.
Second, besides posing a limitation to scalability, storing all hidden
service descriptors on three directory nodes also constitutes a security
@@ -98,7 +101,7 @@
descriptor. It is used to calculate descriptor IDs and to encrypt the
introduction points. This second key can either be given to all clients
together with the hidden service ID, or to a group or a single client as
- authentication token. In the future this second key could be the result of
+ an authentication token. In the future this second key could be the result of
some key agreement protocol between the hidden service and one or more
clients. A new text-based format is proposed for descriptors instead of an
extension of the existing binary format for reasons of future extensibility.
@@ -127,8 +130,13 @@
- routerlist.c: Changed router_get_routerlist() to initialize routing list.
- or.h: Added hs_dirs member to routerlist_t.
- [July 9: Specified and running, though the routing list is compiled for
- each request anew.]
+ - Changed routerlist_free() to free storage held by routing list.
+ - Added UPDATE_HS_DIRS_INTERVAL.
+ - Added update_hs_dir_routing_table().
+ - Changed run_scheduled_events().
+ - Added is_hs_dir member to routerstatus_t.
+
+ [Aug 11: Specified and running.]
/2/ Determine responsible hidden service directory
@@ -143,11 +151,13 @@
- rend-spec.txt, section 1.4: Added description of how to determine the
responsible node(s) for a given descriptor ID.
- - routerlist.c: Added get_responsible_hs_dir() to determine the router that
- is responsible for a given descriptor ID.
- - container.h: Added prototype for smartlist_digest_next_circular().
- - container.c: Added implementation for smartlist_digest_next_circular().
-
+ - routerlist.c: Added get_responsible_hs_dirs() to determine the routers
+ that are responsible for a given descriptor ID.
+
+ - Added is_hs_dir member to routerstatus_t.
+ - Added have_enough_hs_dirs().
+ - Added next_hs_dir().
+
[July 9: Specified and running.]
Hidden service clients and providers:
@@ -219,20 +229,19 @@
- routerparse.c: Added 8 keywords to directory_keyword to parse v2 hidden
service descriptors.
- rendcommon.c: Added rend_cache_store_v2_dir() to allow a hidden service
- directory to store a v2 descriptor in the local cache under its
- descriptor ID instead of its service ID.
- - rendcommon.c: Moved the parsing part from rend_cache_store() to the new
- function rend_cache_store_parse() to reuse it for v2 descriptors.
+ directory to parse a v2 descriptor and store it in the local cache under
+ its descriptor ID instead of its service ID.
- or.h: Added constant REND_DESC_ID_V2_LEN to reflect that v2 descriptor
IDs are longer than v0/1 onion addresses.
- [July 9: Base version specified and running; no checking of published
- descriptors, tunneling over BEGIN_DIR cells not yet implemented.]
+ - Changed directory_handle_command_post().
+
+ [Aug 11: Specified and running.]
/7/ Accept v2 fetch requests
Same as /6/, but with fetch requests for hidden service descriptors.
- (requires /4/)
+ (requires /2/ and /4/)
- rend-spec.txt, section 3.3: Added the processing of v2 fetch requests.
@@ -242,8 +251,9 @@
- or.h: Added constant REND_DESC_ID_V2_LEN to reflect that v2 descriptor
IDs are longer than v0/1 onion addresses.
- [July 9: Base version specified and running; tunneling over BEGIN_DIR
- cells not yet implemented.]
+ - Changed directory_handle_command_get().
+
+ [Aug 11: Specified and running.]
/8/ Replicate descriptors with neighbors
@@ -260,8 +270,19 @@
- rend-spec.txt, section 3.3: Added the replication of v2 descriptors.
- [July 9: To some extend specified, but not yet implemented.]
+ - Added HS_DIR_REPLICATION_INTERVAL.
+ - Added next_hs_dir and previous_hs_dir.
+ - Changed directory_handle_command_get().
+ - Changed run_scheduled_events.
+ - Added hs_dir_perform_replication().
+ - Added rend_cache_lookup_v2_replicas.
+ - Added DIR_PURPOSE_REPLICATE_RENDDESC_V2.
+ - Changed directory_initiate_command.
+ - directory_send_command.
+ - Changed connection_dir_client_reached_eof.
+ [Aug 11: To some extend specified, running.]
+
Authoritative directory nodes:
/9/ Confirm a router's hidden service directory functionality
@@ -285,16 +306,17 @@
"hidden-service-directory" flag in router descriptors.
- routerparse.c: Added 1 keyword to directory_keyword to parse the
"hidden-service-dir" flag in router descriptors.
- - or.h: Added is_hs_dir member to routerinfo_t and to routerstatus_t.
+ - or.h: Added is_hs_dir and wants_to_be_hs_dir members to routerinfo_t.
- dirserv.c: Changed routerstatus_format_entry() to include the "HSDir"
flag in vote and consensus status documents.
- dirserv.c: Changed set_routerstatus_from_routerinfo() to set the "HSDir"
flag.
- [July 9: Base version specified and running in which all nodes that have
- the hidden-service-dir flag set in their router descriptor get the
- HSDir flag, not only those which are running for at least 24 hours.]
+ - Added dirserv_thinks_router_is_hs_dir().
+ - Added MIN_UPTIME_HS_DIR and HS_DIR_REACHABLE_TIMEOUT.
+ [Aug 11: Specified and running.]
+
Hidden service provider:
/10/ Configure v2 hidden service
@@ -338,6 +360,8 @@
service provider uses a freshly generated public key for every
introduction point.
+ - TODO: Change in rend_encode_v2_descriptors.
+
[July 9: Specified, but not yet implemented.]
/12/ Encode v2 descriptors and send v2 publish requests
@@ -351,7 +375,7 @@
the next period. Publication is performed by sending the descriptor to all
hidden service directories that are responsible for keeping replicas for
the descriptor ID. This includes two non-consecutive replicas that are
- stored at 3 consecutive nodes each. (requires /1/ and /3/)
+ stored at 3 consecutive nodes each. (requires /1/, /2/, and /3/)
- rend-spec.txt, section 1.2: Added the new v2 hidden service descriptor
format.
@@ -364,25 +388,20 @@
- rendservice.c: Changed rend_consider_services_upload() to also initiate
the upload of v2 descriptors, if configured.
- rendservice.c: Extended rend_service_t by a member secret_cookie.
- - rendcommon.c: Added rend_compute_v2_descriptor_fields() to prepare the
- encoding of a v2 descriptor.
- rendcommon.c: Added rend_encode_v2_descriptor() to encode a v2
descriptor.
- - or.h: Added 7 new members to rend_service_descriptor_t to store
- v2-specific information.
- or.h: Added constant DIR_PURPOSE_UPLOAD_RENDDESC_V2.
- directory.c: Added directory_post_to_hs_dir().
- directory.c: Changed directory_initiate_command() to also recognize v2
publish requests.
- directory.c: Changed directory_send_command() to also prepare v2 publish
requests.
- - directory.c: Changed directory_handle_command_post() to handle v2 publish
- requests.
- crypto.c: Added implementation for crypto_cipher_encrypt_cbc().
- [July 9: Base version specified and running; yet, replication is not
- implemented, republication does not depend on publication periods, yet.]
+ - Changed connection_dir_client_reached_eof().
+ [Aug 11: Specified and running.]
+
Hidden service client:
/13/ Send v2 fetch requests
@@ -406,10 +425,10 @@
- rendcommon.c: Changed rend_cache_lookup_entry to enable it to also lookup
v2 descriptors.
- - rendcommon.c: Added rend_compute_desc_id() to generate v2 descriptor IDs
+ - rendcommon.c: Added rend_compute_v2_desc_id() to generate v2 descriptor IDs
from v2 onion addresses.
- rendcommon.c: Changed rend_valid_service_id() to also consider v2 onion
- addresses as valid and return the version number of the request (1 or 2).
+ addresses as valid and return the version number of the request (0 or 2).
- rendclient.c: Added rend_client_refetch_v2_renddesc() to fetch v2 service
descriptors using the secret cookie.
- rendclient.c: Changed rend_client_remove_intro_point() to copy the secret
@@ -424,16 +443,14 @@
fetch requests.
- directory.c: Changed directory_send_command() to also prepare v2 fetch
requests.
- - directory.c: Changed directory_handle_command_get() to handle v2 fetch
- requests.
- connection_edge.c: Changed connection_ap_handshake_rewrite_and_attach()
to fetch v2 service descriptors.
- connection_edge.c: Changed parse_extended_hostname() to accept both,
current and v2 onion addresses.
- config.c: Added config options FetchV2HidServDescriptors.
- [July 9: Base version specified and running in which only one node is
- responsible for a specific descriptor ID.]
+ [Aug 11: Base version specified and running, but no memory of failed
+ hidden service directories, yet.]
/14/ Process v2 fetch reply and parse v2 descriptors
@@ -453,15 +470,14 @@
introduction points of v2 hidden service descriptors.
- routerparse.c: Added desc_token_table[] to parse v2 hidden service
descriptors.
- - routerparse.c: Added 8 to directory_keyword to parse v2 hidden service
- descriptors, and 5 to parse the decrypted list of introduction points.
+ - routerparse.c: Added 8 keywords to directory_keyword to parse v2 hidden
+ service descriptors, and 5 to parse the decrypted list of introduction
+ points.
- rendcommon.c: Added rend_cache_store_v2_client() to parse a v2 descriptor
and parse the encrypted list of introduction points.
- - or.h: Added secret_cookie to edge_connection_t, to dir_connection_t, and
- to origin_circuit_t to be able to decrypt introduction points when
- receiving a v2 descriptor.
- - or.h: Added 7 new members to rend_service_descriptor_t to store
- v2-specific information.
+ - or.h: Added rend_version and secret_cookie to edge_connection_t, to
+ dir_connection_t, and to origin_circuit_t to be able to decrypt
+ introduction points when receiving a v2 descriptor.
- directory.c: Changed connection_dir_client_reached_eof() to also parse v2
fetch replies.
- crypto.c: Added implementation for crypto_cipher_decrypt_cbc().
@@ -491,8 +507,6 @@
- or.h: Added secret_cookie to edge_connection_t, to dir_connection_t, and
to origin_circuit_t to be able to decrypt introduction points when
receiving a v2 descriptor.
- - or.h: Added 7 new members to rend_service_descriptor_t to store
- v2-specific information.
- circuitlist.c: Changed _circuit_mark_for_close() to pass the secret
cookie to rend_client_remove_intro_point() when an intro circ has failed.
- circuituse.c: Changed circuit_get_open_circ_or_launch() to fetch a v2
@@ -509,12 +523,12 @@
The new v2 hidden service descriptor format looks like this:
onion-address = h(public-key) + cookie
- descriptor-id = h(h(public-key) + h(time-period + cookie))
+ descriptor-id = h(h(public-key) + h(time-period + cookie + relica))
descriptor-content = {
descriptor-id,
version,
public-key,
- h(time-period + cookie),
+ h(time-period + cookie + replica),
timestamp,
protocol-versions,
{ introduction-points } encrypted with cookie
@@ -530,13 +544,14 @@
Therefore, "descriptor-id" is derived from the "public-key" of the hidden
service provider, the current "time-period" which changes every 24 hours,
- and a secret "cookie" shared between hidden service provider and clients.
- (The "time-period" is constructed in a way that time periods do not change
- at the same moment for all descriptors by deriving a value between 0:00 and
- 23:59 hours from "public-key" and making the descriptors of this hidden
+ a secret "cookie" shared between hidden service provider and clients, and
+ a "replica" denoting the number of this non-consecutive replica. (The
+ "time-period" is constructed in a way that time periods do not change at
+ the same moment for all descriptors by deriving a value between 0:00 and
+ 23:59 hours from h(public-key) and making the descriptors of this hidden
service provider expire at that time of the day.) The "descriptor-id" is
defined to be 160 bits long. [extending the "descriptor-id" length
- suggested by LO]
+ suggested by LØ]
Only the hidden service provider and the clients are able to generate
future "descriptor-ID"s. Hence, the "onion-address" is extended from now
@@ -555,7 +570,7 @@
The "introduction-points" that are included in the descriptor are encrypted
using the same "cookie" that is shared between hidden service provider and
clients. [correction to use another key than h(time-period + cookie) as
- encryption key for introduction points made by LO]
+ encryption key for introduction points made by LØ]
A new text-based format is proposed for descriptors instead of an extension
of the existing binary format for reasons of future extensibility.
@@ -567,7 +582,7 @@
Attacks by authoritative directory nodes
- Authoritative directory nodes are not anymore the single places in the
+ Authoritative directory nodes are no longer the single places in the
network that know about a hidden service's activity and introduction
points. Thus, they cannot perform attacks using this information, e.g.
track a hidden service's activity or usage pattern or attack its
@@ -584,7 +599,8 @@
to have a good probability to become responsible for its changing
descriptor IDs. For each period, the probability is:
- 1-(N-c choose r)/(N choose r) for N-c>=r and 1 else with N as total
+ 1-(N-c choose r)/(N choose r) for N-c>=r and 1 otherwise, with N
+ as total
number of hidden service directories, c as compromised nodes, and r as
number of replicas
@@ -595,11 +611,12 @@
content. The client would detect a false descriptor, because it could not
contain a correct signature. But an old content or an empty reply could
confuse the client. Therefore, the countermeasure is to replicate
- descriptors among a small number of hidden service directories. The
- probability of a group of collaborating nodes to make a hidden service
+ descriptors among a small number of hidden service directories, e.g. 5.
+ The probability of a group of collaborating nodes to make a hidden service
completely unavailable is in each period:
- (c choose r)/(N choose r) for c>=r and N>=r, and 0 else with N as total
+ (c choose r)/(N choose r) for c>=r and N>=r, and 0 otherwise,
+ with N as total
number of hidden service directories, c as compromised nodes, and r as
number of replicas
@@ -937,4 +954,10 @@
Added rend_decrypt_introduction_points() to decrypt and parse the list of
introduction points (/14/).
-
\ No newline at end of file
+Test:
+
+ The changes were tested via test functions in test.c for separate,
+ short-running functionality and using an automatic validation based on
+ PuppeTor.
+
+
\ No newline at end of file
Modified: tor/branches/114-dist-storage/doc/spec/rend-spec.txt
===================================================================
--- tor/branches/114-dist-storage/doc/spec/rend-spec.txt 2007-08-11 15:39:12 UTC (rev 11078)
+++ tor/branches/114-dist-storage/doc/spec/rend-spec.txt 2007-08-11 22:30:44 UTC (rev 11079)
@@ -138,9 +138,12 @@
The first time the OP provides an advertised service, it generates
a public/private keypair (stored locally). Periodically, the OP
- generates a pair of service descriptors, one "V1" and one "V0".
+ generates and publishes a descriptor of type "V0". The V1 descriptor
+ format in 0.1.1.5-alpha-cvs is understood and accepted, but currently
+ no Tors generate them. The more complex V1 descriptor format below
+ is just speculation and has never been used.
- The "V1" descriptor in 0.1.1.6-alpha contains:
+ A hypothetical "V1" descriptor contains:
V Format byte: set to 255 [1 octet]
V Version byte: set to 1 [1 octet]
@@ -195,17 +198,9 @@
KL is the length of PK, in octets.
TS is the number of seconds elapsed since Jan 1, 1970.
- AUTHT specifies which authentication/authorization mechanism is
- required by the hidden service or the introduction point. AUTHD
- is arbitrary data that can be associated with an auth approach.
- Currently only AUTHT of [00 00] is supported, with an AUTHL of 0.
- See section 2 of this document for details on auth mechanisms.
+ The members of Ipt are identity key digests, encoded in hex, and
+ prefixed with a '$'.
- The members of Ipt may be either (a) nicknames, or (b) identity key
- digests, encoded in hex, and prefixed with a '$'. Clients must
- accept both forms. Services must only generate the second form.
- Once 0.0.9.x is obsoleted, we can drop the first form.
-
[It's ok for Bob to advertise 0 introduction points. He might want
to do that if he previously advertised some introduction points,
and now he doesn't have any. -RD]
@@ -300,6 +295,16 @@
A bitmask of allowed rendezvous protocols.
+ "authentication" ...
+
+
+[ AUTHT specifies which authentication/authorization mechanism is ]
+[ required by the hidden service or the introduction point. AUTHD ]
+[ is arbitrary data that can be associated with an auth approach. ]
+[ Currently only AUTHT of [00 00] is supported, with an AUTHL of 0. ]
+[ See section 2 of this document for details on auth mechanisms. ]
+
+
"introduction-points" NL encrypted-string
[Exactly once]
@@ -344,6 +349,9 @@
The public key that can be used to encrypt messages to the hidden
service.
+ "authentication" ...
+
+
"signature" NL signature-string
[At end, exactly once]
@@ -420,7 +428,7 @@
all hidden service directories. Therefore, Bob's OP opens a stream via Tor
to all responsible hidden service directories. (He may re-use old circuits
for this.) Over this stream, Bob's OP makes an HTTP 'POST' request to a URL
- "/tor/rendezvous/publish2" relative to the hidden service directory's root,
+ "/tor/rendezvous2/publish" relative to the hidden service directory's root,
containing as its body Bob's service descriptor.
At any time, there are 6 hidden service directories responsible for keeping
@@ -703,7 +711,7 @@
running for at least 24 hours.) All requests and replies are formatted as
HTTP messages. Requests are contained within BEGIN_DIR cells, directed to
the router's directory port, and formatted as HTTP POST requests to the URL
- "/tor/rendezvous/publish2" relative to the hidden service directory's root,
+ "/tor/rendezvous2/publish" relative to the hidden service directory's root,
containing as its body a v2 service descriptor.
A hidden service directory node parses every received descriptor and only
Modified: tor/branches/114-dist-storage/src/common/crypto.c
===================================================================
--- tor/branches/114-dist-storage/src/common/crypto.c 2007-08-11 15:39:12 UTC (rev 11078)
+++ tor/branches/114-dist-storage/src/common/crypto.c 2007-08-11 22:30:44 UTC (rev 11079)
@@ -1203,7 +1203,7 @@
* <b>key</b> of 16 bytes length to <b>to</b>. The length of <b>to</b> may be
* the length of <b>from</b> minus 16 (up to 16 bytes for padding and exactly
* 16 bytes for the initialization vector). On success, return the number of
- * bytes written, on failure, return -1.
+ * bytes written, on failure (including providing the wrong key), return -1.
*/
int
crypto_cipher_decrypt_cbc(char *key, char *to, const char *from,
@@ -1233,13 +1233,13 @@
if (!EVP_DecryptUpdate(&ctx, (unsigned char *)to, &outlen,
((const unsigned char *)from) + 16,
(int)fromlen - 16)) {
- crypto_log_errors(LOG_WARN, "decrypting");
+ crypto_log_errors(LOG_INFO, "decrypting");
return -1;
}
/* decrypt the final data */
if (!EVP_DecryptFinal_ex(&ctx, ((unsigned char *)to) + outlen, &tmplen)) {
- crypto_log_errors(LOG_WARN, "decrypting the final data");
+ crypto_log_errors(LOG_INFO, "decrypting the final data");
return -1;
}
outlen += tmplen;
@@ -1925,7 +1925,7 @@
tor_assert((nbits/8) <= destlen); /* We need enough space. */
tor_assert(destlen < SIZE_T_CEILING);
- /* convert base32 encoded chars to the 5-bit values that they represent */
+ /* Convert base32 encoded chars to the 5-bit values that they represent. */
tmp = tor_malloc_zero(srclen);
for (j = 0; j < srclen; ++j) {
if (src[j] > 0x60 && src[j] < 0x7B) tmp[j] = src[j] - 0x61;
@@ -1936,7 +1936,7 @@
}
}
- /* assemble result byte-wise by applying the five possible cases */
+ /* Assemble result byte-wise by applying five possible cases. */
for (i = 0, bit = 0; bit < nbits; ++i, bit += 8) {
switch (bit % 40) {
case 0:
Modified: tor/branches/114-dist-storage/src/or/circuitlist.c
===================================================================
--- tor/branches/114-dist-storage/src/or/circuitlist.c 2007-08-11 15:39:12 UTC (rev 11078)
+++ tor/branches/114-dist-storage/src/or/circuitlist.c 2007-08-11 22:30:44 UTC (rev 11079)
@@ -12,9 +12,8 @@
**/
#include "or.h"
+#include "ht.h"
-#include "../common/ht.h"
-
/********* START VARIABLES **********/
/** A global list of all circuits at this hop. */
@@ -904,22 +903,6 @@
}
}
-/** Return 1 if there are any origin circuits that use
- * <b>conn</b> as there first hop. Else return 0. */
-static int
-circuit_any_origin_circs_on_conn(or_connection_t *conn)
-{
- circuit_t *circ;
-
- for (circ=global_circuitlist; circ; circ = circ->next) {
- if (CIRCUIT_IS_ORIGIN(circ) &&
- !circ->marked_for_close &&
- circ->n_conn == conn)
- return 1;
- }
- return 0;
-}
-
/** Mark <b>circ</b> to be closed next time we call
* circuit_close_all_marked(). Do any cleanup needed:
* - If state is onionskin_pending, remove circ from the onion_pending
@@ -958,9 +941,9 @@
file, line, circ->purpose);
}
reason = END_CIRC_REASON_NONE;
- } else if (CIRCUIT_IS_ORIGIN(circ) && reason < _END_CIRC_REASON_MIN) {
- /* We don't send reasons when closing circuits at the origin, but we want
- * to track them anyway so we can give them to the controller. */
+ }
+ if (CIRCUIT_IS_ORIGIN(circ)) {
+ /* We don't send reasons when closing circuits at the origin. */
reason = END_CIRC_REASON_NONE;
}
@@ -1045,12 +1028,7 @@
circ->marked_for_close = line;
circ->marked_for_close_file = file;
- if (CIRCUIT_IS_ORIGIN(circ)) {
- if (circ->n_conn && circ->n_conn->client_used) {
- circ->n_conn->client_used =
- circuit_any_origin_circs_on_conn(circ->n_conn);
- }
- } else {
+ if (!CIRCUIT_IS_ORIGIN(circ)) {
or_circuit_t *or_circ = TO_OR_CIRCUIT(circ);
if (or_circ->rend_splice) {
if (!or_circ->rend_splice->_base.marked_for_close) {
Modified: tor/branches/114-dist-storage/src/or/circuituse.c
===================================================================
--- tor/branches/114-dist-storage/src/or/circuituse.c 2007-08-11 15:39:12 UTC (rev 11078)
+++ tor/branches/114-dist-storage/src/or/circuituse.c 2007-08-11 22:30:44 UTC (rev 11079)
@@ -813,8 +813,9 @@
return circ;
}
-/** Launch a new circuit with purpose <b>purpose</b> and exit node <b>info</b>
- * (or NULL to select a random exit node). If <b>need_uptime</b> is true,
+/** Launch a new circuit with purpose <b>purpose</b> and exit node
+ * <b>extend_info</b> (or NULL to select a random exit node).
+ * If <b>need_uptime</b> is true,
* choose among routers with high uptime. If <b>need_capacity</b> is true,
* choose among routers with high bandwidth. If <b>internal</b> is true, the
* last hop need not be an exit node. Return the newly allocated circuit on
@@ -942,6 +943,7 @@
int check_exit_policy;
int need_uptime, need_internal;
int want_onehop;
+ or_options_t *options = get_options();
tor_assert(conn);
tor_assert(circp);
@@ -952,7 +954,7 @@
want_onehop = conn->socks_request->command == SOCKS_COMMAND_CONNECT_DIR;
need_uptime = (conn->socks_request->command == SOCKS_COMMAND_CONNECT) &&
- smartlist_string_num_isin(get_options()->LongLivedPorts,
+ smartlist_string_num_isin(options->LongLivedPorts,
conn->socks_request->port);
need_internal = desired_circuit_purpose != CIRCUIT_PURPOSE_C_GENERAL;
@@ -966,10 +968,17 @@
if (!want_onehop && !router_have_minimum_dir_info()) {
if (!connection_get_by_type(CONN_TYPE_DIR)) {
- log_notice(LD_APP|LD_DIR,
- "Application request when we're believed to be "
- "offline. Optimistically trying directory fetches again.");
- routerlist_retry_directory_downloads(time(NULL));
+ if (options->UseBridges && bridges_should_be_retried()) {
+ log_notice(LD_APP|LD_DIR,
+ "Application request when we're believed to be "
+ "offline. Optimistically trying known bridges again.");
+ bridges_retry_all();
+ } else if (!options->UseBridges || any_bridge_descriptors_known()) {
+ log_notice(LD_APP|LD_DIR,
+ "Application request when we're believed to be "
+ "offline. Optimistically trying directory fetches again.");
+ routerlist_retry_directory_downloads(time(NULL));
+ }
}
/* the stream will be dealt with when router_have_minimum_dir_info becomes
* 1, or when all directory attempts fail and directory_all_unreachable()
@@ -1003,7 +1012,7 @@
if (desired_circuit_purpose == CIRCUIT_PURPOSE_C_INTRODUCE_ACK_WAIT) {
/* need to pick an intro point */
- extend_info = rend_client_get_random_intro(conn->rend_query);
+ extend_info = rend_client_get_random_intro(conn->rend_query, conn->rend_version);
if (!extend_info) {
log_info(LD_REND,
"No intro points for '%s': refetching service descriptor.",
Modified: tor/branches/114-dist-storage/src/or/config.c
===================================================================
--- tor/branches/114-dist-storage/src/or/config.c 2007-08-11 15:39:12 UTC (rev 11078)
+++ tor/branches/114-dist-storage/src/or/config.c 2007-08-11 22:30:44 UTC (rev 11079)
@@ -17,7 +17,6 @@
#ifdef MS_WINDOWS
#include <shlobj.h>
#endif
-#include "../common/aes.h"
/** Enumeration of types which option values can take */
typedef enum config_type_t {
@@ -146,6 +145,8 @@
VAR("CircuitIdleTimeout", INTERVAL, CircuitIdleTimeout, "1 hour"),
VAR("ClientOnly", BOOL, ClientOnly, "0"),
VAR("ConnLimit", UINT, ConnLimit, "1000"),
+ VAR("ConstrainedSockets", BOOL, ConstrainedSockets, "0"),
+ VAR("ConstrainedSockSize", MEMUNIT, ConstrainedSockSize, "8192"),
VAR("ContactInfo", STRING, ContactInfo, NULL),
VAR("ControlListenAddress",LINELIST, ControlListenAddress, NULL),
VAR("ControlPort", UINT, ControlPort, "0"),
@@ -333,6 +334,11 @@
{ "BandwidthBurst", "Limit the maximum token buffer size (also known as "
"burst) to the given number of bytes." },
{ "ConnLimit", "Maximum number of simultaneous sockets allowed." },
+ { "ConstrainedSockets", "Shrink tx and rx buffers for sockets to avoid "
+ "system limits on vservers and related environments. See man page for "
+ "more information regarding this option." },
+ { "ConstrainedSockSize", "Limit socket buffers to this size when "
+ "ConstrainedSockets is enabled." },
/* ControlListenAddress */
{ "ControlPort", "If set, Tor will accept connections from the same machine "
"(localhost only) on this port, and allow those connections to control "
@@ -711,21 +717,22 @@
extern const char tor_svn_revision[]; /* from tor_main.c */
+static char *_version = NULL;
+
/** Return the current Tor version, possibly */
const char *
get_version(void)
{
- static char *version = NULL;
- if (version == NULL) {
+ if (_version == NULL) {
if (strlen(tor_svn_revision)) {
size_t len = strlen(VERSION)+strlen(tor_svn_revision)+8;
- version = tor_malloc(len);
- tor_snprintf(version, len, "%s (r%s)", VERSION, tor_svn_revision);
+ _version = tor_malloc(len);
+ tor_snprintf(_version, len, "%s (r%s)", VERSION, tor_svn_revision);
} else {
- version = tor_strdup(VERSION);
+ _version = tor_strdup(VERSION);
}
}
- return version;
+ return _version;
}
/** Release all memory and resources held by global configuration structures.
@@ -742,6 +749,7 @@
global_state = NULL;
}
tor_free(torrc_fname);
+ tor_free(_version);
}
/** If options->SafeLogging is on, return a not very useful string,
@@ -750,6 +758,10 @@
const char *
safe_str(const char *address)
{
+ if (!address) { /* XXX020 eventually turn this into an assert */
+ log_warn(LD_BUG, "safe_str() called with NULL address.");
+ return "EMPTY";
+ }
if (get_options()->SafeLogging)
return "[scrubbed]";
else
@@ -2373,6 +2385,8 @@
{
tor_assert(auth);
*auth = NO_AUTHORITY;
+ if (!list) /* empty list, answer is none */
+ return 0;
SMARTLIST_FOREACH(list, const char *, string, {
if (!strcasecmp(string, "v1"))
*auth |= V1_AUTHORITY;
@@ -2448,7 +2462,7 @@
REJECT("DirPort must be defined if DirListenAddress is defined.");
if (options->DNSPort == 0 && options->DNSListenAddress != NULL)
- REJECT("DirPort must be defined if DirListenAddress is defined.");
+ REJECT("DNSPort must be defined if DNSListenAddress is defined.");
if (options->ControlPort == 0 && options->ControlListenAddress != NULL)
REJECT("ControlPort must be defined if ControlListenAddress is defined.");
@@ -2622,7 +2636,7 @@
"extra-info documents. Setting DownloadExtraInfo.");
options->DownloadExtraInfo = 1;
}
- /* XXXX020 Check that at least one of Bridge/HS/V1/V2/V2{AoritativeDir}
+ /* XXXX020 Check that at least one of Bridge/HS/V1/V2/V2{AuthoritativeDir}
* is set. */
}
@@ -2872,6 +2886,37 @@
if (options->HashedControlPassword && options->CookieAuthentication)
REJECT("Cannot set both HashedControlPassword and CookieAuthentication");
+ if (options->ControlListenAddress) {
+ int all_are_local = 1;
+ config_line_t *ln;
+ for (ln = options->ControlListenAddress; ln; ln = ln->next) {
+ if (strcmpstart(ln->value, "127."))
+ all_are_local = 0;
+ }
+ if (!all_are_local) {
+ if (!options->HashedControlPassword && !options->CookieAuthentication) {
+ log_warn(LD_CONFIG, "You have a ControlListenAddress set to accept "
+ "connections from a non-local address. This means that "
+ "any program on the internet can reconfigure your Tor. "
+ "That's so bad that I'm closing your ControlPort for you.");
+ options->ControlPort = 0;
+ } else {
+ log_warn(LD_CONFIG, "You have a ControlListenAddress set to accept "
+ "connections from a non-local address. This means that "
+ "programs not running on your computer can reconfigure your "
+ "Tor. That's pretty bad!");
+ }
+ }
+ }
+
+ if (options->ControlPort && !options->HashedControlPassword &&
+ !options->CookieAuthentication) {
+ log_warn(LD_CONFIG, "ControlPort is open, but no authentication method "
+ "has been configured. This means that any program on your "
+ "computer can reconfigure your Tor. That's bad! You should "
+ "upgrade your Tor controller as soon as possible.");
+ }
+
if (options->UseEntryGuards && ! options->NumEntryGuards)
REJECT("Cannot enable UseEntryGuards with NumEntryGuards set to 0");
@@ -2928,6 +2973,29 @@
}
}
+ if (options->ConstrainedSockets) {
+ /* If the user wants to constrain socket buffer use, make sure the desired
+ * limit is between MIN|MAX_TCPSOCK_BUFFER in k increments. */
+ if (options->ConstrainedSockSize < MIN_CONSTRAINED_TCP_BUFFER ||
+ options->ConstrainedSockSize > MAX_CONSTRAINED_TCP_BUFFER ||
+ options->ConstrainedSockSize % 1024) {
+ r = tor_snprintf(buf, sizeof(buf),
+ "ConstrainedSockSize is invalid. Must be a value between %d and %d "
+ "in 1024 byte increments.",
+ MIN_CONSTRAINED_TCP_BUFFER, MAX_CONSTRAINED_TCP_BUFFER);
+ *msg = tor_strdup(r >= 0 ? buf : "internal error");
+ return -1;
+ }
+ if (options->DirPort) {
+ /* Providing cached directory entries while system TCP buffers are scarce
+ * will exacerbate the socket errors. Suggest that this be disabled. */
+ COMPLAIN("You have requested constrained socket buffers while also "
+ "serving directory entries via DirPort. It is strongly "
+ "suggested that you disable serving directory requests when "
+ "system TCP buffer resources are scarce.");
+ }
+ }
+
if (rend_config_services(options, 1) < 0)
REJECT("Failed to configure rendezvous options. See logs for details.");
@@ -3533,8 +3601,8 @@
*msg = tor_strdup("Wrong number of elements in RedirectExit line");
goto err;
}
- if (parse_addr_and_port_range(smartlist_get(elements,0),&r->addr,&r->mask,
- &r->port_min,&r->port_max)) {
+ if (parse_addr_and_port_range(smartlist_get(elements,0),&r->addr,
+ &r->maskbits,&r->port_min,&r->port_max)) {
*msg = tor_strdup("Error parsing source address in RedirectExit line");
goto err;
}
@@ -4490,8 +4558,9 @@
return 0;
}
-#include "../common/ht.h"
-#include "../common/test.h"
+#include "aes.h"
+#include "ht.h"
+#include "test.h"
extern const char aes_c_id[];
extern const char compat_c_id[];
Modified: tor/branches/114-dist-storage/src/or/connection_edge.c
===================================================================
--- tor/branches/114-dist-storage/src/or/connection_edge.c 2007-08-11 15:39:12 UTC (rev 11078)
+++ tor/branches/114-dist-storage/src/or/connection_edge.c 2007-08-11 22:30:44 UTC (rev 11079)
@@ -59,7 +59,7 @@
else if (SOCKS_COMMAND_IS_RESOLVE(conn->socks_request->command))
connection_ap_handshake_socks_resolved(conn,
RESOLVED_TYPE_ERROR_TRANSIENT,
- 0, NULL, -1);
+ 0, NULL, -1, -1);
else /* unknown or no handshake at all. send no response. */
conn->socks_request->has_finished = 1;
}
@@ -617,14 +617,14 @@
void
addressmap_clear_configured(void)
{
- addressmap_get_mappings(NULL, 0, 0);
+ addressmap_get_mappings(NULL, 0, 0, 0);
}
/** Remove all entries from the addressmap that are set to expire, ever. */
void
addressmap_clear_transient(void)
{
- addressmap_get_mappings(NULL, 2, TIME_MAX);
+ addressmap_get_mappings(NULL, 2, TIME_MAX, 0);
}
/** Clean out entries from the addressmap cache that were
@@ -633,7 +633,7 @@
void
addressmap_clean(time_t now)
{
- addressmap_get_mappings(NULL, 2, now);
+ addressmap_get_mappings(NULL, 2, now, 0);
}
/** Free all the elements in the addressmap, and free the addressmap
@@ -654,24 +654,32 @@
/** Look at address, and rewrite it until it doesn't want any
* more rewrites; but don't get into an infinite loop.
* Don't write more than maxlen chars into address. Return true if the
- * address changed; false otherwise.
+ * address changed; false otherwise. Set *<b>expires_out</b> to the
+ * expiry time of the result, or to <b>time_max</b> if the result does
+ * not expire.
*/
int
-addressmap_rewrite(char *address, size_t maxlen)
+addressmap_rewrite(char *address, size_t maxlen, time_t *expires_out)
{
addressmap_entry_t *ent;
int rewrites;
char *cp;
+ time_t expires = TIME_MAX;
for (rewrites = 0; rewrites < 16; rewrites++) {
ent = strmap_get(addressmap, address);
- if (!ent || !ent->new_address)
+ if (!ent || !ent->new_address) {
+ if (expires_out)
+ *expires_out = expires;
return (rewrites > 0); /* done, no rewrite needed */
+ }
cp = tor_strdup(escaped_safe_str(ent->new_address));
log_info(LD_APP, "Addressmap: rewriting %s to %s",
escaped_safe_str(address), cp);
+ if (ent->expires > 1 && ent->expires < expires)
+ expires = ent->expires;
tor_free(cp);
strlcpy(address, ent->new_address, maxlen);
}
@@ -679,14 +687,18 @@
"Loop detected: we've rewritten %s 16 times! Using it as-is.",
escaped_safe_str(address));
/* it's fine to rewrite a rewrite, but don't loop forever */
+ if (expires_out)
+ *expires_out = TIME_MAX;
return 1;
}
/** If we have a cached reverse DNS entry for the address stored in the
* <b>maxlen</b>-byte buffer <b>address</b> (typically, a dotted quad) then
- * rewrite to the cached value and return 1. Otherwise return 0. */
+ * rewrite to the cached value and return 1. Otherwise return 0. Set
+ * *<b>expires_out</b> to the expiry time of the result, or to <b>time_max</b>
+ * if the result does not expire. */
static int
-addressmap_rewrite_reverse(char *address, size_t maxlen)
+addressmap_rewrite_reverse(char *address, size_t maxlen, time_t *expires_out)
{
size_t len = maxlen + 16;
char *s = tor_malloc(len), *cp;
@@ -702,6 +714,10 @@
strlcpy(address, ent->new_address, maxlen);
r = 1;
}
+
+ if (expires_out)
+ *expires_out = (ent && ent->expires > 1) ? ent->expires : TIME_MAX;
+
tor_free(s);
return r;
}
@@ -765,7 +781,7 @@
log_info(LD_CONFIG, "Addressmap: (re)mapped '%s' to '%s'",
safe_str(address), safe_str(ent->new_address));
- control_event_address_mapped(address, ent->new_address, expires);
+ control_event_address_mapped(address, ent->new_address, expires, NULL);
}
/** An attempt to resolve <b>address</b> failed at some OR.
@@ -903,8 +919,7 @@
* These options are configured by parse_virtual_addr_network().
*/
static uint32_t virtual_addr_network = 0x7fc00000u;
-static uint32_t virtual_addr_netmask = 0xffc00000u;
-static int virtual_addr_netmask_bits = 10;
+static maskbits_t virtual_addr_netmask_bits = 10;
static uint32_t next_virtual_addr = 0x7fc00000u;
/** Read a netmask of the form 127.192.0.0/10 from "val", and check whether
@@ -916,11 +931,11 @@
parse_virtual_addr_network(const char *val, int validate_only,
char **msg)
{
- uint32_t addr, mask;
+ uint32_t addr;
uint16_t port_min, port_max;
- int bits;
+ maskbits_t bits;
- if (parse_addr_and_port_range(val, &addr, &mask, &port_min, &port_max)) {
+ if (parse_addr_and_port_range(val, &addr, &bits, &port_min, &port_max)) {
if (msg) *msg = tor_strdup("Error parsing VirtualAddressNetwork");
return -1;
}
@@ -930,13 +945,6 @@
return -1;
}
- bits = addr_mask_get_bits(mask);
- if (bits < 0) {
- if (msg) *msg = tor_strdup("VirtualAddressNetwork must have a mask that "
- "can be expressed as a prefix");
- return -1;
- }
-
if (bits > 16) {
if (msg) *msg = tor_strdup("VirtualAddressNetwork expects a /16 "
"network or larger");
@@ -946,11 +954,10 @@
if (validate_only)
return 0;
- virtual_addr_network = addr & mask;
- virtual_addr_netmask = mask;
+ virtual_addr_network = addr & (0xfffffffful << (32-bits));
virtual_addr_netmask_bits = bits;
- if ((next_virtual_addr & mask) != addr)
+ if (addr_mask_cmp_bits(next_virtual_addr, addr, bits))
next_virtual_addr = addr;
return 0;
@@ -969,7 +976,8 @@
return 1;
} else if (tor_inet_aton(address, &in)) {
uint32_t addr = ntohl(in.s_addr);
- if ((addr & virtual_addr_netmask) == virtual_addr_network)
+ if (!addr_mask_cmp_bits(addr, virtual_addr_network,
+ virtual_addr_netmask_bits))
return 1;
}
return 0;
@@ -1015,7 +1023,8 @@
log_warn(LD_CONFIG, "Ran out of virtual addresses!");
return NULL;
}
- if ((next_virtual_addr & virtual_addr_netmask) != virtual_addr_network)
+ if (!addr_mask_cmp_bits(next_virtual_addr, virtual_addr_network,
+ virtual_addr_netmask_bits))
next_virtual_addr = virtual_addr_network;
}
return tor_strdup(buf);
@@ -1122,12 +1131,13 @@
/** Iterate over all address mappings which have expiry times between
* min_expires and max_expires, inclusive. If sl is provided, add an
- * "old-addr new-addr" string to sl for each mapping. If sl is NULL,
- * remove the mappings.
+ * "old-addr new-addr expiry" string to sl for each mapping, omitting
+ * the expiry time if want_expiry is false. If sl is NULL, remove the
+ * mappings.
*/
void
addressmap_get_mappings(smartlist_t *sl, time_t min_expires,
- time_t max_expires)
+ time_t max_expires, int want_expiry)
{
strmap_iter_t *iter;
const char *key;
@@ -1146,9 +1156,20 @@
addressmap_ent_remove(key, val);
continue;
} else if (val->new_address) {
- size_t len = strlen(key)+strlen(val->new_address)+2;
+ size_t len = strlen(key)+strlen(val->new_address)+ISO_TIME_LEN+5;
char *line = tor_malloc(len);
- tor_snprintf(line, len, "%s %s", key, val->new_address);
+ if (want_expiry) {
+ if (val->expires < 3 || val->expires == TIME_MAX)
+ tor_snprintf(line, len, "%s %s NEVER", key, val->new_address);
+ else {
+ char time[ISO_TIME_LEN+1];
+ format_iso_time(time, val->expires);
+ tor_snprintf(line, len, "%s %s \"%s\"", key, val->new_address,
+ time);
+ }
+ } else {
+ tor_snprintf(line, len, "%s %s", key, val->new_address);
+ }
smartlist_add(sl, line);
}
}
@@ -1182,6 +1203,7 @@
struct in_addr addr_tmp;
int automap = 0;
char orig_address[MAX_SOCKS_ADDR_LEN];
+ time_t map_expires = TIME_MAX;
tor_strlower(socks->address); /* normalize it */
strlcpy(orig_address, socks->address, sizeof(orig_address));
@@ -1209,12 +1231,14 @@
}
if (socks->command == SOCKS_COMMAND_RESOLVE_PTR) {
- if (addressmap_rewrite_reverse(socks->address, sizeof(socks->address))) {
+ if (addressmap_rewrite_reverse(socks->address, sizeof(socks->address),
+ &map_expires)) {
char *result = tor_strdup(socks->address);
/* remember _what_ is supposed to have been resolved. */
strlcpy(socks->address, orig_address, sizeof(socks->address));
connection_ap_handshake_socks_resolved(conn, RESOLVED_TYPE_HOSTNAME,
- strlen(result), result, -1);
+ strlen(result), result, -1,
+ map_expires);
connection_mark_unattached_ap(conn,
END_STREAM_REASON_DONE |
END_STREAM_REASON_FLAG_ALREADY_SOCKS_REPLIED);
@@ -1222,7 +1246,8 @@
}
} else if (!automap) {
/* For address map controls, remap the address. */
- if (addressmap_rewrite(socks->address, sizeof(socks->address))) {
+ if (addressmap_rewrite(socks->address, sizeof(socks->address),
+ &map_expires)) {
control_event_stream_status(conn, STREAM_EVENT_REMAP,
REMAP_STREAM_SOURCE_CACHE);
}
@@ -1309,7 +1334,7 @@
escaped(socks->address));
connection_ap_handshake_socks_resolved(conn,
RESOLVED_TYPE_ERROR_TRANSIENT,
- 0,NULL,-1);
+ 0,NULL,-1,TIME_MAX);
connection_mark_unattached_ap(conn,
END_STREAM_REASON_SOCKSPROTOCOL |
END_STREAM_REASON_FLAG_ALREADY_SOCKS_REPLIED);
@@ -1321,7 +1346,7 @@
/* remember _what_ is supposed to have been resolved. */
strlcpy(socks->address, orig_address, sizeof(socks->address));
connection_ap_handshake_socks_resolved(conn,RESOLVED_TYPE_IPV4,4,
- (char*)&answer,-1);
+ (char*)&answer,-1,map_expires);
connection_mark_unattached_ap(conn,
END_STREAM_REASON_DONE |
END_STREAM_REASON_FLAG_ALREADY_SOCKS_REPLIED);
@@ -1383,7 +1408,7 @@
log_warn(LD_APP,
"Resolve requests to hidden services not allowed. Failing.");
connection_ap_handshake_socks_resolved(conn,RESOLVED_TYPE_ERROR,
- 0,NULL,-1);
+ 0,NULL,-1,TIME_MAX);
connection_mark_unattached_ap(conn,
END_STREAM_REASON_SOCKSPROTOCOL |
END_STREAM_REASON_FLAG_ALREADY_SOCKS_REPLIED);
@@ -1401,20 +1426,22 @@
log_info(LD_REND,"Got a hidden service request for ID '%s'",
safe_str(conn->rend_query));
/* see if we already have it cached */
- r = rend_cache_lookup_entry(conn->rend_query, -1, &entry);
+ if (rend_valid_service_id(socks->address) == 2) {
+ r = rend_cache_lookup_entry(conn->rend_query, 2, &entry);
+ conn->rend_version = 2;
+ strlcpy(conn->secret_cookie, socks->address + 17,
+ sizeof(conn->secret_cookie));
+ log_info(LD_REND, "Got a hidden service request for a v2 service.");
+ } else {
+ r = rend_cache_lookup_entry(conn->rend_query, 0, &entry);
+ conn->rend_version = 0;
+ }
if (r<0) {
log_warn(LD_BUG,"Invalid service name '%s'",
safe_str(conn->rend_query));
connection_mark_unattached_ap(conn, END_STREAM_REASON_TORPROTOCOL);
return -1;
}
- if (rend_valid_service_id(socks->address) == 2) {
- conn->rend_version = 2;
- strlcpy(conn->secret_cookie, socks->address+17,
- sizeof(conn->secret_cookie));
- } else {
- conn->rend_version = 1;
- }
if (r==0) {
conn->_base.state = AP_CONN_STATE_RENDDESC_WAIT;
log_info(LD_REND, "Unknown descriptor %s. Fetching.",
@@ -1903,7 +1930,7 @@
tor_assert(payload_len <= RELAY_PAYLOAD_SIZE);
}
- log_debug(LD_APP,
+ log_info(LD_APP,
"Sending relay cell to begin stream %d.", ap_conn->stream_id);
if (connection_edge_send_command(ap_conn,
@@ -1954,7 +1981,7 @@
digest, DIGEST_LEN);
}
- conn->_base.address = tor_strdup("(local link)");
+ conn->_base.address = tor_strdup("(Tor_internal)");
conn->_base.addr = 0;
conn->_base.port = 0;
@@ -1977,17 +2004,52 @@
return conn;
}
+/** DOCDOC */
+static void
+tell_controller_about_resolved_result(edge_connection_t *conn,
+ int answer_type,
+ size_t answer_len,
+ const char *answer,
+ int ttl,
+ time_t expires)
+{
+
+ if (ttl >= 0 && (answer_type == RESOLVED_TYPE_IPV4 ||
+ answer_type == RESOLVED_TYPE_HOSTNAME)) {
+ return; /* we already told the controller. */
+ } else if (answer_type == RESOLVED_TYPE_IPV4 && answer_len >= 4) {
+ struct in_addr in;
+ char buf[INET_NTOA_BUF_LEN];
+ in.s_addr = get_uint32(answer);
+ tor_inet_ntoa(&in, buf, sizeof(buf));
+ control_event_address_mapped(conn->socks_request->address,
+ buf, expires, NULL);
+ } else if (answer_type == RESOLVED_TYPE_HOSTNAME && answer_len <256) {
+ char *cp = tor_strndup(answer, answer_len);
+ control_event_address_mapped(conn->socks_request->address,
+ cp, expires, NULL);
+ tor_free(cp);
+ } else {
+ control_event_address_mapped(conn->socks_request->address,
+ "<error>",
+ time(NULL)+ttl,
+ "error=yes");
+ }
+}
+
/** Send an answer to an AP connection that has requested a DNS lookup
* via SOCKS. The type should be one of RESOLVED_TYPE_(IPV4|IPV6|HOSTNAME) or
* -1 for unreachable; the answer should be in the format specified
* in the socks extensions document.
+ * DOCDOC expires
**/
void
connection_ap_handshake_socks_resolved(edge_connection_t *conn,
int answer_type,
size_t answer_len,
const char *answer,
- int ttl)
+ int ttl,
+ time_t expires)
{
char buf[384];
size_t replylen;
@@ -2007,11 +2069,21 @@
}
}
- if (conn->dns_server_request) {
- /* We had a request on our DNS port: answer it. */
- dnsserv_resolved(conn, answer_type, answer_len, answer, ttl);
- conn->socks_request->has_finished = 1;
- return;
+ if (conn->is_dns_request) {
+ if (conn->dns_server_request) {
+ /* We had a request on our DNS port: answer it. */
+ dnsserv_resolved(conn, answer_type, answer_len, answer, ttl);
+ conn->socks_request->has_finished = 1;
+ return;
+ } else {
+ /* This must be a request from the controller. We already sent
+ * a mapaddress if there's a ttl. */
+ tell_controller_about_resolved_result(conn, answer_type, answer_len,
+ answer, ttl, expires);
+ conn->socks_request->has_finished = 1;
+ return;
+ }
+ /* XXXX020 are we freeing conn anywhere? */
}
if (conn->socks_request->socks_version == 4) {
@@ -2242,7 +2314,7 @@
if (circ->purpose == CIRCUIT_PURPOSE_S_REND_JOINED) {
origin_circuit_t *origin_circ = TO_ORIGIN_CIRCUIT(circ);
- log_debug(LD_REND,"begin is for rendezvous. configuring stream.");
+ log_info(LD_REND,"begin is for rendezvous. configuring stream.");
n_stream->_base.address = tor_strdup("(rendezvous)");
n_stream->_base.state = EXIT_CONN_STATE_CONNECTING;
strlcpy(n_stream->rend_query, origin_circ->rend_query,
@@ -2392,7 +2464,7 @@
if (redirect_exit_list) {
SMARTLIST_FOREACH(redirect_exit_list, exit_redirect_t *, r,
{
- if ((addr&r->mask)==(r->addr&r->mask) &&
+ if (!addr_mask_cmp_bits(addr, r->addr, r->maskbits) &&
(r->port_min <= port) && (port <= r->port_max)) {
struct in_addr in;
if (r->is_redirect) {
@@ -2604,7 +2676,6 @@
parse_extended_hostname(char *address)
{
char *s;
-
s = strrchr(address,'.');
if (!s)
return NORMAL_HOSTNAME; /* no dot, thus normal */
@@ -2617,7 +2688,7 @@
/* so it is .onion */
*s = 0; /* nul-terminate it */
- if (rend_valid_service_id(address))
+ if (rend_valid_service_id(address) >= 0)
return ONION_HOSTNAME; /* success */
/* otherwise, return to previous state and return 0 */
Modified: tor/branches/114-dist-storage/src/or/directory.c
===================================================================
--- tor/branches/114-dist-storage/src/or/directory.c 2007-08-11 15:39:12 UTC (rev 11078)
+++ tor/branches/114-dist-storage/src/or/directory.c 2007-08-11 22:30:44 UTC (rev 11079)
@@ -30,13 +30,13 @@
* - connection_dir_finished_connecting(), called from
* connection_finished_connecting() in connection.c
*/
-static void
-directory_send_command(dir_connection_t *conn,
+static void directory_send_command(dir_connection_t *conn,
int purpose, int direct, const char *resource,
const char *payload, size_t payload_len);
static int directory_handle_command(dir_connection_t *conn);
static int body_is_plausible(const char *body, size_t body_len, int purpose);
-static int purpose_needs_anonymity(uint8_t purpose);
+static int purpose_needs_anonymity(uint8_t dir_purpose,
+ uint8_t router_purpose);
static char *http_get_header(const char *headers, const char *which);
static void http_set_address_origin(const char *headers, connection_t *conn);
static void connection_dir_download_networkstatus_failed(
@@ -71,16 +71,20 @@
/** Return true iff the directory purpose 'purpose' must use an
* anonymous connection to a directory. */
static int
-purpose_needs_anonymity(uint8_t purpose)
+purpose_needs_anonymity(uint8_t dir_purpose, uint8_t router_purpose)
{
if (get_options()->AllDirActionsPrivate)
return 1;
- if (purpose == DIR_PURPOSE_FETCH_DIR ||
- purpose == DIR_PURPOSE_UPLOAD_DIR ||
- purpose == DIR_PURPOSE_FETCH_RUNNING_LIST ||
- purpose == DIR_PURPOSE_FETCH_NETWORKSTATUS ||
- purpose == DIR_PURPOSE_FETCH_SERVERDESC ||
- purpose == DIR_PURPOSE_FETCH_EXTRAINFO)
+ if (router_purpose == ROUTER_PURPOSE_BRIDGE)
+ return 1; /* if we have to ask, better make it anonymous */
+ if (dir_purpose == DIR_PURPOSE_FETCH_DIR ||
+ dir_purpose == DIR_PURPOSE_UPLOAD_DIR ||
+ dir_purpose == DIR_PURPOSE_UPLOAD_VOTE ||
+ dir_purpose == DIR_PURPOSE_UPLOAD_SIGNATURES ||
+ dir_purpose == DIR_PURPOSE_FETCH_RUNNING_LIST ||
+ dir_purpose == DIR_PURPOSE_FETCH_NETWORKSTATUS ||
+ dir_purpose == DIR_PURPOSE_FETCH_SERVERDESC ||
+ dir_purpose == DIR_PURPOSE_FETCH_EXTRAINFO)
return 0;
return 1;
}
@@ -147,7 +151,8 @@
* support it.
*/
void
-directory_post_to_dirservers(uint8_t purpose, authority_type_t type,
+directory_post_to_dirservers(uint8_t dir_purpose, uint8_t router_purpose,
+ authority_type_t type,
const char *payload,
size_t payload_len, size_t extrainfo_len)
{
@@ -167,7 +172,7 @@
continue;
found = 1; /* at least one authority of this type was listed */
- if (purpose == DIR_PURPOSE_UPLOAD_DIR)
+ if (dir_purpose == DIR_PURPOSE_UPLOAD_DIR)
ds->has_accepted_serverdesc = 0;
if (extrainfo_len && router_supports_extrainfo(ds->digest, 1)) {
@@ -175,10 +180,10 @@
log_info(LD_DIR, "Uploading an extrainfo (length %d)",
(int) extrainfo_len);
}
- post_via_tor = purpose_needs_anonymity(purpose) ||
+ post_via_tor = purpose_needs_anonymity(dir_purpose, router_purpose) ||
!fascist_firewall_allows_address_dir(ds->addr, ds->dir_port);
- directory_initiate_command_routerstatus(rs, purpose,
- ROUTER_PURPOSE_GENERAL,
+ directory_initiate_command_routerstatus(rs, dir_purpose,
+ router_purpose,
post_via_tor,
NULL, payload, upload_len);
});
@@ -190,79 +195,34 @@
}
}
-/** Start a connection to (1 or more?) hidden service directories, uploading
- * the payload 'payload' (length 'payload_len').
- *
- * desc_id = 32 base32 chars, desc = NUL-terminated ASCII-based v2 descriptor
- *
- * TODO114 enable tunneling when available!!
- */
-void
-directory_post_to_hs_dir(const char *desc_id, const char *desc)
-{
- tor_assert(desc_id);
- tor_assert(strlen(desc_id) == 32);
- tor_assert(desc);
- routerstatus_t *hs_dir = get_responsible_hs_dir(desc_id);
- directory_initiate_command_routerstatus(hs_dir,
- DIR_PURPOSE_UPLOAD_RENDDESC_V2,
- ROUTER_PURPOSE_GENERAL, 0,
- NULL, desc, strlen(desc));
-}
-
-/** Start a connection to (1 or more?) hidden service directories, downloading
- * the payload 'payload' (length 'payload_len').
- *
- * desc_id = 32 base32 chars, query = 16 base32 chars,
- * secret_cookie = 24 base32 chars
- *
- * TODO114 enable tunneling when available!!
- */
-void
-directory_get_from_hs_dir(const char *desc_id, const char *query,
- const char *secret_cookie)
-{
- tor_assert(desc_id);
- tor_assert(strlen(desc_id) == 32);
- tor_assert(query);
- tor_assert(strlen(query) == 16);
- tor_assert(secret_cookie);
- tor_assert(strlen(secret_cookie) == 24);
- routerstatus_t *hs_dir = get_responsible_hs_dir(desc_id);
- char orig_request[16+1+24+1];
- tor_snprintf(orig_request, sizeof(orig_request), "%s.%s", query,
- secret_cookie);
- directory_initiate_command_routerstatus(hs_dir,
- DIR_PURPOSE_FETCH_RENDDESC_V2,
- ROUTER_PURPOSE_GENERAL, 0,
- desc_id, orig_request, 0);
- return;
-}
-
/** Start a connection to a random running directory server, using
- * connection purpose 'purpose' and requesting 'resource'.
+ * connection purpose <b>dir_purpose</b>, intending to fetch descriptors
+ * of purpose <b>router_purpose</b>, and requesting <b>resource</b>.
* If <b>retry_if_no_servers</b>, then if all the possible servers seem
* down, mark them up and try again.
*/
void
-directory_get_from_dirserver(uint8_t dir_purpose, const char *resource,
- int retry_if_no_servers)
+directory_get_from_dirserver(uint8_t dir_purpose, uint8_t router_purpose,
+ const char *resource, int retry_if_no_servers)
{
routerstatus_t *rs = NULL;
or_options_t *options = get_options();
int prefer_authority = server_mode(options) && options->DirPort != 0;
- int directconn = !purpose_needs_anonymity(dir_purpose);
+ int get_via_tor = purpose_needs_anonymity(dir_purpose, router_purpose);
authority_type_t type;
/* FFFF we could break this switch into its own function, and call
* it elsewhere in directory.c. -RD */
switch (dir_purpose) {
case DIR_PURPOSE_FETCH_EXTRAINFO:
- type = EXTRAINFO_CACHE | V2_AUTHORITY;
+ type = EXTRAINFO_CACHE |
+ (router_purpose == ROUTER_PURPOSE_BRIDGE ? BRIDGE_AUTHORITY :
+ V2_AUTHORITY);
break;
case DIR_PURPOSE_FETCH_NETWORKSTATUS:
case DIR_PURPOSE_FETCH_SERVERDESC:
- type = V2_AUTHORITY;
+ type = (router_purpose == ROUTER_PURPOSE_BRIDGE ? BRIDGE_AUTHORITY :
+ V2_AUTHORITY);
break;
case DIR_PURPOSE_FETCH_DIR:
case DIR_PURPOSE_FETCH_RUNNING_LIST:
@@ -279,21 +239,21 @@
if (!options->FetchServerDescriptors && type != HIDSERV_AUTHORITY)
return;
- if (directconn && options->UseBridges) {
- /* want to pick a bridge for which we have a descriptor. */
+ if (!get_via_tor && options->UseBridges) {
+ /* want to ask a running bridge for which we have a descriptor. */
routerinfo_t *ri = choose_random_entry(NULL);
if (ri) {
directory_initiate_command(ri->address, ri->addr,
ri->or_port, 0,
1, ri->cache_info.identity_digest,
dir_purpose,
- ROUTER_PURPOSE_GENERAL,
+ router_purpose,
0, resource, NULL, 0);
} else
log_notice(LD_DIR, "Ignoring directory request, since no bridge "
"nodes are available yet.");
return;
- } else if (directconn) {
+ } else if (!get_via_tor) {
if (prefer_authority) {
/* only ask authdirservers, and don't ask myself */
rs = router_pick_trusteddirserver(type, 1, 1,
@@ -319,10 +279,10 @@
rs = router_pick_trusteddirserver(type, 1, 1,
retry_if_no_servers);
if (!rs)
- directconn = 0; /* last resort: try routing it via Tor */
+ get_via_tor = 1; /* last resort: try routing it via Tor */
}
}
- } else { /* !directconn */
+ } else { /* get_via_tor */
/* Never use fascistfirewall; we're going via Tor. */
if (dir_purpose == DIR_PURPOSE_FETCH_RENDDESC) {
/* only ask hidserv authorities, any of them will do */
@@ -340,15 +300,15 @@
if (rs)
directory_initiate_command_routerstatus(rs, dir_purpose,
- ROUTER_PURPOSE_GENERAL,
- !directconn,
+ router_purpose,
+ get_via_tor,
resource, NULL, 0);
else {
log_notice(LD_DIR,
"While fetching directory info, "
"no running dirservers known. Will try again later. "
"(purpose %d)", dir_purpose);
- if (!purpose_needs_anonymity(dir_purpose)) {
+ if (!purpose_needs_anonymity(dir_purpose, router_purpose)) {
/* remember we tried them all and failed. */
directory_all_unreachable(time(NULL));
}
@@ -359,7 +319,8 @@
* upload or download a server or rendezvous
* descriptor. <b>dir_purpose</b> determines what
* kind of directory connection we're launching, and must be one of
- * DIR_PURPOSE_{FETCH|UPLOAD}_{DIR|RENDDESC|RENDDESC_V2}.
+ * DIR_PURPOSE_{FETCH|UPLOAD}_{DIR|RENDDESC|RENDDESC_V2} or
+ * DIR_PURPOSE_REPLICATE_RENDDESC_V2.
* <b>router_purpose</b> specifies the descriptor purposes we have in mind
* (currently only used for FETCH_DIR).
*
@@ -382,6 +343,7 @@
char address_buf[INET_NTOA_BUF_LEN+1];
struct in_addr in;
const char *address;
+log_warn(LD_REND, "directory_initiate_command_routerstatus 1");
if ((router = router_get_by_digest(status->identity_digest))) {
address = router->address;
} else {
@@ -389,6 +351,7 @@
tor_inet_ntoa(&in, address_buf, sizeof(address_buf));
address = address_buf;
}
+log_warn(LD_REND, "directory_initiate_command_routerstatus 2");
directory_initiate_command(address, status->addr,
status->or_port, status->dir_port,
status->version_supports_begindir,
@@ -436,8 +399,8 @@
conn->_base.purpose == DIR_PURPOSE_FETCH_RUNNING_LIST) {
log_info(LD_DIR, "Giving up on directory server at '%s:%d'; retrying",
conn->_base.address, conn->_base.port);
- directory_get_from_dirserver(conn->_base.purpose, NULL,
- 0 /* don't retry_if_no_servers */);
+ directory_get_from_dirserver(conn->_base.purpose, conn->router_purpose,
+ NULL, 0 /* don't retry_if_no_servers */);
} else if (conn->_base.purpose == DIR_PURPOSE_FETCH_NETWORKSTATUS) {
log_info(LD_DIR, "Giving up on directory server at '%s'; retrying",
conn->_base.address);
@@ -471,8 +434,8 @@
smartlist_t *trusted_dirs = router_get_trusted_dir_servers();
SMARTLIST_FOREACH(trusted_dirs, trusted_dir_server_t *, ds,
++ds->n_networkstatus_failures);
- directory_get_from_dirserver(conn->_base.purpose, "all.z",
- 0 /* don't retry_if_no_servers */);
+ directory_get_from_dirserver(conn->_base.purpose, conn->router_purpose,
+ "all.z", 0 /* don't retry_if_no_servers */);
} else if (!strcmpstart(conn->requested_resource, "fp/")) {
/* We were trying to download by fingerprint; mark them all as having
* failed, and possibly retry them later.*/
@@ -522,6 +485,7 @@
int want_to_tunnel = options->TunnelDirConns && supports_begindir &&
!anonymized_connection && or_port &&
fascist_firewall_allows_address_or(addr, or_port);
+log_warn(LD_REND, "directory_initiate_command 1");
tor_assert(address);
tor_assert(addr);
@@ -550,6 +514,15 @@
case DIR_PURPOSE_UPLOAD_RENDDESC_V2:
log_debug(LD_REND,"initiating hidden-service v2 descriptor upload");
break;
+ case DIR_PURPOSE_REPLICATE_RENDDESC_V2:
+ log_debug(LD_REND,"initiating hidden-service v2 replication fetch");
+ break;
+ case DIR_PURPOSE_UPLOAD_VOTE:
+ log_debug(LD_OR,"initiating server vote upload");
+ break;
+ case DIR_PURPOSE_UPLOAD_SIGNATURES:
+ log_debug(LD_OR,"initiating consensus signature upload");
+ break;
case DIR_PURPOSE_FETCH_RUNNING_LIST:
log_debug(LD_DIR,"initiating running-routers fetch");
break;
@@ -567,9 +540,10 @@
tor_assert(0);
}
+log_warn(LD_REND, "directory_initiate_command 2");
conn = TO_DIR_CONN(connection_new(CONN_TYPE_DIR, AF_INET));
- // TODO114 this is a (dirty) workaround!
+ /* This is a (dirty) workaround, but otherwise we don't get the conn... */
if (dir_purpose == DIR_PURPOSE_FETCH_RENDDESC_V2) {
tor_assert(payload);
strlcpy(conn->rend_query, payload, sizeof(conn->rend_query));
@@ -743,6 +717,18 @@
httpcommand = "POST";
url = tor_strdup("/tor/");
break;
+ case DIR_PURPOSE_UPLOAD_VOTE:
+ tor_assert(!resource);
+ tor_assert(payload);
+ httpcommand = "POST";
+ url = tor_strdup("/tor/post/vote");
+ break;
+ case DIR_PURPOSE_UPLOAD_SIGNATURES:
+ tor_assert(!resource);
+ tor_assert(payload);
+ httpcommand = "POST";
+ url = tor_strdup("/tor/post/vote");
+ break;
case DIR_PURPOSE_FETCH_RENDDESC:
tor_assert(resource);
tor_assert(!payload);
@@ -769,6 +755,15 @@
url = tor_malloc(len);
tor_snprintf(url, len, "/tor/rendezvous2/%s", resource);
break;
+ case DIR_PURPOSE_REPLICATE_RENDDESC_V2:
+ tor_assert(resource);
+ tor_assert(!payload);
+ tor_assert(strlen(resource) <= 2*REND_DESC_ID_V2_LEN+1);
+ httpcommand = "GET";
+ len = strlen(resource)+32;
+ url = tor_malloc(len);
+ tor_snprintf(url, len, "/tor/rendezvous2/%s", resource);
+ break;
case DIR_PURPOSE_UPLOAD_RENDDESC:
tor_assert(!resource);
tor_assert(payload);
@@ -779,7 +774,7 @@
tor_assert(!resource);
tor_assert(payload);
httpcommand = "POST";
- url = tor_strdup("/tor/rendezvous/publish2");
+ url = tor_strdup("/tor/rendezvous2/publish");
break;
default:
tor_assert(0);
@@ -1440,6 +1435,54 @@
* dirservers down just because they don't like us. */
}
+ if (conn->_base.purpose == DIR_PURPOSE_UPLOAD_VOTE) {
+ switch (status_code) {
+ case 200: {
+ log_notice(LD_DIR,"Uploaded a vote to dirserver %s:%d",
+ conn->_base.address, conn->_base.port);
+ }
+ break;
+ case 400:
+ log_warn(LD_DIR,"http status 400 (%s) response after uploading "
+ "vote to dirserver '%s:%d'. Please correct.",
+ escaped(reason), conn->_base.address, conn->_base.port);
+ break;
+ default:
+ log_warn(LD_GENERAL,
+ "http status %d (%s) reason unexpected while uploading "
+ "vote to server '%s:%d').",
+ status_code, escaped(reason), conn->_base.address,
+ conn->_base.port);
+ break;
+ }
+ /* return 0 in all cases, since we don't want to mark any
+ * dirservers down just because they don't like us. */
+ }
+
+ if (conn->_base.purpose == DIR_PURPOSE_UPLOAD_SIGNATURES) {
+ switch (status_code) {
+ case 200: {
+ log_notice(LD_DIR,"Uploaded a signatures to dirserver %s:%d",
+ conn->_base.address, conn->_base.port);
+ }
+ break;
+ case 400:
+ log_warn(LD_DIR,"http status 400 (%s) response after uploading "
+ "signatures to dirserver '%s:%d'. Please correct.",
+ escaped(reason), conn->_base.address, conn->_base.port);
+ break;
+ default:
+ log_warn(LD_GENERAL,
+ "http status %d (%s) reason unexpected while uploading "
+ "signatures to server '%s:%d').",
+ status_code, escaped(reason), conn->_base.address,
+ conn->_base.port);
+ break;
+ }
+ /* return 0 in all cases, since we don't want to mark any
+ * dirservers down just because they don't like us. */
+ }
+
if (conn->_base.purpose == DIR_PURPOSE_FETCH_RENDDESC) {
log_info(LD_REND,"Received rendezvous descriptor (size %d, status %d "
"(%s))",
@@ -1447,7 +1490,7 @@
switch (status_code) {
case 200:
if (rend_cache_store(body, body_len, 0) < 0) {
- log_warn(LD_REND,"Failed to store rendezvous descriptor.");
+ log_warn(LD_REND, "Failed to fetch rendezvous descriptor.");
/* alice's ap_stream will notice when connection_mark_for_close
* cleans it up */
} else {
@@ -1475,14 +1518,18 @@
}
if (conn->_base.purpose == DIR_PURPOSE_FETCH_RENDDESC_V2) {
+ log_info(LD_REND,"Received rendezvous descriptor (size %d, status %d "
+ "(%s))",
+ (int)body_len, status_code, escaped(reason));
switch (status_code) {
case 200:
if (rend_cache_store_v2_client(body, conn->secret_cookie) < 0) {
- log_warn(LD_REND,"Failed to store rendezvous descriptor.");
+ log_warn(LD_REND,"Fetching v2 descriptor failed.");
/* alice's ap_stream will notice when connection_mark_for_close
* cleans it up */
} else {
/* success. notify pending connections about this. */
+ log_info(LD_REND, "Successfully fetched descriptor.");
conn->_base.purpose = DIR_PURPOSE_HAS_FETCHED_RENDDESC;
rend_client_desc_here(conn->rend_query);
}
@@ -1492,20 +1539,52 @@
* connection_mark_for_close cleans it up. */
break;
case 400:
- log_warn(LD_REND,
+ log_warn(LD_REND, "Fetching v2 descriptor failed: "
"http status 400 (%s). Dirserver didn't like our "
"v2 rendezvous query?", escaped(reason));
break;
default:
- log_warn(LD_REND,"http status %d (%s) response unexpected while "
+ log_warn(LD_REND, "Fetching v2 descriptor failed: "
+ "http status %d (%s) response unexpected while "
"fetching v2 hidden service descriptor (server '%s:%d').",
status_code, escaped(reason), conn->_base.address,
conn->_base.port);
break;
}
}
+
+ if (conn->_base.purpose == DIR_PURPOSE_REPLICATE_RENDDESC_V2) {
+ log_info(LD_REND, "Received replicas of rendezvous descriptors (size %d, "
+ "status %d (%s))",
+ (int)body_len, status_code, escaped(reason));
+ switch (status_code) {
+ case 200:
+ if (rend_cache_store_v2_dir(body) < 0) {
+ log_warn(LD_REND, "Fetching v2 descriptor replicas failed.");
+ } else {
+ log_info(LD_REND, "Successfully fetched replicas.");
+ }
+ break;
+ case 400:
+ log_warn(LD_REND, "Fetching v2 descriptor replicas failed: "
+ "http status 400 (%s). Dirserver didn't like our "
+ "v2 rendezvous query?", escaped(reason));
+ break;
+ default:
+ log_warn(LD_REND, "Fetching v2 descriptor replicas failed: "
+ "http status %d (%s) response unexpected while "
+ "fetching v2 replicas (server '%s:%d').",
+ status_code, escaped(reason), conn->_base.address,
+ conn->_base.port);
+ break;
+ }
+ }
- if (conn->_base.purpose == DIR_PURPOSE_UPLOAD_RENDDESC) {
+ if (conn->_base.purpose == DIR_PURPOSE_UPLOAD_RENDDESC ||
+ conn->_base.purpose == DIR_PURPOSE_UPLOAD_RENDDESC_V2) {
+ log_info(LD_REND,"Uploaded rendezvous descriptor (status %d "
+ "(%s))",
+ status_code, escaped(reason));
switch (status_code) {
case 200:
log_info(LD_REND,
@@ -1809,7 +1888,8 @@
/* try to get a new one now */
if (!already_fetching_directory(DIR_PURPOSE_FETCH_DIR) &&
!should_delay_dir_fetches(options))
- directory_get_from_dirserver(DIR_PURPOSE_FETCH_DIR, NULL, 1);
+ directory_get_from_dirserver(DIR_PURPOSE_FETCH_DIR,
+ ROUTER_PURPOSE_GENERAL, NULL, 1);
tor_free(url);
return 0;
}
@@ -1860,7 +1940,8 @@
/* try to get a new one now */
if (!already_fetching_directory(DIR_PURPOSE_FETCH_RUNNING_LIST) &&
!should_delay_dir_fetches(options))
- directory_get_from_dirserver(DIR_PURPOSE_FETCH_RUNNING_LIST, NULL, 1);
+ directory_get_from_dirserver(DIR_PURPOSE_FETCH_RUNNING_LIST,
+ ROUTER_PURPOSE_GENERAL, NULL, 1);
tor_free(url);
return 0;
}
@@ -1889,25 +1970,35 @@
return 0;
}
- if (!strcmpstart(url,"/tor/status/")) {
- /* v2 network status fetch. */
+ if (!strcmpstart(url,"/tor/status/")
+ || !strcmp(url, "/tor/status-vote/current/consensus")
+ || !strcmp(url, "/tor/status-vote/current/consensus.z")) {
+ /* v2 or v3 network status fetch. */
size_t url_len = strlen(url);
int deflated = !strcmp(url+url_len-2, ".z");
smartlist_t *dir_fps = smartlist_create();
+ int is_v3 = !strcmpstart(url, "/tor/status-vote");
const char *request_type = NULL;
const char *key = url + strlen("/tor/status/");
if (deflated)
url[url_len-2] = '\0';
- dirserv_get_networkstatus_v2_fingerprints(dir_fps, key);
- if (!strcmpstart(key, "fp/"))
- request_type = deflated?"/tor/status/fp.z":"/tor/status/fp";
- else if (!strcmpstart(key, "authority"))
- request_type = deflated?"/tor/status/authority.z":
- "/tor/status/authority";
- else if (!strcmpstart(key, "all"))
- request_type = deflated?"/tor/status/all.z":"/tor/status/all";
- else
- request_type = "/tor/status/?";
+ if (!is_v3) {
+ dirserv_get_networkstatus_v2_fingerprints(dir_fps, key);
+ if (!strcmpstart(key, "fp/"))
+ request_type = deflated?"/tor/status/fp.z":"/tor/status/fp";
+ else if (!strcmpstart(key, "authority"))
+ request_type = deflated?"/tor/status/authority.z":
+ "/tor/status/authority";
+ else if (!strcmpstart(key, "all"))
+ request_type = deflated?"/tor/status/all.z":"/tor/status/all";
+ else
+ request_type = "/tor/status/?";
+ } else {
+ smartlist_add(dir_fps, tor_memdup("\0\0\0\0\0\0\0\0\0\0"
+ "\0\0\0\0\0\0\0\0\0\0", 20));
+ request_type = deflated?"v3.z":"v3";
+ }
+
tor_free(url);
if (!smartlist_len(dir_fps)) { /* we failed to create/cache cp */
write_http_status_line(conn, 503, "Network status object unavailable");
@@ -2018,42 +2109,52 @@
if (options->HSDir &&
!strcmpstart(url,"/tor/rendezvous2/")) {
- /* v2 rendezvous descriptor fetch */
- const char *descp;
- size_t desc_len;
- const char *query = url+strlen("/tor/rendezvous2/");
-
- switch (rend_cache_lookup_v2_dir(query, &descp, &desc_len)) {
- case 1: /* valid */
- write_http_response_header(conn, desc_len, "text/plain",
+ /* Handle v2 rendezvous service fetch request. */
+ char *descp;
+ const char *query = url + strlen("/tor/rendezvous2/");
+ if (strlen(query) == REND_DESC_ID_V2_LEN) {
+ switch (rend_cache_lookup_v2_dir(query, &descp)) {
+ case 1: /* valid */
+ write_http_response_header(conn, strlen(descp), "text/plain",
+ NULL, 0);
+ connection_write_to_buf(descp, strlen(descp), TO_CONN(conn));
+ tor_free(descp);
+ break;
+ case 0: /* well-formed but not present */
+ write_http_status_line(conn, 404, "Not found");
+ break;
+ case -1: /* not well-formed */
+ write_http_status_line(conn, 400, "Bad request");
+ break;
+ }
+ /* Handle v2 rendezvous service replica request. */
+ } else if (strlen(query) == 2 * REND_DESC_ID_V2_LEN + 1) {
+ if (rend_cache_lookup_v2_replicas(query, &descp) < 0) {
+ /* not well-formed */
+ write_http_status_line(conn, 400, "Bad request");
+ } else {
+ /* valid */
+ write_http_response_header(conn, strlen(descp), "text/plain",
NULL, 0);
- /* need to send descp separately, because it may include nuls */
- connection_write_to_buf(descp, desc_len, TO_CONN(conn));
- break;
- case 0: /* well-formed but not present */
- write_http_status_line(conn, 404, "Not found");
- break;
- case -1: /* not well-formed */
- write_http_status_line(conn, 400, "Bad request");
- break;
+ connection_write_to_buf(descp, strlen(descp), TO_CONN(conn));
+ tor_free(descp);
+ }
+ } else { /* not well-formed */
+ write_http_status_line(conn, 400, "Bad request");
}
tor_free(url);
return 0;
}
if (options->HSAuthoritativeDir &&
- (!strcmpstart(url,"/tor/rendezvous/") ||
- !strcmpstart(url,"/tor/rendezvous1/"))) {
+ !strcmpstart(url,"/tor/rendezvous/")) {
/* rendezvous descriptor fetch */
const char *descp;
size_t desc_len;
- int versioned = !strcmpstart(url,"/tor/rendezvous1/");
- const char *query = url+strlen("/tor/rendezvous/")+(versioned?1:0);
+ const char *query = url+strlen("/tor/rendezvous/");
- // TODO114 it is not correct to allow all versions, but only
- // 0 or 1; what if we were by chance a v2 client for that service? then we
- // should better not return the v2 descriptor as result to a 0/1 request!
- switch (rend_cache_lookup_desc(query, versioned?-1:0, &descp, &desc_len)) {
+ log_info(LD_REND, "Handling rendezvous descriptor get");
+ switch (rend_cache_lookup_desc(query, 0, &descp, &desc_len)) {
case 1: /* valid */
write_http_response_header(conn, desc_len, "application/octet-stream",
NULL, 0);
@@ -2155,12 +2256,12 @@
}
log_debug(LD_DIRSERV,"rewritten url as '%s'.", url);
+ /* Handle v2 rendezvous service publish request. */
if (options->HSDir &&
- !strcmpstart(url,"/tor/rendezvous/publish2")) {
- /* rendezvous descriptor post */
- if (rend_cache_store_v2_dir(body) < 0) {
- log_fn(LOG_PROTOCOL_WARN, LD_DIRSERV,
- "Rejected rend descriptor (length %d) from %s.",
+ !strcmpstart(url,"/tor/rendezvous2/publish")) {
+ log_info(LD_REND, "Handling v2 rendezvous descriptor post");
+ if (rend_cache_store_v2_dir(body) <= 0) {
+ log_warn(LD_REND, "Rejected rend descriptor (length %d) from %s.",
(int)body_len, conn->_base.address);
write_http_status_line(conn, 400, "Invalid service descriptor rejected");
} else {
@@ -2206,9 +2307,11 @@
if (options->HSAuthoritativeDir &&
!strcmpstart(url,"/tor/rendezvous/publish")) {
+
+ log_info(LD_REND, "Handling rendezvous descriptor post.");
+
/* rendezvous descriptor post */
if (rend_cache_store(body, body_len, 1) < 0) {
-// char tmp[1024*2+1];
log_fn(LOG_PROTOCOL_WARN, LD_DIRSERV,
"Rejected rend descriptor (length %d) from %s.",
(int)body_len, conn->_base.address);
@@ -2219,6 +2322,28 @@
goto done;
}
+ if (authdir_mode_v3(options) &&
+ !strcmp(url,"/tor/post/vote")) { /* server descriptor post */
+ const char *msg = "OK";
+ if (dirvote_add_vote(body, &msg)) {
+ write_http_status_line(conn, 200, "Vote stored");
+ } else {
+ tor_assert(msg);
+ write_http_status_line(conn, 400, msg);
+ }
+ goto done;
+ }
+
+ if (authdir_mode_v3(options) &&
+ !strcmp(url,"/tor/post/consensus-signature")) { /* sigs on consensus. */
+ if (dirvote_add_signatures(body)>=0) {
+ write_http_status_line(conn, 200, "Signatures stored");
+ } else {
+ write_http_status_line(conn, 400, "Unable to store signatures");
+ }
+ goto done;
+ }
+
/* we didn't recognize the url */
write_http_status_line(conn, 404, "Not found");
@@ -2484,3 +2609,190 @@
return 0;
}
+/** Determine the responsible hidden service directories for <b>desc_ids</b>
+ * and upload the appropriate descriptor from <b>descs</b> to them.
+ *
+ * TODO114 enable tunneling when available!!
+ */
+void
+directory_post_to_hs_dir(const char *service_id, char desc_ids[][20],
+ smartlist_t *descs, int seconds_valid)
+{
+ int i, j;
+ routerstatus_t *hs_dirs[NUMBER_OF_CONSECUTIVE_REPLICAS];
+ routerstatus_t *hs_dir;
+log_warn(LD_REND, "directory_post_to_hs_dir 1");
+ for (i = 0; i < NUMBER_OF_NON_CONSECUTIVE_REPLICAS; i++) {
+log_warn(LD_REND, "directory_post_to_hs_dir 2");
+ /* Determine responsible dirs. */
+ if (get_responsible_hs_dirs(desc_ids[i], hs_dirs) < 0) {
+ log_warn(LD_REND, "Could not determine the responsible hidden service "
+ "directories to post descriptors to.");
+ return;
+ }
+log_warn(LD_REND, "directory_post_to_hs_dir 3");
+ for (j = 0; j < NUMBER_OF_CONSECUTIVE_REPLICAS; j++) {
+ hs_dir = hs_dirs[j];
+log_warn(LD_REND, "directory_post_to_hs_dir 4");
+ /* Send publish request. */
+ directory_initiate_command_routerstatus(hs_dir,
+ DIR_PURPOSE_UPLOAD_RENDDESC_V2,
+ ROUTER_PURPOSE_GENERAL, 0,
+ NULL, descs->list[i],
+ strlen(descs->list[i]));
+ log_info(LD_REND, "Sending publish request for v2 descriptor for "
+ "service '%s' with descriptor ID '%s' with validity "
+ "of %d seconds to hidden service directory '%s' on "
+ "port %d.",
+ service_id,
+ desc_ids[i],
+ seconds_valid,
+ hs_dir->nickname,
+ hs_dir->dir_port);
+ }
+log_warn(LD_REND, "directory_post_to_hs_dir 5");
+ }
+log_warn(LD_REND, "directory_post_to_hs_dir 6");
+}
+
+/** Determine the responsible hidden service directories for <b>desc_id</b>
+ * and that descriptor from one of them; <b>query</b> and <b>secret_cookie</b>
+ * are only passed for pretty log statements.
+ *
+ * TODO114 enable tunneling when available!!
+ */
+void
+directory_get_from_hs_dir(const char *desc_id, const char *query,
+ const char *secret_cookie)
+{
+ routerstatus_t *hs_dirs[NUMBER_OF_CONSECUTIVE_REPLICAS];
+ routerstatus_t *hs_dir;
+ char orig_request[16 + 1 + 24 + 1];
+ char desc_id_binary[REND_DESC_ID_V2_LEN + 1];
+ int replica;
+ tor_assert(desc_id);
+ tor_assert(query);
+ tor_assert(strlen(query) == 16);
+ tor_assert(secret_cookie);
+ tor_assert(strlen(secret_cookie) == 24);
+ /* Determine responsible dirs. */
+ if (get_responsible_hs_dirs(desc_id, hs_dirs) < 0) {
+ log_warn(LD_REND, "Could not determine the responsible hidden service "
+ "directories to fetch descriptors.");
+ return;
+ }
+ replica = crypto_rand_int(NUMBER_OF_CONSECUTIVE_REPLICAS);
+ hs_dir = hs_dirs[replica];
+ /* TODO114 if hsdir fails, use another one... */
+ tor_snprintf(orig_request, sizeof(orig_request), "%s.%s", query,
+ secret_cookie);
+ base32_encode(desc_id_binary, REND_DESC_ID_V2_LEN + 1, desc_id, DIGEST_LEN);
+ /* Send fetch request. */
+ directory_initiate_command_routerstatus(hs_dir,
+ DIR_PURPOSE_FETCH_RENDDESC_V2,
+ ROUTER_PURPOSE_GENERAL, 0,
+ desc_id_binary, orig_request, 0);
+ log_info(LD_REND, "Sending fetch request for v2 descriptor for "
+ "service '%s' with descriptor ID '%s' from hidden "
+ "service directory '%s' on port %d.",
+ query, desc_id_binary, hs_dir->nickname, hs_dir->dir_port);
+}
+
+/** Send a fetch request for replicas in interval from <b>from_id</b> to
+ * <b>to_id</b> to hidden service directory <b>hs_dir</b>. */
+void
+hs_dir_fetch_replicas(routerstatus_t *hs_dir, const char *from_id,
+ const char *to_id)
+{
+ char request[REND_DESC_ID_V2_LEN + 1 + REND_DESC_ID_V2_LEN + 1];
+ tor_assert(hs_dir);
+ tor_assert(from_id);
+ tor_assert(to_id);
+ tor_assert(strlen(from_id) == 32);
+ tor_assert(strlen(to_id) == 32);
+ tor_snprintf(request, sizeof(request), "%s-%s", from_id, to_id);
+ directory_initiate_command_routerstatus(hs_dir,
+ DIR_PURPOSE_REPLICATE_RENDDESC_V2,
+ ROUTER_PURPOSE_GENERAL, 0,
+ request, NULL, 0);
+ log_info(LD_REND, "Sending replication request for v2 descriptors in "
+ "interval '%s' to '%s' from hidden service "
+ "directory '%s' on port %d.",
+ from_id,
+ to_id,
+ hs_dir->nickname,
+ hs_dir->dir_port);
+}
+
+/** If currently acting as hidden service directory, request replicas from
+ * predecessors, request descriptors for which this node is primarily
+ * responsible from successors, and clean up all descriptors for which this
+ * node is not (any more) responsible. */
+void
+hs_dir_perform_replication(void)
+{
+ const char *me;
+ const char *predecessor0;
+ const char *predecessor1;
+ char from_id_base32[REND_DESC_ID_V2_LEN+1];
+ char to_id_base32[REND_DESC_ID_V2_LEN+1];
+ int i;
+ const char *direct_predecessor;
+ const char *successor;
+ local_routerstatus_t *pred_router;
+ routerstatus_t *pred_status;
+ local_routerstatus_t *succ_router;
+ routerstatus_t *succ_status;
+ /* Check if I am acting as hidden service directory and there are enough
+ * other hidden service directories available; otherwise there is no
+ * replication necessary/possible. */
+log_info(LD_REND, "KL8 are we acting as hs dir? %d", acting_as_hs_dir());
+ if (!acting_as_hs_dir())
+ return;
+ /* Get descriptors of which I should hold replicas from
+ * NUMBER_OF_CONSECUTIVE_REPLICAS - 1 predecessors. */
+ me = router_get_my_routerinfo()->cache_info.identity_digest;
+ predecessor0 = me;
+ predecessor1 = previous_hs_dir(me);
+ for (i = 0; i < NUMBER_OF_CONSECUTIVE_REPLICAS - 1; i++) {
+ predecessor0 = predecessor1;
+ predecessor1 = previous_hs_dir(predecessor0);
+ pred_router = router_get_combined_status_by_digest(predecessor0);
+ pred_status = &(pred_router->status);
+ base32_encode(from_id_base32, REND_DESC_ID_V2_LEN + 1, predecessor1,
+ DIGEST_LEN);
+ base32_encode(to_id_base32, REND_DESC_ID_V2_LEN + 1, predecessor0,
+ DIGEST_LEN);
+ log_debug(LD_REND, "Requesting descriptors in interval %s to %s as "
+ "replicas from predecessor.",
+ from_id_base32,
+ to_id_base32);
+log_info(LD_REND, "hs_dir_perform_replication 1");
+ hs_dir_fetch_replicas(pred_status, from_id_base32, to_id_base32);
+ }
+ /* Get descriptors for which I am primarily responsible from
+ * NUMBER_OF_CONSECUTIVE_REPLICAS - 1 successors. */
+ direct_predecessor = previous_hs_dir(me);
+ successor = next_hs_dir(me);
+ for (i = 0; i < NUMBER_OF_CONSECUTIVE_REPLICAS - 1; i++) {
+ succ_router = router_get_combined_status_by_digest(successor);
+ succ_status = &(succ_router->status);
+ base32_encode(from_id_base32, REND_DESC_ID_V2_LEN + 1, direct_predecessor,
+ DIGEST_LEN);
+ base32_encode(to_id_base32, REND_DESC_ID_V2_LEN + 1, me,
+ DIGEST_LEN);
+ log_debug(LD_REND, "Requesting descriptors in interval %s to %s for "
+ "which I am primarily responsible from successor.",
+ from_id_base32,
+ to_id_base32);
+log_info(LD_REND, "hs_dir_perform_replication 2");
+ hs_dir_fetch_replicas(succ_status, from_id_base32, to_id_base32);
+ successor = next_hs_dir(successor);
+ }
+ /* Clean up descriptors for which I am not reponsible (neither primarily
+ * nor for replication). */
+ log_debug(LD_REND, "Cleaning up all descriptors that are not (any more) in "
+ "the interval for which i am responsible.");
+ rend_cache_clean_up();
+}
+
Modified: tor/branches/114-dist-storage/src/or/dirserv.c
===================================================================
--- tor/branches/114-dist-storage/src/or/dirserv.c 2007-08-11 15:39:12 UTC (rev 11078)
+++ tor/branches/114-dist-storage/src/or/dirserv.c 2007-08-11 22:30:44 UTC (rev 11079)
@@ -12,7 +12,7 @@
* \file dirserv.c
* \brief Directory server core implementation. Manages directory
* contents and generates directories.
- **/
+ */
/** How far in the future do we allow a router to get? (seconds) */
#define ROUTER_ALLOW_SKEW (60*60*12)
@@ -32,6 +32,13 @@
static int runningrouters_is_dirty = 1;
static int the_v2_networkstatus_is_dirty = 1;
+/** Most recently generated encoded signed v1 directory. (v1 auth dirservers
+ * only.) */
+static cached_dir_t *the_directory = NULL;
+
+/** For authoritative directories: the current (v1) network status. */
+static cached_dir_t the_runningrouters = { NULL, NULL, 0, 0, 0, -1 };
+
static void directory_remove_invalid(void);
static cached_dir_t *dirserv_regenerate_directory(void);
static char *format_versions_list(config_line_t *ln);
@@ -693,7 +700,8 @@
if (r & FP_REJECT) {
log_info(LD_DIRSERV, "Router '%s' is now rejected: %s",
ent->nickname, msg?msg:"");
- routerlist_remove(rl, ent, i--, 0);
+ routerlist_remove(rl, ent, 0);
+ i--;
changed = 1;
continue;
}
@@ -771,11 +779,26 @@
directory_set_dirty(void)
{
time_t now = time(NULL);
+ int set_v1_dirty=0;
- if (!the_directory_is_dirty)
- the_directory_is_dirty = now;
- if (!runningrouters_is_dirty)
- runningrouters_is_dirty = now;
+#ifdef FULL_V1_DIRECTORIES
+ set_v1_dirty = 1;
+#else
+ /* Regenerate stubs only every 8 hours. XXXX020 */
+#define STUB_REGENERATE_INTERVAL (8*60*60)
+ if (!the_directory || !the_runningrouters.dir)
+ set_v1_dirty = 1;
+ else if (the_directory->published < now - STUB_REGENERATE_INTERVAL ||
+ the_runningrouters.published < now - STUB_REGENERATE_INTERVAL)
+ set_v1_dirty = 1;
+#endif
+
+ if (set_v1_dirty) {
+ if (!the_directory_is_dirty)
+ the_directory_is_dirty = now;
+ if (!runningrouters_is_dirty)
+ runningrouters_is_dirty = now;
+ }
if (!the_v2_networkstatus_is_dirty)
the_v2_networkstatus_is_dirty = now;
}
@@ -946,14 +969,22 @@
char *buf = NULL;
size_t buf_len;
size_t identity_pkey_len;
+ time_t now = time(NULL);
+#ifdef FULL_V1_DIRECTORIES
routerlist_t *rl = router_get_routerlist();
- time_t now = time(NULL);
+#else
+ (void)complete;
+#endif
tor_assert(dir_out);
*dir_out = NULL;
+#ifdef FULL_V1_DIRECTORIES
if (list_server_status(rl->routers, &router_status, 0))
return -1;
+#else
+ router_status = tor_strdup("");
+#endif
if (crypto_pk_write_public_key_to_string(private_key,&identity_pkey,
&identity_pkey_len)<0) {
@@ -968,9 +999,11 @@
buf_len = 2048+strlen(recommended_versions)+
strlen(router_status);
+#ifdef FULL_V1_DIRECTORIES
SMARTLIST_FOREACH(rl->routers, routerinfo_t *, ri,
if (complete || router_is_active(ri, now))
buf_len += ri->cache_info.signed_descriptor_len+1);
+#endif
buf = tor_malloc(buf_len);
/* We'll be comparing against buf_len throughout the rest of the
function, though strictly speaking we shouldn't be able to exceed
@@ -991,6 +1024,7 @@
tor_free(identity_pkey);
cp = buf + strlen(buf);
+#ifdef FULL_V1_DIRECTORIES
SMARTLIST_FOREACH(rl->routers, routerinfo_t *, ri,
{
size_t len = ri->cache_info.signed_descriptor_len;
@@ -1005,6 +1039,7 @@
*cp++ = '\n'; /* add an extra newline in case somebody was depending on
* it. */
});
+#endif
*cp = '\0';
/* These multiple strlcat calls are inefficient, but dwarfed by the RSA
@@ -1035,10 +1070,6 @@
return -1;
}
-/** Most recently generated encoded signed v1 directory. (v1 auth dirservers
- * only.) */
-static cached_dir_t *the_directory = NULL;
-
/* Used only by non-v1-auth dirservers: The v1 directory and
* runningrouters we'll serve when requested. */
static cached_dir_t *cached_directory = NULL;
@@ -1048,6 +1079,9 @@
* cached_dir_t. */
static digestmap_t *cached_v2_networkstatus = NULL;
+/** DOCDOC */
+static cached_dir_t *cached_v3_networkstatus = NULL;
+
/** Possibly replace the contents of <b>d</b> with the value of
* <b>directory</b> published on <b>when</b>, unless <b>when</b> is older than
* the last value, or too far in the future.
@@ -1092,7 +1126,7 @@
/** Allocate and return a new cached_dir_t containing the string <b>s</b>,
* published at <b>published</b>. */
-static cached_dir_t *
+cached_dir_t *
new_cached_dir(char *s, time_t published)
{
cached_dir_t *d = tor_malloc_zero(sizeof(cached_dir_t));
@@ -1214,6 +1248,17 @@
}
}
+/* DOCDOC */
+void
+dirserv_set_cached_networkstatus_v3(const char *networkstatus,
+ time_t published)
+{
+ if (cached_v3_networkstatus)
+ cached_dir_decref(cached_v3_networkstatus);
+ cached_v3_networkstatus = new_cached_dir(
+ tor_strdup(networkstatus), published);
+}
+
/** Remove any v2 networkstatus from the directory cache that was published
* before <b>cutoff</b>. */
void
@@ -1341,9 +1386,6 @@
return the_directory;
}
-/** For authoritative directories: the current (v1) network status. */
-static cached_dir_t the_runningrouters = { NULL, NULL, 0, 0, 0, -1 };
-
/** Only called by v1 auth dirservers.
* Replace the current running-routers list with a newly generated one. */
static cached_dir_t *
@@ -1357,11 +1399,17 @@
crypto_pk_env_t *private_key = get_identity_key();
char *identity_pkey; /* Identity key, DER64-encoded. */
size_t identity_pkey_len;
+#ifdef FULL_V1_DIRECTORIES
routerlist_t *rl = router_get_routerlist();
+#endif
+#ifdef FULL_V1_DIRECTORIES
if (list_server_status(rl->routers, &router_status, 0)) {
goto err;
}
+#else
+ router_status = tor_strdup("");
+#endif
if (crypto_pk_write_public_key_to_string(private_key,&identity_pkey,
&identity_pkey_len)<0) {
log_warn(LD_BUG,"write identity_pkey to string failed!");
@@ -1412,6 +1460,12 @@
"v1 network status list", V1_AUTHORITY);
}
+cached_dir_t *
+dirserv_get_consensus(void)
+{
+ return cached_v3_networkstatus;
+}
+
/** For authoritative directories: the current (v2) network status. */
static cached_dir_t *the_v2_networkstatus = NULL;
@@ -1436,6 +1490,12 @@
* network using allegedly high-uptime nodes, displacing all the
* current guards. */
#define UPTIME_TO_GUARANTEE_STABLE (3600*24*30)
+/** Similarly, we protect sufficiently fast nodes from being pushed
+ * out of the set of Fast nodes. */
+#define BANDWIDTH_TO_GUARANTEE_FAST (100*1024)
+/** Similarly, every node with sufficient bandwidth can be considered
+ * for Guard status. */
+#define BANDWIDTH_TO_GUARANTEE_GUARD (250*1024)
/* Thresholds for server performance: set by
* dirserv_compute_performance_thresholds, and used by
@@ -1474,12 +1534,31 @@
(unsigned)uptime < UPTIME_TO_GUARANTEE_STABLE)
return 1;
}
- if (need_capacity &&
- router_get_advertised_bandwidth(router) < fast_bandwidth)
- return 1;
+ if (need_capacity) {
+ uint32_t bw = router_get_advertised_bandwidth(router);
+ if (bw < fast_bandwidth && bw < BANDWIDTH_TO_GUARANTEE_FAST)
+ return 1;
+ }
return 0;
}
+/** Return 1 if <b>router</b> has an uptime of at least 24 hours and is
+ * reachable, else 0.
+ */
+static int
+dirserv_thinks_router_is_hs_dir(routerinfo_t *router, time_t now)
+{
+ int uptime = real_uptime(router, now);
+
+
+
+ return (router->wants_to_be_hs_dir &&
+ (unsigned)uptime > MIN_UPTIME_HS_DIR &&
+ ((router_is_me(router) && !we_are_hibernating()) ||
+ (now < router->last_reachable + HS_DIR_REACHABLE_TIMEOUT)));
+}
+
+
/** Helper: returns a tristate based on comparing **(uint32_t**)<b>a</b>
* to **(uint32_t**)<b>b</b>. */
static int
@@ -1678,6 +1757,86 @@
DIGEST_LEN);
}
+/** DOCDOC
+ *
+ * sort first by addr, and then by descending order of usefulness.
+ **/
+static int
+_compare_routerinfo_by_ip_and_bw(const void **a, const void **b)
+{
+ routerinfo_t *first = *(routerinfo_t **)a, *second = *(routerinfo_t **)b;
+ int first_is_auth, second_is_auth;
+ uint32_t bw_first, bw_second;
+
+ /* we return -1 if first should appear before second... that is,
+ * if first is a better router. */
+ if (first->addr < second->addr)
+ return -1;
+ else if (first->addr > second->addr)
+ return 1;
+
+ /* XXX020 k n lg n memcmps could show up bigtime in profiling. If
+ * they do, I suggest we just give authorities a free pass. -RD */
+ /* I think we're fine. Remember, in nearly all cases, the addresses
+ * will be different and we'll never actually reach this point. -NM */
+
+ first_is_auth =
+ router_digest_is_trusted_dir(first->cache_info.identity_digest);
+ second_is_auth =
+ router_digest_is_trusted_dir(second->cache_info.identity_digest);
+
+ if (first_is_auth && !second_is_auth)
+ return -1;
+ else if (!first_is_auth && second_is_auth)
+ return 1;
+
+ else if (first->is_running && !second->is_running)
+ return -1;
+ else if (!first->is_running && second->is_running)
+ return 1;
+
+ bw_first = router_get_advertised_bandwidth(first);
+ bw_second = router_get_advertised_bandwidth(second);
+
+ if (bw_first > bw_second)
+ return -1;
+ else if (bw_first < bw_second)
+ return 1;
+
+ /* They're equal! Compare by identity digest, so there's a
+ * deterministic order and we avoid flapping. */
+ return _compare_routerinfo_by_id_digest(a, b);
+}
+
+/** DOCDOC takes list of routerinfo */
+static digestmap_t *
+get_possible_sybil_list(const smartlist_t *routers)
+{
+ digestmap_t *omit_as_sybil;
+ smartlist_t *routers_by_ip = smartlist_create();
+ uint32_t last_addr;
+ int addr_count;
+ smartlist_add_all(routers_by_ip, routers);
+ smartlist_sort(routers_by_ip, _compare_routerinfo_by_ip_and_bw);
+ omit_as_sybil = digestmap_new();
+
+#define MAX_WITH_SAME_ADDR 3
+ last_addr = 0;
+ addr_count = 0;
+ SMARTLIST_FOREACH(routers_by_ip, routerinfo_t *, ri,
+ {
+ if (last_addr != ri->addr) {
+ last_addr = ri->addr;
+ addr_count = 1;
+ } else if (++addr_count > MAX_WITH_SAME_ADDR) {
+ digestmap_set(omit_as_sybil, ri->cache_info.identity_digest, ri);
+ }
+ });
+
+ smartlist_free(routers_by_ip);
+ return omit_as_sybil;
+}
+
/** DOCDOC */
static void
set_routerstatus_from_routerinfo(routerstatus_t *rs,
@@ -1710,11 +1869,13 @@
rs->is_valid = ri->is_valid;
rs->is_possible_guard = rs->is_fast && rs->is_stable &&
(!rs->is_exit || exits_can_be_guards) &&
- router_get_advertised_bandwidth(ri) >=
- (exits_can_be_guards ? guard_bandwidth_including_exits :
- guard_bandwidth_excluding_exits);
+ (router_get_advertised_bandwidth(ri) >= BANDWIDTH_TO_GUARANTEE