[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
[minion-cvs] Add draft specifications for a nymserver protocol and a...
Update of /home/minion/cvsroot/doc/spec
In directory moria.mit.edu:/tmp/cvs-serv26827/spec
Added Files:
api-spec.txt nym-spec.txt
Log Message:
Add draft specifications for a nymserver protocol and a C API.
Check 'em out?
--- NEW FILE: api-spec.txt ---
$Id: api-spec.txt,v 1.1 2003/07/24 08:02:28 nickm Exp $
MIX3:ClientAPI
Type III (Mixminion) Mix Client API Recommendations
Status of this Document
This is a proposed C API for client functionality for Type III
(Mixminion) mixes. Right now, it's in a very rough state -- it
needs to be audited for completeness fairly thoroughly.
Also, for completeness, there should be companion Java/C++/C#/Perl/
Python APIs as examples for how to wrap Type III mix clients in
these OO languages. I think I'll work on the Python one for
Mixminion 0.0.6, and delay others till the C and Python APIs are
nailed down.
======================================================================
/***
* This header file ("mix3.h") should be included by any C file
* using type III Mix client functionality.
*/
#ifndef _MIX3_H
#define _MIX3_H
/* For size_t */
#include <stdlib.h>
/* For FILE */
#include <stdio.h>
/* ==================== */
/* General-purpose */
/* ==================== */
#ifndef C99
typedef int bool;
#endif
/* We allow the caller to give us a special-purpose allocator -- for
* example, in order to use mlocked memory.
*/
typedef void *(*mix3_allocator)(size_t n);
typedef void (*mix3_deallocator)(void *ptr);
void mix3_set_allocator(mix3_allocator allocator,
mix3_deallocator deallocator);
/* Anything that will be freed by a mix3_* function must be
* mix3_alloc'd; anything that was alloced by a mix3_* function must
* be mix3_free'd.
*/
void *mix3_alloc(size_t n);
void mix3_free(void *ptr);
/* ==================== */
/* Basic crypto */
/* ==================== */
/* XXXX This needs to be specified. */
typedef struct mix3_public_key mix3_public_key;
typedef struct mix3_secret_key mix3_secret_key;
/* ==================== */
/* Error handling */
/* ==================== */
/* Nearly all mix3 functions return a mix3_status code to indicate
* whether they have succeeded -- and if not, why they have failed.
*
* (In another language, we'd use exceptions for this.)
*/
typedef enum {
/* The operation completed successfully. */
MIX3_OK = 0,
/* The operation failed due to lack of memory. */
MIX3_NOMEM,
/* The operation failed because the arguments given to the
* function were invalid.
*/
MIX3_BADARGS,
/* The operation failed because the directory, server
* descriptor, message, etc provided was not correctly
* formatted or signed.
*/
MIX3_BADFORMAT,
/* The operation failed because a provided output structure
* is not long enough.
*/
MIX3_NOSPACE,
/* The operation failed because more fragments need to be
* collected before we can reconstruct the message.
*/
MIX3_NEED_MORE_FRAGMENTS,
/* The network operation will block until the underlying file
* descriptor has data to read. */
MIX3_WANT_READ,
/* The network operation will block until the underlying file
* descriptor is ready for writing. */
MIX3_WANT_WRITE,
/* The network operation failed because the underlying file
* decriptor is closed. */
MIX3_CLOSED,
/* XXXX We need __way__ more error codes. I don't know */
} mix3_status;
/* Effects: Given a mix3_status code, return a English string.
* Allocates: nothing.
*/
const char *mix3_status_string(mix3_status status);
/* ==================== */
/* Directories */
/* ==================== */
/* Opaque struct to hold a Type III directory */
typedef struct mix3_directory mix3_directory;
/* Opaque struct to hold a Type III server descriptor */
typedef struct mix3_serverdesc mix3_serverdesc;
/* Struct to describe a part of a Type III path. A path component may
* either specify a server by nickname (type == MIX3_PC_NICKNAME); provide
* a server explicitly (type == MIX3_PC_DESCRIPTOR); request a number
* of randomly chosen servers (type == MIX3_PC_RANDOM); or specify a
* swap point (type == MIX3_PC_SWAP).
*
* 'name' is used for MIX3_PC_NICKNAME; 'server' is used for
* MIX3_PC_DESCRIPTOR; and 'n' is used for MIX3_PC_RANDOM. If n>=0,
* then exactly n random servers are chosen. If n<0, then
* approximately -n random servers are chosen.
*/
typedef struct mix3_pathcomponent {
enum { MIX3_PC_NICKNAME, MIX3_PC_DESCRIPTOR, MIX3_PC_RANDOM,
MIX3_PC_SWAP } type;
union {
char *name;
mix3_serverdesc *server;
int n;
} info;
} mix3_pathcomponent;
/* Struct to describe a requested Type III path. */
typedef struct mix3_pathspec {
int n_specifiers;
mix3_pathcomponent *specifiers;
} mix3_pathspec;
/* Struct to describe a Type III directory server. */
typedef struct mix3_directory_server {
char *server_name;
char *server_directory_url;
char *server_identity_fingerprint;
} mix3_directory_server;
/* Struct to specify a Type III server's basic capabilities. */
typedef struct mix3_capability_spec {
bool email;
bool news;
bool relay;
} mix3_capability_spec;
/* Struct to specify the destination of a Type III message. */
typedef struct mix3_address_spec {
enum { MIX3_ADDR_EMAIL, MIX3_ADDR_MBOX, MIX3_ADDR_NEWS,
MIX3_ADDR_DROP, MIX3_ADDR_OTHER, MIX3_ADDR_REPLY, } type;
union {
char *address;
struct {
unsigned int routing_type;
int routinginfo_len;
char *routinginfo;
char *opt_exit_server_name;
} raw_address;
} val;
mix3_public_key *recipient_key; /* XXXX describe */
} mix3_address_spec;
/* Effects: given NUL-terminated a string encoding a Type
* III directory parse it into a new directory, and
* point *dir_out at the new directory.
* Arguments:
* dir_out -- A pointer to a pointer to hold the allocated
* directory.
* dir_string -- The string to parse.
* opt_prev_directory -- NULL, or a previous directory to compare
* this one to.
* known_directory_servers -- a NULL-terminated array of pointers
* to known mix3_directory_servers, and is used to check
* signatures.
* validate -- if false, the function does not check the
* directory's signature.
* Allocates: *dir_out, which should be freed with mix_directory_free.
*/
mix3_status mix3_directory_parse_from_string(
mix3_directory **dir_out,
char *dir_string,
mix3_directory_server **known_directory_servers,
mix3_directory *opt_prev_directory,
bool validate);
/* Same as mix3_directory_parse_from_string, but reads from a stdio
* FILE.
*/
mix3_status mix3_directory_parse_from_file(
mix3_directory **dir_out,
FILE *dir_file,
mix3_directory_server **known_directory_servers,
mix3_directory *opt_prev_directory,
bool validate);
/* Returns the number of server descriptors in a directory.
*/
unsigned int mix3_directory_get_n_descriptors(mix3_directory *dir);
/* Returns true iff the provided directory is valid at a given time. */
bool mix3_directory_is_current(mix3_directory *dir, time_t when);
/* Try to download a directory, and store the result in a string.
* This function may block for a while.
*
* Arguments:
* result -- a buffer to hold the downloaded string.
* result_len -- the usable length of the output buffer.
* mix3_directory_server -- the server to use.
*
* XXXX Should we try for an async version of this?
*/
mix3_status mix3_directory_download_to_string(
char *result,
int result_len,
mix3_directory_server *server);
/* Same as mix3_directory_download_to_string, but write the result to
* a stdio FILE.
*/
mix3_status mix3_directory_download_to_file(
FILE *result,
mix3_directory_server *server);
/* Allocate a new NULL-terminated array of server descriptors from a
* directory that match certain properties.
*
* Arguments:
* servers_out -- If the function succeeds, *servers_out will be a
* NULL-terminated array of pointers to server descriptors.
* byNickname -- If not NULL, the function only yields server
* descriptors whose nicknames match byNickname.
* byCapability -- If not NULL, the function only yields server
* descriptors that have at leas tthe capabilities set in
* *byCapability.
* validBy -- If not 0, the function only yields server descriptors
* that become valid at some time before validBy.
* validUntil -- if not 0, the function only yields server
* descriptors that do not become invalid before validUntil.
*
* Allocates: servers_out; it must be freed.
*/
mix3_status mix3_directory_get_servers(
mix3_serverdesc ***servers_out,
char *byNickname,
mix3_capability_spec *byCapability,
time_t validBy,
time_t validUntil);
/* Deallocate a mix3_directory and associated server descriptors. */
void mix3_directory_free(mix3_directory *directory);
/* ==================== */
/* Servers */
/* ==================== */
/* Returns the nickname of a server descriptor. The returned string
* must not be modified, and has the same lifetime as 'desc'. */
const char *mix3_serverdesc_get_nickname(mix3_serverdesc *desc);
/* Returns true iff a provided descriptor has at least given set of
* capabilities. */
bool mix3_serverdesc_has_capabilities(mix3_serverdesc *desc,
mix3_capability_spec *capabilities);
/* Sets *becomes_valid_out and *becomes_invalid_out to match the
* lifespan of 'desc'. */
void mix3_serverdesc_get_lifespan(mix3_serverdesc *desc,
time_t *becomes_valid_out,
time_t *becomes_invalid_out);
/* === Helper functions for server descriptors. === */
/* Return a raw field from a server descriptor, by section and field
* name.
*/
const char *mix3_serverdesc_get_field(mix3_serverdesc *desc,
const char *section,
const char *field);
/* Parse a server descriptor from a string.
*
* Arguments:
* serverdesc_out -- If the function is successful,
* *serverdesc_out will point to a freshly allocated
* serverdesc.
* string -- Before the function is called, *string points to the
* descriptor to parse. On success, after the function is
* called, *string points to the position immediately after the
* server descriptor.
* validate -- If true, check the signature on the descriptor.
*
* Allocates: *serverdesc_out, which must be freed with
* _mix3_serverdesc_free.
*/
mix3_status _mix3_serverdesc_parse(
mix3_serverdesc **serverdesc_out,
char **string,
bool validate);
/* Deallocates a mix3_serverdesc. */
void _mix3_serverdesc_free(mix3_serverdesc *server);
/* Compares two nicknames lexically; returns -1,0,or 1.*/
int mix3_nickname_cmp(const char *name1, const char *name2);
/* Returns true iff name could be a valid nickname. */
bool mix3_nickname_is_valid(const char *name);
/* ==================== */
/* Path-related functionality. */
/* ==================== */
/* Given a string describing a path (e.g., "Foo:Bar,?,Baz,*2,~1"),
* returns a mix3_pathspec.
*
* Allocates: *path_out, which must be freed with mix3_pathspec_free.
*/
mix3_status mix3_path_parse_specifier(
mix3_pathspec **path_out,
char *path);
/* Frees a pathspec allocated by mix3_parse_path_specifier. */
void mix3_pathspec_free(mix3_pathspec *pathspec);
/* Generates a path from a mix3_pathspec.
*
* Arguments:
* path1_out, path2_out -- On success, *path1_out and *path2_out
* are NULL-terminated arrays of pointers to mix3_serverdescs
* from directory.
* directory -- A directory to use as a source of server
* descriptors.
* pathspec -- A description of the path to generate.
* address -- Optionally, the exit address to use. If the exit
* address is a reply block, only path1_out is generated.
* single_leg -- If true, generate the path in a single leg (as for
* a SURB.)
* Allocates: *path1_out and possibly *path2_out, which should be
* passed to mix3_free.
**/
mix3_status mix3_path_generate(
mix3_serverdesc ***path_out1,
mix3_serverdesc ***path_out2,
mix3_directory *directory,
mix3_pathspec *pathspec,
mix3_address_spec *opt_address,
bool single_leg);
/* ==================== */
/* Message-generation functionality */
/* ==================== */
/* A structure to hold a payload and decoding handle for a single
* packet. */
typedef struct mix3_payload {
char decoding_handle[20];
char payload[28*1024];
} mix3_payload;
/* A structure to hold a complete 32KB packet.
*/
typedef struct mix3_packet {
char packet[32*1024];
} mix3_packet;
/* A structure to specify a message header for a SMTP or NEWS message.
*/
typedef struct mix3_header_spec {
enum { MIX3_HDR_FROM, MIX3_HDR_SUBJECT, MIX3_HDR_IN_REPLY_TO,
MIX3_HEADER_REFERENCES } header;
char *value;
} mix3_header_spec;
/* An opaque structure to represent a Type III SURB. */
typedef struct mix3_surb mix3_surb;
/* Generate a set of message payloads from a message, fragmenting,
* compressing, and encoding the message as needed.
*
* Arguments:
* mix3_payload_out -- On success, *mix3_payload_out points to a
* newly allocated array of pointers to mix3_payload.
* n_payloads_out -- *n_payloads_out is set to the number of
* payloads allocated
* message -- the incoming message to encode.
* message_len -- The length of message, in bytes.
* address -- the destination address to which the message will be
* sent.
* headers -- an optional NULL-terminated array of mix3_header_spec.
*
* Allocates: *mix3_payload_out. Each element of *mix3_payload_out[]
* should be passed to mix3_free, followed by *mix3_payload_out
* itself.
*
* XXXX This fails if the message won't all fit into memory with the
* payloads. That's probably okay.
*/
mix3_status mix3_package_message(
mix3_payload ***mix3_payload_out,
int *n_payloads_out,
const char *message,
size_t message_len,
mix3_address_spec *address,
mix3_header_spec **headers);
/* Generate a single mix3_packet from a payload and a pair of paths.
*
* Arguments:
* packet_out -- On success, *packet_out points to newly
* allocated mix3_packet.
* path1 -- A NULL-terminated array of servers to use for the
* first leg of the path.
* path2_opt -- A NULL-termanated array of servers to use for the
* second leg of the path, or NULL if using a SURB.
* surb_opt -- A SURB to use for the final leg of the path, or
* NULL.
* address -- The destination for the packet.
* payload -- An encoded payload as returned by mix3_package_message
*
* Allocates: *mix3_packet, which should be passed to mix3_free.
*/
mix3_status mix3_generate_packet(
mix3_packet **packet_out,
mix3_serverdesc **path1,
mix3_serverdesc **path2_opt,
mix3_surb *surb_opt,
mix3_address_spec *address,
mix3_payload *payload);
/* ==================== */
/* SURB functionality */
/* ==================== */
/* A structure to hold a secret used to generate SURBs for an identity. */
typedef struct mix3_surb_secret {
char secret[20];
} mix3_surb_secret;
/* Generate a SURB.
*
* Arguments:
* surb_out -- On success, *surb_out points to a newly allocated
* mix3_surb.
* path -- A NULL-terminated array of server desc pointers.
* address -- the exit address to use.
* valid_min,valid_max -- An interval during which the SURB should
* be usable.
* secret -- The secret for this SURB identity.
*
* Allocates: *surb_out, which must be freed with mix3_surb_free.
*/
mix3_status mix3_surb_generate(
mix3_surb **surb_out,
mix3_serverdesc **path,
mix3_address_spec *address,
time_t valid_min,
time_t valid_max,
mix3_surb_secret secret);
/* Write a SURB to a string.
*
* Arguments:
* surb_out -- A string buffer to receive a representation of the
* SURB.
* surb_out_len -- The available space in surb_out.
* surb -- the SURB to encode.
* binary -- Use a text encoding iff binary is false.
*/
mix3_status mix3_surb_to_string(
char **surb_out,
int surb_out_len,
mix3_surb *surb,
bool binary);
/* Same as mix3_surb_to_string, but write to a stdio FILE. */
mix3_status mix3_surb_to_file(
FILE *out,
mix3_surb *surb,
bool binary);
/* Read a SURB from a string.
*
* Arguments:
* surb_out -- On success, *surb_out points to the parsed SURB.
* strp -- Before invocation, *strp points to the start of a SURB.
* After successful invocation, *strp points to immediately
* after the end of the parsed SURB.
* str_len -- *str_len is the number of bytes in *strp.
*
* Allocates: *surb_out, which should be freed by mix3_surb_free.
*/
mix3_status mix3_surb_from_string(
mix3_surb **surb_out,
char **strp,
int *str_len);
/* Same as mix3_surb_from_file, but reads from a stdio FILE. */
mix3_status mix3_surb_from_file(
mix3_surb **surb_out,
FILE *file);
/* Deallocate a mix3_surb */
void mix3_surb_free(mix3_surb *surb);
/* XXXX Functions to access the rest of the SURB fields */
/* ==================== */
/* MMTP */
/* ==================== */
/* Opaque structure to represent an MMTP connection -- possibly
* closed */
typedef struct mix3_mmtp_connection mix3_mmtp_connection;
/* Allocates a new mix3_mttp_connection to connect to a given
* server. Performs no network activity.
*
* Arguments:
* out -- On success, *out holds a newly allocated
* mix3_mmtp_connection.
* server -- The server to connect to.
* blocking -- Should operations on this connection be blocking?
* In nonblocking mode, other operations may return
* MIX3_WANTREAD or MIX3_WANTWRITE as status.
*
* Allocates: *mix3_mmtp_connection, which must be freed with
* mix3_mmtp_connection_free.
*/
mix3_status mix3_mmtp_new_connection(
mix3_mmtp_connection **out,
mix3_serverdesc *server,
bool blocking);
/* XXXX What to do about DNS, if we ever do DNS. */
/* Return the server descriptor associated with a connection.
*/
mix3_serverdesc *mix3_mmtp_connection_get_serverdesc(
mix3_mmtp_connection *conn);
/* Return the filedes associated with a connection. */
int mix3_mmtp_connection_get_socket(mix3_mmtp_connection *conn);
/* Try to connect to an MMTP server and handshake with it. Return
* MIX3_OK on success, MIX3_WANT* if a nonblocking connection would
* block.
*/
mix3_status mix3_mmtp_connect(mix3_mmtp_connection *conn);
/* Send a packet to a connected MMTP server. If the operation blocks,
* it must be called again later with the same arguments.
*
* Return MIX3_OK on success, MIX3_WANT* if a nonblocking connection
* would block.
*/
mix3_status mix3_mmtp_send_packet(mix3_mmtp_connection *conn,
mix3_packet *packet);
/* Send padding to a connected MMTP server. If the operation blocks,
* it must be called again later with the same arguments.
*
* Return MIX3_OK on success, MIX3_WANT* if a nonblocking connection
* would block.
*/
mix3_status mix3_mmtp_send_padding(mix3_mmtp_connection *conn,
mix3_packet *padding);
/* Renegotiate a key with a connected MMTP server. Returns as above.
*/
mix3_status mix3_mmtp_renegotiate(mix3_mmtp_connection *conn);
/* Shut down an MMTP connection. Returns as above. */
mix3_status mix3_mmtp_close(mix3_mmtp_connection *conn);
void mix3_mmtp_connection_free(mix3_mmtp_connection *conn);
/* ==================== */
/* Decoding packets */
/* ==================== */
/* Opaque structure holding a decoded (but not yet decompressed or
* reassembled) packet. */
typedef struct mix3_decoded_packet mix3_decoded_packet;
/* Decode a single type III packet.
*
* Arguments:
* packet_out -- On success, *packet_out holds a freshly allocated
* mix3_decoded_packet.
* inp -- An input string to read a packet payload from. Advanced
* on success.
* inp_len -- The length of the input string. Decremented on
* success.
* is_text -- Flag: is the input ascii-armored.
* keys -- NULL-terminated array of secret keys to try decrypting with.
* secrets -- NULL-terminated array of SURB secrets to try
* decrypting with.
*
* Allocates: mix3_decoded_packet, which must be freed with
* mix3_decoded_packet_free.
*/
mix3_status mix3_packet_decode(mix3_decoded_packet **packet_out,
const char **inp,
int *inp_len,
bool is_text,
mix3_secret_key **keys,
mix3_surb_secret **secrets);
/* Return true iff pkt is a fragment. */
bool mix3_decoded_packet_is_fragment(mix3_decoded_packet *pkt);
/* Return true iff packet was unencrypted */
bool mix3_decoded_packet_was_plaintext(mix3_decoded_packet *pkt);
/* If pkt was encrypted with a surb, return the index of the identity
* key among the array of secrets passed to mix3_packet_decode.
* Otherwise return -1. */
int mix3_decoded_packet_reply_idx(mix3_decoded_packet *pkt);
/* If pkt was encrypted with a public key, return the index of the
* secret key key among the array of secret keys passed to
* mix3_packet_decode. Otherwise return -1. */
int mix3_decoded_packet_key_idx(mix3_decoded_packet *pkt);
/* Struct to hold a message ID for a fragmented message.
*/
typedef struct mix3_fragment_message_id {
char id[20];
} mix3_fragment_message_id;
/* Identity of a fragment within a fragmented message */
typedef unsigned long mix3_fragment_fragment_id;
/* Return the length of the (compressed) message corresponding to
* pkt. */
size_t mix3_decoded_packet_get_length(mix3_decoded_packet *pkt);
/* Gets the message id for a fragment in pkt. */
mix3_status mix3_decoded_fragment_get_mid(mix3_fragment_message_id *out,
mix3_decoded_packet *pkt);
/* Gets the fragment id for a fragment in pkt. */
mix3_fragment_fragment_id mix3_decoded_fragment_get_fid(
mix3_decoded_packet *pkt);
/* Extracts a message from a singleton packet.
* XXXX DOCDOC
*/
mix3_status mix3_decoded_packet_get_msg(char *message_out,
int message_out_len,
mix3_decoded_packet *pkt);
/* Extracts a message from a set of packets.
* XXXX DOCDOC
*/
mix3_status mix3_decoded_fragments_get_msg(char *message_out,
int message_out_len,
int n_fragments,
mix3_decoded_packet **fragments);
void mix3_free_decoded_packet(mix3_decoded_packet *pkt);
#endif /* _MIX3_H */
======================================================================
--- NEW FILE: nym-spec.txt ---
$Id: nym-spec.txt,v 1.1 2003/07/24 08:02:28 nickm Exp $
Mix3:Nym
Underhill: A Proposed Type 3 Nymserver Protocol Specification
Nick Mathewson
Status of this document
This document is an initial draft for a specification of nymservers
designed to work with Type III (Mixminion) remailers. It has not
yet been reviewed by anybody.
This protocol is tentatively codenamed "Underhill", until somebody
comes up with a more descriptive name. Who knows; maybe it will
stick.
Abstract
XXXX write me.
Acknowledgments
Thanks are due to George Danezis and Roger Dingledine, and
everybody else who has worked on the Type III remailer
specification, especially the members of the Mixmaster team,
including Len Sassaman and Peter Palfrader.
The idea of using SURBs to implement a conversation between
nymserver and nymholder was hit upon at a meeting between me,
Roger, George, and David Mazieres. [XXXX who else?] The
necessity of a POP or IMAP-like protocol was [XXXX AFAICR] first
suggested there.
Earlier drafts and sketches of nymserver protocols for a type III
remailer network have been written by George Danezis and Peter
McIntyre. This document incorporates certain elements of their
designs.
Table of Contents
Status of this Document X
Abstract
Acknowledgments
Table of Contents
1. Introduction
1.1. Roles
1.2. Terminology
1.3. Design goals
1.3.1. Non-goals
1.4. Overview
2. Nymserver operation
2.1. Information associated with each nym
2.2. Processing incoming emails
2.3. Relaying emails to the user
2.4. Deleting emails
3. Nymholder client operation
4. Message formats
4.1. Generating synopses
4.2. Message encryption
4.3. Control message format: messages to the nymserver
4.3.1. CREATE [0x00]
4.3.2. CREATE2 [0x01]
4.3.3. SURB [0x02]
4.3.4. NEWPK [0x03]
4.3.5. RELAY [0x04]
4.3.6. GET [0x05]
4.3.7. SUMMARIZE [0x06]
4.3.8. DELETE [0x07]
4.3.9. POLICY [0x08]
4.4. Control message format: messages to the nymholder
4.4.1. CREATED [0x00]
4.4.2. STATUS [0x01]
4.4.3. SUMMARY [0x02]
4.4.4. MSGS [0x03]
4.4.5. DROPPED [0x04]
4.4.6 ERROR [0x05]
5. Filtering and abuse prevention
1. Introduction
A nymserver provides long-term pseudonymous identities.
As in previous nymserver designs, this protocol uses an anonymity
network to relay messages from the holder of a pseudonym to a
nymserver, and uses reply blocks to send messages from the
nymserver back to the holder of the pseudonym. Unlike previous
designs, this one uses Single-Use Reply Blocks (SURBs) as provided
by the Type III anonymity network design.
Because we use SURBs, the nymserver can no longer relay an
arbitrarily large number of emails to the nymholder with a single
reply block. Instead, the nymholder must periodically send a fresh
supply of SURBs to the nymserver. If the nymserver runs out of
SURBs, it must hold messages until the client can retrieve them.
To minimize the volume of messages sent over the anonymity network,
and reduce the traffic-analysis impact of a flooding or spamming
attack, we allow clients to view synopses of emails, and
selectively download only the ones they choose and delete the rest.
This explicit email selection is used only when accounts are
heavily loaded --- when traffic is light, the nymserver relays all
email automatically after batching them into type III packets.
When traffic becomes heavier, the nymserver sends only synopses of
incoming emails. When the nymserver runs out of SURBs entirely,
the nymholder sends control messages to explicitly retrieve the
synopses and emails she wants.
The nymserver tries to provide a reliable service on top of an
unreliable message protocol. Thus, when the nymserver has the
space to do so, it holds emails that it has relayed until they have
been acknowledged by their recipients. When the nymholder sends an
important control message to the nymserver, it MAY choose to
remember the message until the nymserver has acknowledged it.
1.1. Roles
Nymserver - A piece of software running on a well-known host.
Provides pseudonymous identities.
Nym - The pseudonymous identity of a user.
Nymholder - The user who controls a given nym.
Sender - A user who sends email to a nym
Recipient - A user who receives email sent by a nym.
1.2. Terminology
Control Message - A message sent via Type III remailers; either
from the nymserver to the nymholder, or from the nymholder to
the nymserver.
Email - A message relayed via a nymserver; either from a sender to
the nymholder, or from the nymholder to a recipient or set or
recipients.
Packet, SURB - As used in the Type III remailer specification.
Additionally, we use below a number of cryptographic operations
described more fully in the type III remailer specification.
1.3. Design goals
Pseudonymity:
- There should be no feasible way for an attacker to gain
information about the identity of the holder of a given nym,
even if the attacker compromises or controls the nymserver.
- There should be no feasible way for an attacker to learn
which nyms a given user holds, even if the attacker
compromises or controls the nymserver.
- If the nymserver is not compromised, only a nymholder should
be able to receive emails sent to a given nym.
- If the nymserver is not compromised, only a nymholder should
be able to send emails originating from a given nym.
Message secrecy
- The protocol should be compatible with systems such as
OpenPGP and SMTP/TLS that seek to prevent an attacker from
eavesdropping on mail. When possible, the protocol should
facilitate use of these systems.
No software for non-users
- Users who do not receive pseudonymity from the system should
not need to run any software in order to send emails to nyms
or receive emails from nyms.
- Additionally, users can shut down their accounts without
special software by sending a shutdown phrase to the
nymserver via ordinary or anonymous email.
Forward security (limited)
- If an attacker compromises a nymserver, the attacker should
gain as little additional information as possible about
previous use of the system.
Availability
- Nymholders should receive their emails in as timely a manner
as is possible given their security requirements.
- The system should be resilient to spam, abuse, and DoS
attacks.
1.3.1. Non-goals
The following properties are *not* goals or requirements of this
protocol.
- Full transport neutrality.
(Although this protocol does not require a Type III mix
network specifically, it assumes that its underlying transport
has many properties of a Type III mixnet. Specifically, it
assumes the existence of a high-latency, message-based open
anonymity network with single-use reply blocks. It further
assumes that the underlying transport prevents forward
messages to a given remailer from being received or decrypted
by an attacker; and that the transport also prevents replies
from being received or decrypted by anybody but the SURB
generator.)
- Minimal nymserver storage requirement
(This protocol assumes that nymservers will have to devote a
certain amount of space to each nym in order to store SURBs
(under ordinary load) and pending messages (under high load).
Although we allow nymservers to drop pending messages, there
seems no way to provide reliable service to nymholders under
high load without storing at least some messages on the
nymserver.)
- Latency-free retransmission.
- Design minimalism.
1.4. Overview
In this design, a Nymserver provides pseudonymous identities to its
users as follows:
- Internally, it associates an identity public key with each
pseudonym. Nymholders anonymously send control messages to the
nymserver and authenticated with their private identity key.
- To communicate anonymously with the nymserver, the nymholder
uses the Type III remailer network to send its control messages
to the nymserver. Every nymserver acts as a Type III Remailer
for the purposes of accepting Incoming/MMTP. A nymserver need
not support Outgoing/MMTP, or any delivery type but 'NYM3'.
(Because Type III messages to a Type III node are encrypted to
the node's public key, no further encryption is needed to
ensure that only the nymserver can read messages from the
nymholder.)
(Because Type III is replay-proof, no further replay
prevention is needed.)
- To allow the nymserver to communicate back to the anonymous
nymholder, the nymholder provides the nymserver with a stock of
SURBs, and replenishes them periodically.
(Because the nymholder uses a separate SURB identity secret for
each pseudonym it generates, it can distinguish reply messages
sent to these SURBs from any other messages it might receive.
Thus, because only the nymserver receives these SURBs, no
further authentication from the nymserver to the nymholder is
needed. Because replies can only be read by the user who
generated the SURB, no further encryption is needed from the
nymserver to the nymholder.)
(Because Type III is replay-proof, no further replay
prevention is needed.)
- To send a pseudonymous email to a given recipient, a nymholder
sends a control message to the nymserver, containing the text
of the email to send. The nymserver checks the authentication,
and relays the email as appropriate.
[XXXX Should there be some way for the nymholder to
authenticate to the final recipient? I'd say, "Just use
PGP", but PGP distinguishability is an issue.]
- Nymservers accept emails to nyms from senders via SMTP and
SMTP/TLS, and relay them to the appropriate nymholders.
- Because a SURB can only be used once, nymservers try to send as
few messages as possible to users. They do this by batching
multiple incoming emails into single Type III packets.
- To prevent flooding attacks against nymholders, the nymserver
does not automatically relay all emails to their user.
Nymservers support incoming email blocking, configured by the
holder of each nym.
- To reduce message load, but still retain performance under
load or DoS attacks, the nymserver and client follow different
behavior patterns under different loads. (See section 1 above
for a description.)
- To minimize the impact of a compromise, a nymserver encrypts
all incoming email upon receipt. Below, we specify a way for
the nymserver to do so while at the same time allowing users
to receive header synopses, and disguising email sizes.
[XXXX There's a neat way to hide linkage between emails sent
to the same identity, but it makes a lot of other stuff really
hard to do. Thus, I'm not going to recommend it.]
2. Nymserver operation
2.1. Information associated with each nym
For each nym, a nymserver maintains the following information:
[User adjustable options]
- Identity public key of the nymholder
- Encryption public key of the nymholder
- A filtering policy (described below)
- A maximum email latency, in 30-minute increments. (described below)
- A maximum synopsis latency, in 30-minute increments. (described
below)
- A maximum SURB usage rate, in SURBs-per-day.
- A maximum unencrypted synopsis time, in 30-minute
increments. (described below)
- A hold-until-ack level (one of "Never", "Quota", or "Always")
- The hash of a 'shutdown' passphrase
[XXXX I'm not so sure that it's a good idea to have so many
user-adjustable options, but I can't think of a single policy
that allows both security and any reasonable efficiency.
Others may be able to strike a better balance.]
[Administrator adjustable options]
- Maximum number of queued emails
- Maximum email size
- Maximum disk space used
Additionally, the nymserver stores on behalf of the user:
- A set of emails, each consisting of:
An encrypted synopsis
An encrypted emails
Each message also has:
A 32-byte message identifier
A 'sent' tristate: "Nothing sent", "Synopsis-sent", "Sent-In-Full".
A time of receipt.
- A set of SURBs
2.2. Processing incoming emails
When a nymserver receives an incoming email, it follows these
steps:
1. The nymserver decides whether to accept the email. If any
of the following apply, the nymserver rejects the email:
- The email is not addressed to a valid nym.
- The email violates the nymserver's abuse or spam policies.
- Accepting the email would violate the nym's quota.
- The email is rejected by the nym's filtering policies.
[XXXX when do we send a bounce?]
2. The nymserver then forms a synopsis of the email; generates a
random 20-octet message ID for the email; encrypts the email
for receipt by the nymholder, and decrements the email's size
from the user's available quota.
2.3. Relaying emails to the user
Later, the nymserver will send a synopsis of a pending message to a
nymholder if:
- The nymholder has requested the synopsis, and the user has any
stored SURBs.
- The nymholder's maximum synopsis latency has passed, and there
are enough SURBs to send the synopsis to the nymholder with
at least N [XXXX 2?] left over, and sending the message would
not exceed the max-surbs-per-day rate.
- The nymserver is sending a control message to the nymholder,
and there is enough unused space in the message's packets to
include the synopsis, and the synopsis is older than any
currently unsent synopsis.
After sending a synopsis, the nymserver marks the corresponding
email as 'synopsis-sent'.
Also, the nymserver will send the email to the nymholder if any
of the following conditions hold:
- The user has requested the email, and there are enough SURBs
stored for the user to send the email.
- The user's maximum email latency has passed since the email
arrived, and there are enough SURBs to send the email to the
nymholder with at least 5 left over, and sending the message
would not exceed the max-surbs-per-day rate.
- The nymserver is sending a control message to the nymholder,
and there is enough unused space in the message's packets to
include the email, and the email is older than any other email
that could be sent.
After sending an email to a nymholder, the nymserver marks that
as 'sent-in-full'.
If the rules above allow either email or synopses to be sent, first
priority is given to sending the user some information about every
email, and second priority is given to sending the most information
possible about each email. (Thus, if there is room for every email
to be sent in full, every email is sent in full and no synopses are
sent. But if only some emails can be sent in full, it is better
to synopsize as many emails as possible, and then include as many
emails as fit.)
[XXXX I need to clean this section up--it's more verbose and
byzantine than it needs to be.]
2.4. Deleting emails
A nymserver deletes a stored email if any of the following reasons apply:
- The nym's account has been closed.
- The nymholder has requested that the email be deleted.
- The email's state is "Sent-In-Full", and the nym's
hold-until-ack level is "Never".
- The email's state is "Sent-In-Full", and deleting the email
would free enough room to allow the nymserver to accept an
incoming email for the nym, and the nym's hold-until-ack level
is "Quota". (When choosing between multiple emails in this
case, the nymserver deletes the oldest.)
3. Nymholder client operation
[XXXX This section is grossly underwritten.]
In this design, nymholders must run client software in order to
decode control messages from the nymserver; to resupply the
nymserver with SURBs; and to send other control messages to the
server.
When the load is light enough for the server to relay all emails to
the nymholder, the client simply replenishes the nymserver's SURB
pool when it is low, and tells the server to delete all the
messages it receives.
Under heavier load, the client retrieves synopses explicitly as
needed; presents the user with a list of known retrievable
messages, and requests that the nymserver retrieve/delete messages
as appropriate.
When sending an 'update' command to the nymserver (for example, to
change the nymholder's public key or configuration options), the
client keeps a copy of the command in local storage until the
server has acknowledged it.
4. Message formats
4.1. Generating synopses
The 'synopsis' of an email is an RFC822-formatted octet sequence
containing a subset of the email's RFC822 headers, and the first
128 characters of the email body. The following headers are
included in the synopsis, if they present:
Cc
From
Date
In-Reply-To
Message-Id
References
Subject
To
X-Anonymous
X-Spam-Level
The final received header (added by the nymserver's MTA) is also
included, as well as a 'X-Octets' header indicating the total size
of the email. [XXXX is there a standard way to do this?]
To prevent DoS attacks, only the first 80 characters of any header
are included in the synopsis.
4.2. Message Encryption
After synopsizing an email, the nymserver encrypts it immediately
with the nymholder's private key.
If a nymserver holds a set of synopses for longer than the
nymholder-specified length of time, it encrypts those synopses with
the nymholder's private key.
To encrypt an octet sequence, the nymserver first compresses the
octet sequence (as described in E2E-spec.txt). Next, the nymserver
pads the octet sequence to the nearest multiple of 128 octets in
length. The nymserver then generates a random 128-bit key;
LIONESS-encrypts the padded compressed data with the key; and
prepends to the encrypted data the RSA-encrypted key. We use the
same trick as minion-spec.txt to minimize wasted space.
PROCEDURE: Encrypt an octet sequence to a nymholder.
INPUTS:
PK_nym -- The nymholder's encryption public key
M -- The octet sequence to encrypt
P -- The padding granularity -- either 1024 or 128.
Let M_C = COMPRESS(M).
Let PADDING_LEN = CEIL(LEN(M_C)/P) - LEN(M_C).
Let M_P = M | Z(PADDING_LEN).
Let K = R(16).
Let M_Enc = SPRP_Encrypt(K, "", M_P)
Let RSA_LEN = Len(PK_nym) - OAEP_OVERHEAD - 16
Let RSA_PART = PK_Encrypt(PK_nym, K | M_Enc[0:RSA_LEN])
Return RSA_PART | M_Enc[RSA_LEN:Len(M_Enc)-RSA_LEN]
For efficiency reasons, a nymserver tries to encrypt several
synopses at once, so that zlib can compress them effectively.
(This can increase by a factor of 2 to 3 the number of synopses
that fit in a single type III packet.) There is a tension here
between forward security (which would mandate encrypting each
synopsis as soon as possible) and between efficiency and
traffic-analysis resistance (which would mandate minimizing the
number of packets sent between client and server).
We strike the following compromise: a nymserver encrypts synopses
for a user whenever it has eight or more pending synopses to
encrypt, or whenever a synopsis has waited without encryption for a
given interval of time, or whenever the synopses are about to be
sent to the nymholder.
When encrypting a group of synopses, the nymserver first packages them
by prepending to each one the following 22 octet header, and then
concatenating the synopses.
ID [20 octets] (A randomly generated message ID.)
LEN [2 octets] (Length of the synopsis, in octets.)
(Note that the nymserver must associate with each encrypted set of
synopses the message ID of every synopsis, in order, contained
therein. It can only delete the set of synopses when it has
deleted every corresponding email.)
4.3. Control message format: messages to the nymserver
This section describes the control messages sent from the
nymholder to the nymserver.
A control message is a signed list of commands from the nymholder
to the nymserver. Commands are identified by command type, and are
of variable length.
All of the control messages described below follow the following
format.
Header:
SIG Signature (PK_LEN=256 octets)
NL Nym Length (1 octet)
NYM Nym (variable length)
SEQNO Sequnce # (20 octets)
Body:
Sequence of:
CT Command type (1 octet)
CS Command data size (3 octets)
CD Command data (variable length)
The 'Signature' field is equal to the RSA-OAEP+ signature of a
SHA-1 hash of the remainder of the message. The NL field is equal
to the length of NYM. The NYM field is equal to the Nym to which
these commands apply. The SEQNO field holds a random value used
by the nymserver later to acknowledge this message.
The value of each 'CS' field must be the big-endian representation
of the size of the immediately following CD field.
The following values for CT are recognized:
0x00 : CREATE (Create a new nym)
0x01 : CREATE2 (Confirm a new nym)
0x02 : SURB (Supply the nymserver with new set of SURBs)
0x03 : NEWPK (Set or replace public keys)
0x04 : RELAY (Send an email to a recipient)
0x05 : GET (Retrieve a list of emails)
0x06 : SUMMARY (Retrieve a list of synopses)
0x07 : DELETE (Expunge a list of emails from the nymserver)
0x08 : POLICY (Adjust user settings)
4.3.1. CREATE [0x00]
A CREATE command begins the nym creation handshake.
The body of a CREATE command has the following structure:
NNym Number of candidate nyms (1 octet)
PW Proof of work (??? octets)
Sequence of:
NL Candidate Nym Length (1 octet)
Nym Candidate Nym (variable length)
(To create a new Nym, a nymholder send a new control message
containing a CREATE command, a NEWPK command, and a SURB command
with at least 3 SURBs. This initial control message should have an
empty (0-octet) nym, and should be signed with the identity key
given in the NEWPK command. The nymserver replies with a CREATED
command, which the nymholder confirms with a CREATE2 command.)
[XXXX specify a proof-of-work system.]
4.3.2. CREATE2 [0x01]
A CREATE2 command creates the nym creation handshake.
The body of a CREATE2 command has the following structure:
CR Response to challenge (variable length)
4.3.3. SURB [0x02]
A SURB command provides one or more SURBs for the server to contact
the client.
The body of a SURB command has the following structure:
SURB (variable length)
If the body of the SURB command is empty, the nymserver deletes
all currently held SURBs.
4.3.4. NEWPK [0x03]
A NEWPK command sets the nymholder's public keys at the server.
The body of a SURB command has the following structure:
Identity key length (2 octets)
Identity key (variable length)
Encryption key length (2 octets)
Encryption key (variable length)
The key length fields MUST be 128 or 256. The key fields hold
ASN.1 encoded RSA public keys. Their exponents must be 65537.
Because the nymserver may not receive the message, the nymholder
should continue to sign commands with its previous identity key
until the new one has been acknowledged, and accept messages
encrypted with the previous encryption key until a given amount of
time has elapsed.
4.3.5. RELAY [0x04]
A RELAY command tells the nymserver to send an email from the nym.
The body of a RELAY command has the following structure:
Destination:
RS Routing Size (2 octets)
RT Routing Type (2 octets)
RI Routing Info (Variable length; RS=Len(RI))
Message
BL Body length (4 octets)
BODY Email body (Variable length; BL=Len(BODY))
The routing fields are as in "minion-spec.txt". The email body
is prefixed with headers as in "E2E-spec.txt", but is otherwise
unencoded. When the server delivers the email, it adds a From
line with the correct nym mailbox, and sets the name as given in
the headers.
4.3.6. GET [0x05]
A GET command requests that the server transfer a set of emails to
nymholder.
The body of a GET command has the following structure:
Sequence of:
MSGID (20 octets)
4.3.7. SUMMARIZE [0x06]
A SUMMARIZE command requests that the server transfer a set of
summaries to the client.
The body of a SUMMARIZE command has the following structure:
NUM (2 octets)
AFTER (20 octets)
The NUM field specifies a maximum number of summaries to retrieve;
the AFTER field requests that only summaries newer than the
message ID specified be returned.
4.3.8. DELETE [0x07]
A DELETE command tells the server to delete a number of specified
emails, by ID.
The body of a DELETE command has the following structure:
Sequence of:
MSGID (20 octets)
4.3.9. POLICY [0x08]
A POLICY command sets a nymholder-specified configuration option
on the nymserver. The body of a POLICY command has the following
structure:
OPTION_LEN (1 octet)
OPTION_NAME (variable length; len(OPTION_NAME) = OPTION_LEN.)
VALUE (variable length)
Recognized options include:
"SendMsgAfter" -- max time to hold a sendable email without
sending, when enough SURBs are held.
"SendSummaryAfter" -- max time to hold sendable synopses
without sending, when low on SURBs.
"EncryptSummaryAfter" -- max time to hold summary without encrypting
"MaxSURBsPerDay" -- maximum number of SURBs to use for
automatically generated traffic in a single day
"HoldSURBs" -- minimum # of SURBs to hold for status messages.
"HoldUntilAck" -- 'never', 'quota', 'always'.
"ShutdownPhrase" -- Hash of a passphrase useable to shutdown the
nym.
"Filtering" -- specify a filtering policy; see section 5 below.
Times are specified as a decimal number of minutes; they may be
rounded up to the nearest half hour by thhe server. A time of "0"
signifies "immediately", a time of "-1" signifies "never".
All other numbers are encoded in decimal.
4.4. Control message format: messages to the nymholder
This section describes the control messages sent from the nymserver
to the nymholder.
A control message is a list of commands from the nymserver to the
nymholder nymserver. Commands are identified by command type, and
are of variable length.
All of the control messages described below follow the following
format.
Sequence of:
CT Command type (1 octet)
CS Command data size (3 octets)
CD Command data (variable length)
The NYM field is equal to the Nym to which these commands apply.
The NONCE field holds a random value used by the nymserver later to
acknowledge this message.
The value of each 'CS' field must be the big-endian representation
of the size of the immediately following CD field.
The following values for CT are recognized:
0x00 : CREATED (Acknowledge a new nym)
0x01 : STATUS (Acknowledge messages and describe status)
0x02 : SUMMARY (Describe pending emails)
0x03 : MSGS (Send emails to the client)
0x04 : DROPPED (Tell the client about dropped emails)
0x05 : ERROR (Tell the client about an error condition)
4.4.1. CREATED [0x00]
A CREATED command acknowledges a client's CREATE or CREATE2
command. A CREATED command tells the client which Nym was
assigned, and (optionally) demands a challenge response from the
client before opening the Nym.
The body of a CREATED command has the following structure:
NL Length of the assigned nym (1 octet)
NYM Assigned nym (variable length)
CHALLENGE Optionally, a challenge (Variable length)
[XXXX The challenge-response protocol is as yet unspecified.]
4.4.2. STATUS [0x01]
A STATUS command tells the nymholder about the state of the
nymserver _after_ the current control message is processed. A
nymserver SHOULD send a STATUS command in every possible control
message.
A status command has the following structure:
N_MSGS Number of emails waiting (4 octets)
N_SURBS Number of SURBs available (2 octets)
QUOTA Maximum storage on nymserver, in bytes. (4 octets)
USED Currently used storage on nymserver, in bytes. (4 octets)
Sequence of:
SEQNO Nonce of acknowledged client commands (20 octets)
A nymserver MUST acknowledge every client control message at least
once, and MAY acknowledge a client control message more than once.
When sending a control message to the client, the nymserver SHOULD
acknowledge every client control message that it has received but
not yet acknowledged.
If a control message with an important state-changing command (such
as NEWPK or POLICY) goes a long time without acknowledgment, the
client SHOULD re-send the command in a later control message.
4.4.3. SUMMARY [0x02]
A SUMMARY command tells the client about a set of one or more
waiting emails. Because the synopsis for a waiting email may be
encrypted together with synopses for emails that have already been
deleted, each SUMMARY includes a bitfield to identify which
synopses correspond to available messages.
A SUMMARY command has the following body structure:
VALID_BF Bitfield: which entries in ES have an email? (2 octets)
ES Encrypted set of synopses. (variable length)
The LSB in Valid_BF corresponds to the first synopsis in ES, and so
on.
4.4.4. MSG [0x03]
A MSG command relays an email to the client.
The body of a SUMMARY command has the following structure.
MSGID Message ID (20 octets)
MSG Encrypted email (variable length)
Section 2.3 describes circumstances under which a nymserver
generates MSG commands.
4.4.5. DROPPED [0x04]
A DROPPED command tells the nymholder that the nymserver has
discarded a list of emails, either at the nymholder's request or
for another reason.
Sequence of:
MSGID Message ID (20 octets)
Section 2.4 describes circumstances under which a nymserver
generates DROPPED commands.
4.4.6. ERROR [0x05]
NONCE Nonce from client message; Z(20) if none. (20 octets)
ERROR English-language error message (variable width)
5. Filtering and abuse prevention
[XXXX write more here. We should provide some built-in means for
operators to block incoming spam and floods. We should also
allow users to block messages that they don't want to receive,
perhaps by sender or subject line or size or spamassassin status
or something.
Note that blindly allowing regex support is probably a _bad_
idea; it's not hard to find pathological input texts that make
perl-style backtracking regex engines behave very badly.]
X. Open issues
- Should there be some versioning here?
- We need some kind of entry format for nymservers' server
descriptors.
- What is in George's and Peter's nymserver specifications that I
missed?
-