[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
[or-cvs] r15174: Wrote results section (with all results we got so far) and r (projects/hidserv/trunk/doc)

To: or-cvs@xxxxxxxxxxxxx
Subject: [or-cvs] r15174: Wrote results section (with all results we got so far) and r (projects/hidserv/trunk/doc)
From: kloesing@xxxxxxxx
Date: Thu, 12 Jun 2008 20:13:38 -0400 (EDT)
Delivered-to: archiver@xxxxxxxx
Delivered-to: or-cvs-outgoing@xxxxxxxx
Delivered-to: or-cvs@xxxxxxxx
Delivery-date: Thu, 12 Jun 2008 20:13:42 -0400
Reply-to: or-dev@xxxxxxxxxxxxx
Sender: owner-or-cvs@xxxxxxxxxxxxx
Author: kloesing
Date: 2008-06-12 20:13:38 -0400 (Thu, 12 Jun 2008)
New Revision: 15174

Added:
   projects/hidserv/trunk/doc/duration.png
   projects/hidserv/trunk/doc/opentime.png
   projects/hidserv/trunk/doc/publtime.png
   projects/hidserv/trunk/doc/reqtime.png
   projects/hidserv/trunk/doc/restime.png
Modified:
   projects/hidserv/trunk/doc/report.pdf
   projects/hidserv/trunk/doc/report.tex
Log:
Wrote results section (with all results we got so far) and refined description of measurement setup.

Added: projects/hidserv/trunk/doc/duration.png
===================================================================
(Binary files differ)


Property changes on: projects/hidserv/trunk/doc/duration.png
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream

Added: projects/hidserv/trunk/doc/opentime.png
===================================================================
(Binary files differ)


Property changes on: projects/hidserv/trunk/doc/opentime.png
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream

Added: projects/hidserv/trunk/doc/publtime.png
===================================================================
(Binary files differ)


Property changes on: projects/hidserv/trunk/doc/publtime.png
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream

Modified: projects/hidserv/trunk/doc/report.pdf
===================================================================
(Binary files differ)

Modified: projects/hidserv/trunk/doc/report.tex
===================================================================
--- projects/hidserv/trunk/doc/report.tex	2008-06-12 22:39:13 UTC (rev 15173)
+++ projects/hidserv/trunk/doc/report.tex	2008-06-13 00:13:38 UTC (rev 15174)
@@ -1,4 +1,7 @@
 \documentclass{article}
+\usepackage[dvips]{graphicx}
+\usepackage{graphics}
+\usepackage{color}
 \usepackage{url}
 \begin{document}
 \title{Tor Hidden Service Performance Analysis}
@@ -13,7 +16,7 @@
 \end{abstract}
 
 \section{Motivation}
-Tor Hidden Services\footnote{\url{https://tor-svn.freehaven.net/svn/tor/trunk/doc/design-paper/tor-design.pdf}}
+Tor Hidden Services
 allow users to set up anonymous information services, like websites, that
 can only be accessed through the Tor network and are protected against
 identification of the host that runs the services. The most critical
@@ -23,37 +26,39 @@
 Tor protocol, the connection to a new hidden service can take several
 minutes, which leads most users to give up before the connection has been
 established. Using Tor Hidden Services for direct interactive user-to-user
-communication (e.g. messaging) is nearly impossible due to the high latency
-of hidden service circuit setup. 
+communication (e.g.\ messaging) is nearly impossible due to the high
+latency of hidden service circuit setup. 
 
 This document describes measurements of setting up and accessing a Tor
 Hidden Service. The first part of these measurements consist of an
-\emph{outside view} of a user experiencing the delay in publishing a hidden
+\emph{external view} of a user experiencing the delay in publishing a
+hidden
 service, establishing a connection to an existing hidden service, sending
 and receiving messages, and the durability of an established connection.
 The intention of the second part of these measurements is to obtain an
 \emph{internal view} of the delay that is introduced by single substeps of
-connection establishing. The following data was collected:
+connection establishing. The following questions shall be answered:
 
 \begin{description}
-\item[Service Publication] How long does it take from starting Tor
-configured to provide a hidden service until the hidden service is
-advertised in the network and accessible by clients?
+\item[Service Publication] How long does it take from a starting Tor
+process that is configured to provide a hidden service until the hidden
+service is advertised in the network and accessible by clients?
 \item[Connection Establishment] How long does it take for a client to open
 a connection to an existing hidden service. As this includes multiple
 substeps, there are two distinct measurements to answer this question, one
 measuring the overall connection establishment time and one measuring the X
-internal substeps\footnote{TODO Christian: How many substeps are there?}
+internal substeps\footnote{\emph{TODO Christian: How many substeps are
+there in your setup?}}
 that are involved in this process.
 \item[Message Transfer] What is the delay of transferring messages from
 client to server and back over an established connection?
 \item[Connection Durability] How long do connections persist before
-breaking due to some network failure?
+breaking?
 \end{description}
 
 \section{Measurement Setup}
 
-\subsection{Outside View Measurements}
+\subsection{External View Measurements}
 
 The first set of measurements aims at the user experience of a hidden
 service provider or client requesting an existing hidden service. The setup
@@ -61,9 +66,10 @@
 process} that is configured to provide a hidden service and the other one
 being the \emph{client-side Tor process} that is used to access the hidden
 service. 15 minutes after starting the two Tor processes, two more
-processes are started: a \emph{server process} acts as hidden server and
-listens for requests from the server-side Tor process and a \emph{client
-process} uses the client-side Tor process to establish a connection to the
+processes are started: a \emph{server process} that acts as hidden server
+and listens for requests from the server-side Tor process and a
+\emph{client process} that uses the client-side Tor process to establish a
+connection to the
 hidden service. The client process, after establishing the connection to
 the server process via Tor, sends numbered echo messages every minute
 until the connection breaks (or four hours elapse). The server process
@@ -91,76 +97,302 @@
   descriptor stored"))
 \end{verbatim}
 
-Using these log statements both \emph{initialization time} that is required
-to download enough directory information (here: 9.051 seconds) and
-\emph{publication time} that is used to establish introduction points and
-upload a descriptor (here: 71.727 seconds) can be calculated by subtracting two
-timestamps.
+Using these log statements both, \emph{initialization time} (time that is
+required to download enough directory information; here: 9.051 seconds) and
+\emph{service establishment time} (time that is used to establish
+introduction points and upload a descriptor; here: 71.727 seconds) can be
+calculated by subtracting the appropriate two timestamps.
 
 \paragraph{Connection Establishment}
 
-The connection time---from a user perspective---is the time that it takes
+The connection time (from a user perspective) is the time that it takes
 to open a socket connection on client-side. In the client process this is
 performed by a single line of code (with a previous initialization to use
 the client-side Tor process as SOCKS proxy):
 
 \begin{verbatim}
-socket.connect(hiddenServiceAddress);
+mySocket.connect(hiddenServiceAddress);
 \end{verbatim}
 
 Connection establishment time is measured by firing a log event immediately
 before and after this code line and subtracting the first timestamp from
-the second.
+the second. The corresponding log statements (created by the client and
+server processes, not by Tor) are: (connection time: 37.679 seconds)
 
+\begin{verbatim}
+FINE event.hidservmeas observeEvent Observed event EventImpl:
+  occurenceTime=21:05:03.158, source="clientapp",
+  type=CLIENT_OPENING_CONNECTION, message="Client opening
+  connection." from source clientapp!
+FINE event.hidservmeas observeEvent Observed event EventImpl:
+  occurenceTime=21:05:40.837, source="clientapp",
+  type=CLIENT_OPENED_CONNECTION, message="Client opened
+  connection." from source clientapp!
+\end{verbatim}
+
 \paragraph{Message Transfer}
 
 The client process sends a numbered message to the server process every
-minute and logs the sending event locally. Upon receiving the message, the
-server logs receipt including message number and replies immediately. As
-soon as the client receives the reply, it also logs receipt including
-message number. 
+minute and logs the sending locally. Upon receiving the message, the
+server logs receipt and replies immediately. As soon as the client
+receives the reply, it also logs receipt. The corresponding log statements
+for the first message exchange with message number 0 are: (request time:
+5.559 seconds, response time 3.576 seconds)
 
+\begin{verbatim}
+FINE event.hidservmeas observeEvent Observed event EventImpl:
+  occurenceTime=21:05:40.838, source="clientapp",
+  type=CLIENT_SENDING_REQUEST, message="Client sending request
+  0." from source clientapp!
+FINE event.hidservmeas observeEvent Observed event EventImpl:
+  occurenceTime=21:05:46.397, source="serverapp",
+  type=SERVER_RECEIVING_REQUEST_SENDING_REPLY, message="Server
+  receiving request 0." from source serverapp!
+FINE event.hidservmeas observeEvent Observed event EventImpl:
+  occurenceTime=21:05:49.973, source="clientapp",
+  type=CLIENT_REPLY_RECEIVED, message="Client receiving
+  response 0." from source clientapp!
+\end{verbatim}
+
 \paragraph{Connection Durability}
 
-If the client process observe a connection break before the test duration
+If the client process observes a connection break before the test duration
 of four hours elapses, it logs this event locally. The difference between
 the timestamp of having established the connection and the timestamp of the
-connection break is considered the connection durability.
+connection break is considered as the connection durability. If there is no
+such log event, the last message receipt time is used instead. All times
+are truncated at 3:45 hours. The relevant log statements are: (connection
+duration: 3:23:53.992 hours)
 
+\begin{verbatim}
+FINE event.hidservmeas observeEvent Observed event EventImpl:
+  occurenceTime=21:05:40.837, source="clientapp",
+  type=CLIENT_OPENED_CONNECTION, message="Client opened
+  connection." from source clientapp!
+FINE event.hidservmeas observeEvent Observed event EventImpl:
+  occurenceTime=00:29:34.829, source="clientapp",
+  type=CLIENT_CLOSING_CONNECTION, message="Client is no longer
+  connected." from source clientapp!
+\end{verbatim}
+
 \subsection{Connection Establishment}
 
-TODO Christian: describe setup here, possibly re-use parts of diploma
-thesis.
+\emph{TODO Christian: Describe setup here, possibly re-use parts of diploma
+thesis.}
 
 \section{Results}
 
-The \emph{outside view} measurements were performed between June 1, 2:50pm
+The \emph{external view} measurements were performed between June 1, 2:50pm
 (starting time of first test run) and June 5, 9:35am (starting time of last
 test run), resulting in a total of 1,090 data samples. A tarball with all
 log files is available
 online.\footnote{\url{http://freehaven.net/~karsten/hidserv/test-env.tar.gz}}
 
-TODO Christian add meta-data of your measurements here
+\emph{TODO Christian: Add meta-data of your measurements here; we should
+consider making your raw data available, too.}
 
 \subsection{Service Publication}
 
-TODO Karsten
+Figure~\ref{fig:publtime} shows the overall service publication times as a
+user experiences the process of starting up Tor until a hidden service is
+available for clients. Table~\ref{tab:publtime} contains the corresponding
+summary of values.
 
+\begin{figure}
+\centering
+\includegraphics[width=0.8\textwidth]{publtime.png}
+\caption{Histogram of overall service publication times}
+\label{fig:publtime}
+\end{figure}
+
+\begin{table}
+\centering
+\caption{Summary of overall service publication times}
+\label{tab:publtime}
+\begin{tabular}{rrrrrrrr}\hline
+Min. & 1st Qu. & Median & Mean & 3rd Qu. & Max. & StdDev & IQR\\\hline
+22.85 & 70.15 & 90.26 & 131.90 & 130.40 & 3685.00 & 191.80 & 60.21
+\end{tabular}
+\end{table}
+
+However, these values consist of at least two parts of which one is
+independent from hidden services: a) initialization time and b) service
+establishment time. Therefore, Figure~\ref{fig:esttime} shows only service
+establishment times and Table~\ref{tab:esttime} the corresponding summary
+of values.
+
+\begin{figure}
+\centering
+%\includegraphics{esttime.png}
+\emph{TODO Karsten: Generate histogram similar to that in
+Figure~\ref{fig:publtime} with establishment times only.}
+\caption{Histogram of service establishment times}
+\label{fig:esttime}
+\end{figure}
+
+\begin{table}
+\centering
+\caption{Summary of service establishment times}
+\label{tab:esttime}
+\begin{tabular}{rrrrrrrr}\hline
+Min. & 1st Qu. & Median & Mean & 3rd Qu. & Max. & StdDev & IQR\\\hline
+12.85 & 50.60 & 58.27 & 66.82 & 69.74 & 3665.00 & 111.00 & 19.14
+\end{tabular}
+\end{table}
+
+There are at least two observations from these data:
+\begin{enumerate}
+\item The minimum value of $12.85$ seconds is a good result, but
+unfortunately it is lower than it ought to be. There should be a 30-second
+delay meaning that a descriptor is only uploaded if it is unchanged for at
+least 30 seconds. Hence, the desciptor was uploaded prematurely in the
+considered case. In the whole set of 1,090 samples, there were four
+service establishment times below 30 seconds.
+
+It turned out that the reason for this is a bug in the code which is now
+fixed. See SVN revision r15113 for
+details.\footnote{\url{http://archives.seul.org/or/cvs/Jun-2008/msg00231.html}}
+This is not meant as confirmation for the usefulness of the 30-second
+delay, but only to make the implementation consistent with the
+specification.
+
+\item On the other end, the maximum value of 3,665 seconds (1:01:05 hours)
+appears to be unbelievably high for a service establishment time. In fact,
+it turned out that this data sample is an outlier with the next smaller
+service establishment time of 361.5 seconds (6:01.500 minutes) which is
+more reasonable.
+
+However, a more in-depth analysis of the log files with high service
+establishment times revealed a more severe bug. While setting up a hidden
+service, some valid introduction circuits were overlooked and given up.
+This bug is also fixed in SVN revision
+r15149.\footnote{\url{http://archives.seul.org/or/cvs/Jun-2008/msg00268.html}}
+It is a bugfix on Tor version 0.2.0.13-alpha which was released on
+December~21, 2007.
+\end{enumerate}
+
+Even though these two bugfixes are probably a good improvement for service
+publication times and hidden services in general, they prevent a deeper
+analysis of the reasons for high service establishment times for the time
+being. The next step will be to perform new measurements with a fixed Tor
+version and further investigate those results.
+
+\emph{TODO Karsten: Does the bugfix improve publication significantly,
+e.g. by reducing StdDev? how can we show whether the 30-second delay is
+useful or not?}
+
+\begin{table}
+\centering
+\caption{Summary of service establishment times with bugfixed Tor version;
+\textbf{Warning: these numbers are based on only 75 data samples;
+measurements are still in progress until June 13, 15:00 UTC.}}
+\label{tab:esttime2}
+\begin{tabular}{rrrrrrrr}\hline
+Min. & 1st Qu. & Median & Mean & 3rd Qu. & Max. & StdDev & IQR\\\hline
+38.84 & 46.29 & 52.29 & 57.31 & 63.67 & 108.10 & 16.26 & 17.38
+\end{tabular}
+\end{table}
+
 \subsection{Connection Establishment}
 
-TODO Christian: your results here
+\begin{figure}
+\centering
+\includegraphics[width=0.8\textwidth]{opentime.png}
+\caption{Histogram of connection opening times}
+\label{fig:opentime}
+\end{figure}
 
+\emph{TODO Christian: Your results here, possibly (or probably?) replacing
+Figure~\ref{fig:opentime}}
+
 \subsection{Message Transfer}
 
-TODO Karsten
+\begin{figure}
+\centering
+\includegraphics[width=0.8\textwidth]{reqtime.png}
+\caption{Histogram of request transmission times}
+\label{fig:reqtime}
+\end{figure}
 
+\begin{figure}
+\centering
+\includegraphics[width=0.8\textwidth]{restime.png}
+\caption{Histogram of response transmission times}
+\label{fig:restime}
+\end{figure}
+
+\begin{table}
+\centering
+\caption{Summary of request and response times}
+\label{tab:reqresp}
+\begin{tabular}{lrrrrrr}\hline
+ & Min. & 1st Qu. & Median & Mean & 3rd Qu. & Max.\\\hline
+Request Time & 0.061 & 1.447 & 2.561 & 11.350 & 5.164 & 2752.000\\
+Response Time & 0.061 & 1.500 & 2.545 & 8.568 & 4.720 & 2931.000\\
+\end{tabular}
+\end{table}
+
+\emph{TODO Steven: Any insights from variable correlations here?}
+
+\emph{TODO Karsten: What to do with the results in
+Figures~\ref{fig:reqtime} and \ref{fig:restime} and
+Table~\ref{tab:reqresp}?}
+
 \subsection{Connection Durability}
 
-TODO Karsten
+Figure~\ref{fig:duration} shows connection durations and
+Table~\ref{tab:duration} contains the corresponding summary of values.
+The artitifical upper limit of 13,500 seconds (3:45 hours) comes from the
+maximum allowed test time of four hours after a connection has been
+established. The unexpectedly high number of 650 out of 962 (67.57\%)
+established connections did not break within the first 3:45 hours.
 
+\begin{figure}
+\centering
+\includegraphics[width=0.8\textwidth]{duration.png}
+\caption{Histogram of connection durations}
+\label{fig:duration}
+\end{figure}
+
+\begin{table}
+\centering
+\caption{Summary of connection durations}
+\label{tab:duration}
+\begin{tabular}{rrrrrr}\hline
+Min. & 1st Qu. & Median & Mean & 3rd Qu. & Max.\\\hline
+15.54 & 9953.00 & 13500.00 & 11170.00 & 13500.00 & 13500.00
+\end{tabular}
+\end{table}
+
+The remaining part of breaking connections is displayed in
+Figure~\ref{fig:broken}. There is no noticeable point of time at which
+connections break, but connection breaks seem to be equally distributed
+over the whole time interval.
+
+\begin{figure}
+\centering
+%\includegraphics{broken.png}
+\emph{TODO Karsten: Generate histogram as in Figure~\ref{fig:duration},
+but only with broken connections}
+\caption{Histogram of connection durations of breaking connections}
+\label{fig:broken}
+\end{figure}
+
+Altogether, hidden service connections appear to be quite stable, so that
+there is no need to put special focus on it in the attempt to improve the
+hidden service protocol.
+
 \section{Discussion}
 
 Ideas what changes are most likely to improve the overall performance.
 
+\begin{itemize}
+\item Fix more bugs and produce less new ones.
+\item Think about 30-seconds delay when publishing descriptor.
+\item $\cdots$
+\end{itemize}
+
+\emph{TODO Karsten: When this list is reasonably populated, make two
+paragraphs out of it.}
 \end{document}
 

Added: projects/hidserv/trunk/doc/reqtime.png
===================================================================
(Binary files differ)


Property changes on: projects/hidserv/trunk/doc/reqtime.png
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream

Added: projects/hidserv/trunk/doc/restime.png
===================================================================
(Binary files differ)


Property changes on: projects/hidserv/trunk/doc/restime.png
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream
Prev by Author: [or-cvs] r15163: Wrote motivation section and first half of measurement setup (projects/hidserv/trunk/doc)
Next by Author: [or-cvs] r15206: Changed parsing script for request and response times so tha (projects/hidserv/trunk)
Previous by thread: [or-cvs] r15173: Implement code to manually force the OpenSSL client cipher l (in tor/trunk: . doc doc/spec src/common)
Next by thread: [or-cvs] r15175: Add proposal 138: remove routers that are not Running from c (tor/trunk/doc/spec/proposals)
Index(es):
- Author
- Thread