[or-cvs] r15224: Improved service publication subsection in results and added (projects/hidserv/trunk/doc)

[kloesing@xxxxxxxx]

Author: kloesing
Date: 2008-06-13 13:42:47 -0400 (Fri, 13 Jun 2008)
New Revision: 15224

Added:
   projects/hidserv/trunk/doc/esttime2.png
Modified:
   projects/hidserv/trunk/doc/esttime.png
   projects/hidserv/trunk/doc/publtime.png
   projects/hidserv/trunk/doc/report.pdf
   projects/hidserv/trunk/doc/report.tex
Log:
Improved service publication subsection in results and added new measurement data of bugfixed Tor.

Modified: projects/hidserv/trunk/doc/esttime.png
===================================================================
(Binary files differ)

Added: projects/hidserv/trunk/doc/esttime2.png
===================================================================
(Binary files differ)


Property changes on: projects/hidserv/trunk/doc/esttime2.png
___________________________________________________________________
Name: svn:mime-type
   + application/octet-stream

Modified: projects/hidserv/trunk/doc/publtime.png
===================================================================
(Binary files differ)

Modified: projects/hidserv/trunk/doc/report.pdf
===================================================================
(Binary files differ)

Modified: projects/hidserv/trunk/doc/report.tex
===================================================================
--- projects/hidserv/trunk/doc/report.tex	2008-06-13 16:35:27 UTC (rev 15223)
+++ projects/hidserv/trunk/doc/report.tex	2008-06-13 17:42:47 UTC (rev 15224)
@@ -193,59 +193,57 @@
 
 \subsection{Service Publication}
 
-Figure~\ref{fig:publtime} shows the overall service publication times as a
-user experiences the process of starting up Tor until a hidden service is
-available for clients. Table~\ref{tab:publtime} contains the corresponding
-summary of values.
+From the 1,090 measured publication times, the 1\% highest values (698,
+901, 1214, 1220, 1222, 1224, 1228, 1241, 1290, 3073, and 3685 seconds) were
+considered as outliers and thereby discarded in the statistical analysis.
+(Nevertheless the corresponding test cases need to be analyzed in detail to
+find out the reasons for these high delays.) Figure~\ref{fig:publtime}
+shows the overall service publication times as a user experiences the
+process of starting up Tor until a hidden service is available for clients.
 
 \begin{figure}
 \centering
 \includegraphics[width=0.8\textwidth]{publtime.png}
-\caption{Histogram of overall service publication times}
+\begin{tabular}{rrrrrrr}
+Min. & 1st Qu. & Median & Mean & 3rd Qu. & Max. & StdDev\\\hline
+22.85 & 69.91 & 89.68 & 118.10 & 129.20 & 698.10 & 93.77
+% x <- subset(publtime, inittime+publtime<699000)
+% summary((x$inittime+x$publtime)/1000)
+% sd((x$inittime+x$publtime)/1000)
+\end{tabular}
+\caption{Overall service publication times}
 \label{fig:publtime}
 \end{figure}
 
-\begin{table}
-\centering
-\caption{Summary of overall service publication times}
-\label{tab:publtime}
-\begin{tabular}{rrrrrrrr}\hline
-Min. & 1st Qu. & Median & Mean & 3rd Qu. & Max. & StdDev & IQR\\\hline
-22.85 & 70.15 & 90.26 & 131.90 & 130.40 & 3685.00 & 191.80 & 60.21
-\end{tabular}
-\end{table}
-
 However, these values consist of at least two parts of which one is
 independent from hidden services: a) initialization time and b) service
 establishment time. Therefore, Figure~\ref{fig:esttime} shows only service
-establishment times and Table~\ref{tab:esttime} the corresponding summary
-of values.
+establishment times. This time only the top two value had to be discarded
+as outliers (362 and 3665 seconds).
 
 \begin{figure}
 \centering
 \includegraphics[width=0.8\textwidth]{esttime.png}
-\caption{Histogram of service establishment times}
+\begin{tabular}{rrrrrrr}
+Min. & 1st Qu. & Median & Mean & 3rd Qu. & Max. & StdDev\\\hline
+12.85 & 50.59 & 58.27 & 63.25 & 69.73 & 142.20 & 18.67
+% x <- subset(publtime, publtime<361000)
+% summary(x$publtime/1000)
+% sd(x$publtime/1000)
+\end{tabular}
+\caption{Service establishment times}
 \label{fig:esttime}
 \end{figure}
 
-\begin{table}
-\centering
-\caption{Summary of service establishment times}
-\label{tab:esttime}
-\begin{tabular}{rrrrrrrr}\hline
-Min. & 1st Qu. & Median & Mean & 3rd Qu. & Max. & StdDev & IQR\\\hline
-12.85 & 50.60 & 58.27 & 66.82 & 69.74 & 3665.00 & 111.00 & 19.14
-\end{tabular}
-\end{table}
+\paragraph{Too Small Minimum Value}
 
-There are at least two observations from these data:
-\begin{enumerate}
-\item The minimum value of $12.85$ seconds is a good result, but
-unfortunately it is lower than it ought to be. There should be a 30-second
-delay meaning that a descriptor is only uploaded if it is unchanged for at
-least 30 seconds. Hence, the desciptor was uploaded prematurely in the
-considered case. In the whole set of 1,090 samples, there were four
-service establishment times below 30 seconds.
+The first observation is that the minimum value of $12.85$ seconds
+is---albeit being a good result---lower than it ought to be. Actually,
+there is a 30-second delay meaning that a descriptor is only uploaded if it
+is unchanged for at least 30 seconds. Hence, the desciptor was uploaded
+prematurely in the considered case. In the whole set of 1,090 samples,
+there are four service establishment times below 30 seconds (13, 18, 27,
+and 28 seconds).
 
 It turned out that the reason for this is a bug in the code which is now
 fixed. See SVN revision r15113 for
@@ -254,43 +252,60 @@
 delay, but only to make the implementation consistent with the
 specification.
 
-\item On the other end, the maximum value of 3,665 seconds (1:01:05 hours)
-appears to be unbelievably high for a service establishment time. In fact,
-it turned out that this data sample is an outlier with the next smaller
-service establishment time of 361.5 seconds (6:01.500 minutes) which is
-more reasonable.
+\paragraph{Slow Decrease of Frequencies}
 
-However, a more in-depth analysis of the log files with high service
-establishment times revealed a more severe bug. While setting up a hidden
-service, some valid introduction circuits were overlooked and given up.
-This bug is also fixed in SVN revision
+Figure~\ref{fig:esttime} shows an unexpectedly large number of service
+establishment times of 70 seconds or greater. From the histogram one could
+assume that establishment times would steadily decline from their maximum
+at 50 seconds which however is not the case.
+
+An in-depth analysis of the log files has revealed an even more severe bug.
+While setting up a hidden service, some valid introduction circuits were
+overlooked and abandoned. This leads to random delay in establishing
+introduction points and publishing a descriptor. This bug is now fixed in
+SVN revision
 r15149.\footnote{\url{http://archives.seul.org/or/cvs/Jun-2008/msg00268.html}}
-It is a bugfix on Tor version 0.2.0.13-alpha which was released on
-December~21, 2007.
-\end{enumerate}
 
-Even though these two bugfixes are probably a good improvement for service
-publication times and hidden services in general, they prevent a deeper
-analysis of the reasons for high service establishment times for the time
-being. The next step will be to perform new measurements with a fixed Tor
-version and further investigate those results.
+Figure~\ref{fig:esttime2} shows service establishment times of a subsequent
+measurement with a bugfixed Tor version. Again, three values were discarded
+as outliers (3665, 3680, and 10931 seconds). These new results already show
+an improvement of mean times and standard deviations.
 
-\emph{TODO Karsten: Does the bugfix improve publication significantly,
-e.g. by reducing StdDev? how can we show whether the 30-second delay is
-useful or not?}
+\emph{TODO: Does the bugfix improve publication times
+\textbf{significantly}?}
 
-\begin{table}
+\begin{figure}
 \centering
-\caption{Summary of service establishment times with bugfixed Tor version;
-\textbf{Warning: these numbers are based on only 75 data samples;
-measurements are still in progress until June 13, 15:00 UTC.}}
-\label{tab:esttime2}
-\begin{tabular}{rrrrrrrr}\hline
-Min. & 1st Qu. & Median & Mean & 3rd Qu. & Max. & StdDev & IQR\\\hline
-38.84 & 46.29 & 52.29 & 57.31 & 63.67 & 108.10 & 16.26 & 17.38
+\includegraphics[width=0.8\textwidth]{esttime2.png}
+\begin{tabular}{rrrrrrr}
+Min. & 1st Qu. & Median & Mean & 3rd Qu. & Max. & StdDev\\\hline
+35.55 & 46.73 & 51.99 & 56.73 & 61.42 & 141.60 & 15.82
+% x <- subset(publtime, publtime<3600000)
+% summary(x$publtime/1000)
+% sd(x$publtime/1000)
 \end{tabular}
-\end{table}
+\caption{Service establishment times with bugfixed Tor version}
+\label{fig:esttime2}
+\end{figure}
 
+\paragraph{Outliers}
+
+The outlier values 362 and 3665 in the original measurements appear to be
+unbelievably high for a service establishment time.
+
+\emph{TODO Karsten: What happened there?}
+
+\paragraph{30-Second Delay}
+
+%Even though these two bugfixes are probably a good improvement for service
+%publication times and hidden services in general, they prevent a deeper
+%analysis of the reasons for high service establishment times for the time
+%being. The next step will be to perform new measurements with a fixed Tor
+%version and further investigate those results.
+
+\emph{TODO Karsten: How can we show whether the 30-second delay is
+useful or not?}
+
 \subsection{Connection Establishment}
 
 \begin{figure}
@@ -308,38 +323,33 @@
 \begin{figure}
 \centering
 \includegraphics[width=0.8\textwidth]{reqtime.png}
-\caption{Histogram of request transmission times}
+\begin{tabular}{rrrrrr}
+Min. & 1st Qu. & Median & Mean & 3rd Qu. & Max.\\\hline
+0.061 & 1.447 & 2.561 & 11.350 & 5.164 & 2752.000\\
+\end{tabular}
+\caption{Request transmission times}
 \label{fig:reqtime}
 \end{figure}
 
 \begin{figure}
 \centering
 \includegraphics[width=0.8\textwidth]{restime.png}
-\caption{Histogram of response transmission times}
+\begin{tabular}{rrrrrr}
+Min. & 1st Qu. & Median & Mean & 3rd Qu. & Max.\\\hline
+0.061 & 1.500 & 2.545 & 8.568 & 4.720 & 2931.000\\
+\end{tabular}
+\caption{Response transmission times}
 \label{fig:restime}
 \end{figure}
 
-\begin{table}
-\centering
-\caption{Summary of request and response times}
-\label{tab:reqresp}
-\begin{tabular}{lrrrrrr}\hline
- & Min. & 1st Qu. & Median & Mean & 3rd Qu. & Max.\\\hline
-Request Time & 0.061 & 1.447 & 2.561 & 11.350 & 5.164 & 2752.000\\
-Response Time & 0.061 & 1.500 & 2.545 & 8.568 & 4.720 & 2931.000\\
-\end{tabular}
-\end{table}
-
 \emph{TODO Steven: Any insights from variable correlations here?}
 
 \emph{TODO Karsten: What to do with the results in
-Figures~\ref{fig:reqtime} and \ref{fig:restime} and
-Table~\ref{tab:reqresp}?}
+Figures~\ref{fig:reqtime} and \ref{fig:restime}?}
 
 \subsection{Connection Durability}
 
-Figure~\ref{fig:duration} shows connection durations and
-Table~\ref{tab:duration} contains the corresponding summary of values.
+Figure~\ref{fig:duration} shows connection durations.
 The artitifical upper limit of 13,500 seconds (3:45 hours) comes from the
 maximum allowed test time of four hours after a connection has been
 established. The unexpectedly high number of 650 out of 962 (67.57\%)
@@ -348,19 +358,13 @@
 \begin{figure}
 \centering
 \includegraphics[width=0.8\textwidth]{duration.png}
-\caption{Histogram of connection durations}
-\label{fig:duration}
-\end{figure}
-
-\begin{table}
-\centering
-\caption{Summary of connection durations}
-\label{tab:duration}
-\begin{tabular}{rrrrrr}\hline
+\begin{tabular}{rrrrrr}
 Min. & 1st Qu. & Median & Mean & 3rd Qu. & Max.\\\hline
 15.54 & 9953.00 & 13500.00 & 11170.00 & 13500.00 & 13500.00
 \end{tabular}
-\end{table}
+\caption{Connection durations}
+\label{fig:duration}
+\end{figure}
 
 The remaining part of breaking connections is displayed in
 Figure~\ref{fig:broken}. There is no noticeable point of time at which
@@ -370,7 +374,7 @@
 \begin{figure}
 \centering
 \includegraphics[width=0.8\textwidth]{broken.png}
-\caption{Histogram of connection durations of breaking connections}
+\caption{Connection durations of breaking connections}
 \label{fig:broken}
 \end{figure}