[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]
[tor-commits] [collector/master] Split up bridge descriptor tarballs into one per type.
commit f608c94c7f731241bf7ee8e627ca1da98c23d858
Author: Karsten Loesing <karsten.loesing@xxxxxxx>
Date: Sun Sep 18 16:09:52 2016 +0200
Split up bridge descriptor tarballs into one per type.
---
CHANGELOG.md | 4 ++++
src/main/resources/create-tarballs.sh | 26 ++++++++++++++++++++------
src/main/webapp/index.html | 19 +++++++------------
3 files changed, 31 insertions(+), 18 deletions(-)
diff --git a/CHANGELOG.md b/CHANGELOG.md
index e17abad..99b2918 100644
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -6,6 +6,10 @@
minutes to finish properly, rather than killing them immediately.
- Replace TCP ports with hashes in @type bridge-network-status 1.1
and @type bridge-server-descriptor 1.2.
+ - Split up bridge descriptor tarballs into one tarball per month
+ and descriptor type: bridge-statuses-YYYY-MM.tar.xz,
+ bridge-server-descriptors-YYYY-MM.tar.xz and
+ bridge-extra-infos-YYYY-MM.tar.xz.
* Minor changes
- Remove quotes around base URL in index.json.
diff --git a/src/main/resources/create-tarballs.sh b/src/main/resources/create-tarballs.sh
index cc75219..4f2c6eb 100755
--- a/src/main/resources/create-tarballs.sh
+++ b/src/main/resources/create-tarballs.sh
@@ -56,8 +56,12 @@ TARBALLS=(
server-descriptors-$YEARTWO-$MONTHTWO
extra-infos-$YEARONE-$MONTHONE
extra-infos-$YEARTWO-$MONTHTWO
- bridge-descriptors-$YEARONE-$MONTHONE
- bridge-descriptors-$YEARTWO-$MONTHTWO
+ bridge-statuses-$YEARONE-$MONTHONE
+ bridge-statuses-$YEARTWO-$MONTHTWO
+ bridge-server-descriptors-$YEARONE-$MONTHONE
+ bridge-server-descriptors-$YEARTWO-$MONTHTWO
+ bridge-extra-infos-$YEARONE-$MONTHONE
+ bridge-extra-infos-$YEARTWO-$MONTHTWO
)
TARBALLS=($(printf "%s\n" "${TARBALLS[@]}" | uniq))
@@ -77,8 +81,12 @@ DIRECTORIES=(
$OUTDIR/relay-descriptors/server-descriptor/$YEARTWO/$MONTHTWO/
$OUTDIR/relay-descriptors/extra-info/$YEARONE/$MONTHONE/
$OUTDIR/relay-descriptors/extra-info/$YEARTWO/$MONTHTWO/
- $OUTDIR/bridge-descriptors/$YEARONE/$MONTHONE/
- $OUTDIR/bridge-descriptors/$YEARTWO/$MONTHTWO/
+ $OUTDIR/bridge-descriptors/$YEARONE/$MONTHONE/statuses/
+ $OUTDIR/bridge-descriptors/$YEARTWO/$MONTHTWO/statuses/
+ $OUTDIR/bridge-descriptors/$YEARONE/$MONTHONE/server-descriptors/
+ $OUTDIR/bridge-descriptors/$YEARTWO/$MONTHTWO/server-descriptors/
+ $OUTDIR/bridge-descriptors/$YEARONE/$MONTHONE/extra-infos/
+ $OUTDIR/bridge-descriptors/$YEARTWO/$MONTHTWO/extra-infos/
)
DIRECTORIES=($(printf "%s\n" "${DIRECTORIES[@]}" | uniq))
@@ -109,8 +117,14 @@ cd $CURRENTPATH
echo `date` "Finished tarball creation. Starting symlink-update ..."
-mkdir -p $ARCHIVEDIR/bridge-descriptors/
-ln -f -s -t $ARCHIVEDIR/bridge-descriptors/ $TARBALLTARGETDIR/bridge-descriptors-20??-??.tar.xz
+mkdir -p $ARCHIVEDIR/bridge-descriptors/statuses/
+ln -f -s -t $ARCHIVEDIR/bridge-descriptors/statuses/ $TARBALLTARGETDIR/bridge-statuses-20??-??.tar.xz
+
+mkdir -p $ARCHIVEDIR/bridge-descriptors/server-descriptors/
+ln -f -s -t $ARCHIVEDIR/bridge-descriptors/server-descriptors/ $TARBALLTARGETDIR/bridge-server-descriptors-20??-??.tar.xz
+
+mkdir -p $ARCHIVEDIR/bridge-descriptors/extra-infos/
+ln -f -s -t $ARCHIVEDIR/bridge-descriptors/extra-infos/ $TARBALLTARGETDIR/bridge-extra-infos-20??-??.tar.xz
mkdir -p $ARCHIVEDIR/bridge-pool-assignments/
ln -f -s -t $ARCHIVEDIR/bridge-pool-assignments/ $TARBALLTARGETDIR/bridge-pool-assignments-20??-??.tar.xz
diff --git a/src/main/webapp/index.html b/src/main/webapp/index.html
index bbc70f2..7ed9145 100644
--- a/src/main/webapp/index.html
+++ b/src/main/webapp/index.html
@@ -168,21 +168,21 @@
<td>Bridge Network Statuses</td>
<td><tt>@type bridge-network-status 1.1</tt></td>
<td><a href="recent/bridge-descriptors/statuses/" class="btn btn-primary btn-xs">recent</a></td>
- <td><a href="archive/bridge-descriptors/" class="btn btn-primary btn-xs">archive</a></td>
+ <td><a href="archive/bridge-descriptors/statuses/" class="btn btn-primary btn-xs">archive</a></td>
<td><a href="#type-bridge-network-status" class="btn btn-default btn-xs">format</a></td>
</tr>
<tr>
<td>Bridge Server Descriptors</td>
<td><tt>@type bridge-server-descriptor 1.2</tt></td>
<td><a href="recent/bridge-descriptors/server-descriptors/" class="btn btn-primary btn-xs">recent</a></td>
- <td><a href="archive/bridge-descriptors/" class="btn btn-primary btn-xs">archive</a></td>
+ <td><a href="archive/bridge-descriptors/server-descriptors/" class="btn btn-primary btn-xs">archive</a></td>
<td><a href="#type-bridge-server-descriptor" class="btn btn-default btn-xs">format</a></td>
</tr>
<tr>
<td>Bridge Extra-info Descriptors</td>
<td><tt>@type bridge-extra-info 1.3</tt></td>
<td><a href="recent/bridge-descriptors/extra-infos/" class="btn btn-primary btn-xs">recent</a></td>
- <td><a href="archive/bridge-descriptors/" class="btn btn-primary btn-xs">archive</a></td>
+ <td><a href="archive/bridge-descriptors/extra-infos/" class="btn btn-primary btn-xs">archive</a></td>
<td><a href="#type-bridge-extra-info" class="btn btn-default btn-xs">format</a></td>
</tr>
<tr>
@@ -500,8 +500,6 @@ descriptor including all signatures.</li>
Sanitized bridge network statuses are similar to version 2 relay network
statuses, but with only a <tt>published</tt> line in the header and
without any lines in the footer.
-The bridge descriptor archive tarballs contain all bridge
-descriptors of a given month, not just network statuses.
The format has changed over time to accomodate changes to the sanitizing
process, with earlier versions being:
</p>
@@ -523,10 +521,8 @@ ports.</li>
<p>
Bridge server descriptors follow the same format as relay server
descriptors, except for the sanitizing steps described above.
-The bridge descriptor archive tarballs contain all bridge
-descriptors of a given month, not just server descriptors.
-These tarballs contain one descriptor per file, whereas recently published bridge descriptor
-files
+The bridge server descriptor archive tarballs contain one descriptor per
+file, whereas recently published bridge server descriptor files
contain all descriptors collected in an hour concatenated into a single
file to reduce the number of files.
The format has changed over time to accomodate changes to the sanitizing
@@ -579,9 +575,8 @@ published by bridges using an Ed25519 master key.</li>
</ul>
<p>
-The bridge descriptor archive tarballs contain all bridge
-descriptors of a given month, not just extra-info descriptors.
-These tarballs contain one descriptor per file, whereas recently published bridge descriptor
+The bridge extra-info descriptor archive tarballs contain one descriptor
+per file, whereas recently published bridge extra-info descriptor
files contain all descriptors collected in an hour concatenated into a single
file to reduce the number of files.
</p>
_______________________________________________
tor-commits mailing list
tor-commits@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-commits