On Tue, May 28, 2013 at 2:50 AM, Damian Johnson
<atagar@xxxxxxxxxxxxxx> wrote:
So far, so good. By my read of the man pages this means that gzip or
python's zlib module should be able to handle the decompression.
However, I must be missing something...
% wget http://128.31.0.34:9131/tor/server/all.z
[...]
% python
>>> import zlib
>>> with open('all.z') as desc_file:
... Â print zlib.decompress(desc_file.read())
...
Traceback (most recent call last):
 File "<stdin>", line 2, in <module>
zlib.error: Error -5 while decompressing data: incomplete or truncated stream
This seemed peculiar, so I tried it out. Each time I wget all.z from that address, it's always a different one; I guess that's how it should be, but it seems that sometimes not all of it gets downloaded (hence the actually legit zlib error.)
I was able to make it work after my second download attempt (with your exact code); zlib handles it well. So far it's worked every time since.
This is probably not good if the source may sometimes deliver an incomplete stream.
TL;DR try wget'ing multiple times and getting even more puzzled (?)