[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

[tor-talk] wget - secure?

To: tor-talk@xxxxxxxxxxxxxxxxxxxx
Subject: [tor-talk] wget - secure?
From: helpfulnoob@xxxxxxxxxxxxx
Date: Sat, 26 May 2012 18:23:36 -0400
Delivered-to: archiver@xxxxxxxx
Delivery-date: Sat, 26 May 2012 19:27:08 -0400
Domainkey-signature: a=rsa-sha1; q=dns; c=nofws; s=N1-0105; d=Safe-mail.net; b=RtcESy5rIiTiYfP9WgE3q1n6zrYaTB6CkZmkFjckbvGKTsNbteFpb2mfFmc4L1Aa E/Djbi0i0kagq1CszmKhLZU46ZqDpZ0LVkxqS1cNIR6YnTOe/6GBSo2ZTABtOioO lhhSl3HLVSrlIk6PtGisegWRBanxtQcMGIzW3jviqSY=;
List-archive: <http://lists.torproject.org/pipermail/tor-talk>
List-help: <mailto:tor-talk-request@lists.torproject.org?subject=help>
List-id: "This mailing list is for all discussion about theory, design, and development of Onion Routing." <tor-talk.lists.torproject.org>
List-post: <mailto:tor-talk@lists.torproject.org>
List-subscribe: <https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk>, <mailto:tor-talk-request@lists.torproject.org?subject=subscribe>
List-unsubscribe: <https://lists.torproject.org/cgi-bin/mailman/options/tor-talk>, <mailto:tor-talk-request@lists.torproject.org?subject=unsubscribe>
Reply-to: tor-talk@xxxxxxxxxxxxxxxxxxxx
Sender: tor-talk-bounces@xxxxxxxxxxxxxxxxxxxx

Hi there!

>On Wed, Apr 18, 2012 at 11:37, Robert Ransom <rransom.8774 at gmail.com> wrote:
>
>>On 2012-04-18, Maxim Kammerer <mk at dee.su> wrote:
>>
>> TL;DR: wget is 100% safe to use with Tor and it does not leak DNS
>> (also true for curl, by the way).
>
>Which version of wget did you audit?  What information leaks did you
>check for during your audit?
>
>Which SSL library did you configure wget to use?  Which version of
>hat SSL library did you audit?
>
> ...
>
>Which configuration of wget makes it use Tor â100% safeâly?
>
>
>Robert Ransom

I like your answer Robert Ransom, so, you motivates me to test GNU Wget 1.13.4 on Windows, for DNS [1], Header [2], and FTP [3] leaks mentioned so far in this talk and the talk "Download Manger" [4].

But, I'm only a helpfulnoob, not a helpfulJediTorMasterNinja, so, I'm not that helpful after all, I guess. I hope my little contributions below does someone some good, it was neat to learn and I needed a good download manger for Tor, anyway! :)

TL;DR: 
Wget v1.13.4 (openssl 1.0.0g), Privoxy v3.0.19, , and Wireshark 1.6.8, on Windows 7 x64 Home Premium  SP1: no DNS and no header(?) leaks for SOCKS4a and SOCKS5, tested hidden service and normal website; I didn't know how to test IP leak over FTP PORT, so I couldn't test. 

If anyone sees anything dumb, please point it out to me. Thanks! I didn't know how to make any sense of out Wireshark for scanning the Wget headers (i.e., reducing the "Limit Each Packet To" X bytes setting, I tried 58). Thankfully, it's easy to see the headers from Wget, and the website, using Privoxy's 'debug 8' setting (âshow header parsingâ); at least as far as this noob understands.

[1] https://lists.torproject.org/pipermail/tor-talk/2012-April/024014.html
[2] https://lists.torproject.org/pipermail/tor-talk/2012-April/023947.html
[3] https://lists.torproject.org/pipermail/tor-talk/2012-April/024040.html
[4] https://lists.torproject.org/pipermail/tor-talk/2012-April/023918.html

Here's my WGETRC.TXT file, with lots of comments about the testing, etc. This file is setup for downloading whole web sites, but d/ling single files is simple via. command line or batch file (just point URL to a file, not a dir, and using the "-e" command to override settings in the wgetrc.txt file, if needed). I might have done something stupid here, so, I don't advise anyone uses this until other people (not noobs like me) comment.

------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
# NOTE: Use the following command line in terminal, or batch script, when running Wget:
#wget -c http://site.onion

# GLOBAL Wget (v1.13.4) SETTINGS TO POLITELY DOWNLOAD (MIRROR) WHOLE HIDDEN SERIVCE WEBSITE OR INTERENT WEBSITE
#
# I personally verified no DNS leaks, and AFAIU no header leaks, with the following settings using Wget v1.13.4 with 
# openssl 1.0.0g, Privoxy v3.0.19, and Wireshark 1.6.8, on Windows 7 x64 Home Premium  SP1. I followed the 
# directions for DNS [1] and for http headers [2], while downloading the Tor Project Hidden Service website 
# (http://idnxcnkne4qt76tg.onion/) and the DuckDuckGo website (http://duckduckgo.com/). 
# However, I couldnât make heads nor tails out of Wireshark for http headers [2], so instead I used Privoxy debug 
# option 8 (âshow header parsingâ). I did however serach for my IP address [3], after downloading from an FreeBSD FTP server, 
# but I didn't know what to look for in Wirehshark, specifically; I ended up blocking FPT via. my firewall while running Wget...
# 
# The Wget v1.13.4 Windows binary is from (http://opensourcepack.blogspot.com/2010/05/wget-112-for-windows.html), 
# and I checked it with VirusTotal (two flags [4]), and locally installed Kaspersky 2012 (clean), Malware Bytes' 
# Anti-Malware (clean), and SUPERAntiSpyware (clean), all versions and updates current as of 2012/05/26.

# https://www.gnu.org/software/wget/manual/wget.html#Wgetrc-Commands
# https://lists.torproject.org/pipermail/tor-talk/2012-April/024016.html
# https://lists.torproject.org/pipermail/tor-talk/2012-April/024040.html
# https://lists.torproject.org/pipermail/tor-talk/2012-April/024014.html
# https://lists.torproject.org/pipermail/tor-talk/2012-April/023948.html
# https://lists.torproject.org/pipermail/tor-talk/2012-April/024021.html
# https://lists.torproject.org/pipermail/tor-talk/2012-April/024035.html
# https://lists.torproject.org/pipermail/tor-talk/2012-April/024016.html
# [1] (WireShark DNS) https://lists.torproject.org/pipermail/tor-talk/2012-April/024026.html
# [2] (WireShark HTTP headers) http://ask.wireshark.org/questions/4137/capturing-headers-only
# [3] (WireShark IP address) http://portforward.com/networking/wireshark.htm
# [4] https://www.virustotal.com/file/b56cae743aac0d0e66df77dc2107b68d7ea2f99f8f9d17cdab35e98b7503e37f/analysis/1338056337/
# http://www.reaper-x.com/2007/09/15/using-wget-on-windows/
# https://seogadget.co.uk/download-your-website-with-wget/

use_proxy = on

http_proxy = http://127.0.0.1:8118/

# The following user_agent, header, connect_timeout, and http_keep_alive are meant to mirror 
# headers of TorBrowserBundel v2.2.3-13 and TorButton v1.4.5.1

user_agent = Mozilla/5.0 (Windows NT 6.1; rv:5.0) Gecko/20100101 Firefox/5.0
header = Accept-Language: en-us,en;q=0.5
header = Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
connect-timeout = 250
# The following Accept-Encoding header breaks Wget downloading
#header = Accept-Encoding: gzip, deflate
# The following http_keep_alive is the defualt setting for Wget
#http_keep_alive = on

# The following referer [sic] can be configured (string) for the website to be downloaded; 
# this sets the HTTP âReferer:â header
#referer = http://site.onion

timestamping = on

tries = 5

# Increase the following reclevel to increase recursive retrieval depth

reclevel = 5

robots = off

random_wait = on

limit_rate = 30K

recursive = on

# The following no_clobber cannot be used if convert_links is also used concurrently, Wget will
# default to disabling no_clobber and only using convert_links. 
#no_clobber = on

page_requisites = on

html-extension = on

# The following restrict-file-names is only for Windows operating systems
restrict-file-names = windows

convert_links = on

# The following backup_converted is used when the above, convert_links is set to 'on'
backup_converted = on

dirstruct = on
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

Here's the Tor and Wget relvent parts of my Privoxy CONFIG.TXT file:

------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
# Configuration for Privoxy use by Wget, into Tor 
# SOCKS4a and SOCKS5 worked equally well, to prevent DNS leaks

# https://trac.torproject.org/projects/tor/wiki/doc/PrivoxyConfig
# http://pseudo-flaw.net/content/tor/vidalia-insecure-privoxy-configuration/

forward-socks4a / 127.0.0.1:9050 .
listen-address 127.0.0.1:8118

# Mirror TorBrowserBundle v2.2.3-13 and TorButton -- about:config (v1.4.5.1)
keep-alive-timeout 20 
max-client-connections 256
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

Here's the script to run Wget from a TrueCrypt container, I set system environmental variables for C:\Wget and wgetrc (I couldn't cd into the volume, for some reason):

------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
@echo off

wget -c http://site.onion/
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------

Here's the Privoxy debug outputs from setting '8', showing the headers from Wget (I'm trying to match the headres of TorBrowser...):

------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
Downloading whole DuckDuckGo website :

2012-05-26 15:02:38.570 00000d60 Header: scan: GET http://duckduckgo.com/ HTTP/1.1
2012-05-26 15:02:38.570 00000d60 Header: scan: Referer: http://duckduckgo.com/
2012-05-26 15:02:38.586 00000d60 Header: scan: User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:5.0) Gecko/20100101 Firefox/5.0
2012-05-26 15:02:38.586 00000d60 Header: scan: Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
2012-05-26 15:02:38.586 00000d60 Header: scan: Host: duckduckgo.com
2012-05-26 15:02:38.586 00000d60 Header: scan: Connection: Close
2012-05-26 15:02:38.586 00000d60 Header: scan: Proxy-Connection: Keep-Alive
2012-05-26 15:02:38.586 00000d60 Header: scan: Accept-Language: en-us,en;q=0.5
2012-05-26 15:02:38.586 00000d60 Header: Keeping the client header 'Connection: Close' around. The connection will not be kept alive.
2012-05-26 15:02:38.586 00000d60 Header: crumble crunched: Proxy-Connection: Keep-Alive!
2012-05-26 15:02:38.586 00000d60 Header: New HTTP Request-Line: GET / HTTP/1.1

Downloading whole Tor Project hidden service website:

2012-05-26 14:54:11.964 00000bac Header: scan: GET http://idnxcnkne4qt76tg.onion/ HTTP/1.1
2012-05-26 14:54:11.979 00000bac Header: scan: Referer: http://idnxcnkne4qt76tg.onion/
2012-05-26 14:54:11.979 00000bac Header: scan: User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:5.0) Gecko/20100101 Firefox/5.0
2012-05-26 14:54:11.979 00000bac Header: scan: Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
2012-05-26 14:54:11.979 00000bac Header: scan: Host: idnxcnkne4qt76tg.onion
2012-05-26 14:54:11.979 00000bac Header: scan: Connection: Close
2012-05-26 14:54:11.979 00000bac Header: scan: Proxy-Connection: Keep-Alive
2012-05-26 14:54:11.979 00000bac Header: scan: Accept-Language: en-us,en;q=0.5
2012-05-26 14:54:11.979 00000bac Header: Keeping the client header 'Connection: Close' around. The connection will not be kept alive.
2012-05-26 14:54:11.979 00000bac Header: crumble crunched: Proxy-Connection: Keep-Alive!
2012-05-26 14:54:11.979 00000bac Header: New HTTP Request-Line: GET / HTTP/1.1
------------------------------------------------------------------------------------------------
------------------------------------------------------------------------------------------------
_______________________________________________
tor-talk mailing list
tor-talk@xxxxxxxxxxxxxxxxxxxx
https://lists.torproject.org/cgi-bin/mailman/listinfo/tor-talk

Prev by Author: Re: [tor-talk] wget - secure?
Next by Author: Re: [tor-talk] wget - secure?
Previous by thread: Re: [tor-talk] wget - secure?
Next by thread: Re: [tor-talk] wget - secure?
Index(es):
- Author
- Thread