[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

[tor-dev] Re: entrymap: A fast, modular scanner for Entry Guards & HSDirs (Complementing exitmap)



Hey 1AEO and the tor-dev team,                                                                                                                                                                                                        
                                                                                                                                                                                                                                         
    First off, a massive thank you for taking the time to give our initial proposal such a thorough, hostile-in-the-best-way review. We know pointing Astra and Carbon at a project and pulling together 16 spec citations isn't free—we  
    really appreciate you burning the compute and effort to help ground this in canonical Tor reality rather than letting us run with half-baked assumptions.                                                                            
                                                                                                                                                                                                                                         
    You guys were spot-on across the board. We took every single critique to heart, spent the last few days in the specs, and did a complete refactor of the codebase and test suites to align 1:1 with your notes.                      
                                                                                                                                                                                                                                         
    The full repository and testbenches are public here:                                                                                                                                                                                  
    https://github.com/Boxfrog1001/entrymap                                                                                                                                                                                              
    Spec Registry: https://github.com/Boxfrog1001/entrymap/blob/main/doc/TOR_SPEC_GROUND_TRUTH.md                                                                                                                                        
    Changelog & Fixes: https://github.com/Boxfrog1001/entrymap/blob/main/CHANGELOG.md                                                                                                                                                    
                                                                                                                                                                                                                                         
    (Note on OPSEC & Proxy Options: In the README we specifically laid out an "OPSEC & Privacy Considerations for Tor Users" section covering the operational boundary between passive modules and active wire probes. For passive        
    directory/sidecar lookups, users also have the option to run entrymap zero-daemon standalone, over Tor/SOCKS wrappers like torsocks or proxychains, or through an explicit HTTP/HTTPS proxy via --proxy alongside --no-cache. Active  
    wire probes connect directly to the target ORPort and should only be used against your own configured guard, an isolated measurement node, or a local Chutney instance.)                                                              
                                                                                                                                                                                                                                         
    Here is the quick rundown of what we fixed based directly on your review:                                                                                                                                                            
                                                                                                                                                                                                                                         
    1. Microdescriptors & CM37 (Sources [1–4]):                                                                                                                                                                                          
    You were 100% right about microdescriptors ditching the legacy 'family' lines. We updated family_leak.py and cidr_cluster.py to check DirAuth-computed 'effective_family' and strip both self-fingerprints and '$' prefixes cleanly.  
    We benchmarked it offline against the three prsv.ch relays you cited, and it passes with zero false positives.                                                                                                                        
                                                                                                                                                                                                                                         
    2. Tor Default 2^30 Bandwidth (Sources [5–6]):                                                                                                                                                                                        
    Total misread on our part conflating Tor’s compiled default 2^30 B/s (1 GiB/s) limit with signed 32-bit ceilings or rate gaming. ghost_bw.py now explicitly tags relays as [Tor Default 2^30 B/s], pulls sbws consensus weights into  
    view, and frames the delta strictly as measurement-bootstrap latency rather than inflation.                                                                                                                                          
                                                                                                                                                                                                                                         
    3. /16 vs /24 Path Selection (Sources [7–8]):                                                                                                                                                                                        
    We aligned with path-spec §2.2.1 (EnforceDistinctSubnets 1) and vanguards-spec. We stopped calling /24 density "Sybil clusters" and reframed it as what it actually is: physical hosting provider concentration, acknowledging that  
    Tor's client-side /16 rule prevents intra-circuit collisions anyway.
 
    4. Residential & reject *:* Relays (Sources [9–11]):
    Ditched the naive binary flags. residential.py now uses a conservative composite score that requires multiple heavy triggers (sustained high unmetered residential upstream, dual Guard+HSDir roles, empty families). We also made    
    sure the docs explicitly state that home-hosted middle relays and reject *:* nodes are welcome and healthy contributors to network capacity.
 
    5. v3 HSDir Threat Model (Sources [12–13]):
    Updated hsdir_canary.py and the docs to cite rend-spec-v3 §2.5 blinded keys. We made it crystal clear that HSDirs cannot decrypt v3 descriptors or harvest .onion addresses without already knowing them, and decoupled ring offsets  
    from WHERE-HSDESC placement.
 
    6. 7aSecurity Hardening & Spec Linting (Sources [14–16]):
    Clarified that our verifier is an automated schema/grammar linter against dir-spec §2.1.1, not a heuristic security judge. We also baked in the 2025 7aSecurity recommendations: strict regex sanitization, 0600 file permissions on  
    consensus caches, a zero-disk --no-cache mode for amnesic OSs (like Tails), and a 100% offline synthetic testbench (test/verify_specs.py).
 
    --- The Fun Stuff: Active Wire Probes ---
 
    While fixing the directory parsing, we also wanted to tackle the bigger question: what can entrymap do that Onionoo can't? To answer "What are entry nodes actually doing on the wire?", we added an active link-layer probing suite  
    that runs zero-daemon:
 
    * Ed25519 CERTS MITM Check (cert-spec §2.1 & tor-spec §4.2): Since outer X.509 certs are unauthenticated decoys, probe_tls_fingerprint.py cryptographically verifies that the outer DER cert's SHA-256 digest is signed by the Type 5
    (ED25519_TLS_LINK) cert and authenticated by the Type 4 master key, catching active ORPort TLS proxies.
    * CREATE_FAST KDF-TOR Engine (tor-spec §5.1.5 & §5.2.1): probe_flow_control.py establishes a one-hop circuit, verifies the relay's CREATED_FAST KH derivative, and derives AES-128-CTR + SHA-1 keys to send encrypted RELAY_DROP and  
    RELAY_SENDME cells to test queue drainage.
    * Baseline Jitter & Timing: probe_timing_jitter.py calibrates in-tunnel cell RTT against TCP connect RTT (cell_to_tcp_ratio) to filter out normal WAN latency from actual relay delay watermarking.
    * Prop 251 vs 254 Padding: probe_padding.py opens a baseline circuit first to spin up C-Tor's channel timers before testing padding negotiation.
    * --check-my-guards: Lets users audit their own active guards straight out of ~/.tor/state with one command.
 
    We've put the codebase through two local sandbox test suites so far (both passing 100%):
    * test/verify_specs.py: Audits CM37 family parsing, 2^30 defaults, subnet constraints, residential weighting, and v3 HSDir digests against canonical specs with zero network IO.
    * test/test_active_probes_sandbox.py: Runs a loopback adversarial harness testing timing watermark traps, padding strippers, flow starvation, and degraded TLS/Ed25519 MITM handling.
 
    Both run completely offline out of the box, and we would genuinely welcome more rigorous tests, edge cases, or adversarial fixtures from the community to push the scanner further.
 
 . We'd love to know what you think and would be thrilled to collaborate with anyone on the Network Health/tor-dev side. Also, the project is completely open for
    contributions—whether that’s PRs, new probe ideas, or co-designing modules. Given your deep domain knowledge of Tor metrics and directory data, we would absolutely love to have you folks contribute to entrymap directly if you're  
    up for it!
 
    Cheers,
 
    Adam (boxfrog)
    FrogSec Research
    https://github.com/Boxfrog1001/entrymap
_______________________________________________
tor-dev mailing list -- tor-dev@xxxxxxxxxxxxxxxxxxxx
To unsubscribe send an email to tor-dev-leave@xxxxxxxxxxxxxxxxxxxx