[Author Prev][Author Next][Thread Prev][Thread Next][Author Index][Thread Index]

Re: About HTTP 1.1 Cache



Juliusz Chroboczek <Juliusz.Chroboczek@xxxxxxxxxxxxxx> wrote:

> > That's why Privoxy can randomize the time within ranges that
> > are unlikely to cause revalidation failures:
> > http://www.privoxy.org/user-manual/actions-file.html#HIDE-IF-MODIFIED-SINCE
> > http://www.privoxy.org/user-manual/actions-file.html#OVERWRITE-LAST-MODIFIED
> 
> I believe you're referring to the following functionality:
> 
>   > The "randomize" option overwrites the value of the
>   > "Last-Modified:" header with a randomly chosen time between the
>   > original value and the current time.

Indeed.
 
> Most servers treat Last-Modified values as opaque validators --
> i.e. ``IMS: foo'' matches if the modification date is not exactly foo,
> rather than matching if the modification date is more recent than foo.
> Hence, in practice the randomize setting will bust the cache.

I'm aware that Gatling does that, but the others I checked
-- including IIS and Apache -- don't. Privoxy log excerpt for the latter:

Initial request:

14:22:35.875 283a8500 Header: GET http://www.apache.org/favicon.ico HTTP/1.1
14:22:35.875 283a8500 Header: Tagger 'http-method' added tag 'GET'. No action bits update necessary.
14:22:35.876 283a8500 Header: Tagger 'client-ip-address' added tag 'IP-ADDRESS: 10.0.0.1'. No action bits update necessary.
14:22:35.876 283a8500 Header: scan: User-Agent: curl/7.16.3 (i386-portbld-freebsd7.0) libcurl/7.16.3 OpenSSL/0.9.8e zlib/1.2.3
14:22:35.876 283a8500 Header: Tagger 'user-agent' added tag 'User-Agent: curl/7.16.3 (i386-portbld-freebsd7.0) libcurl/7.16.3 OpenSSL/0.9.8e zlib/1.2.3'. Action bits updated accordingly.
14:22:35.876 283a8500 Header: scan: Host: www.apache.org
14:22:35.876 283a8500 Header: scan: Pragma: no-cache
14:22:35.876 283a8500 Header: scan: Accept: */*
14:22:35.876 283a8500 Header: scan: Proxy-Connection: Keep-Alive
14:22:35.876 283a8500 Header: Modified: User-Agent: Mozilla/5.0 (X11; U; FreeBSD sparc64; nl-NL; rv:1.8.1.6) Gecko/20070805 Firefox/2.0.0.6
14:22:35.876 283a8500 Header: crumble crunched: Proxy-Connection: Keep-Alive!
14:22:35.877 283a8500 Header: Adding: Connection: close
14:22:35.877 283a8500 Connect: Overriding forwarding settings based on 'forward-socks4a tor-jail:9050 .'
14:22:35.877 283a8500 Header: New HTTP Request-Line: GET /favicon.ico HTTP/1.1
14:22:35.877 283a8500 Redirect: Decoding /favicon.ico if necessary.
14:22:35.877 283a8500 Redirect: Checking /favicon.ico for redirects.
14:22:35.877 283a8500 Request: www.apache.org/favicon.ico
14:22:35.877 283a8500 Connect: to www.apache.org
14:22:38.900 283a8500 Header: scan: HTTP/1.1 200 OK
14:22:38.900 283a8500 Header: scan: Date: Sun, 23 Sep 2007 12:22:38 GMT
14:22:38.900 283a8500 Header: scan: Server: Apache/2.3.0-dev (Unix)
14:22:38.900 283a8500 Header: scan: Last-Modified: Fri, 19 Aug 2005 07:37:30 GMT
14:22:38.900 283a8500 Header: scan: ETag: "31a763-e36-66f00280"
14:22:38.900 283a8500 Header: scan: Accept-Ranges: bytes
14:22:38.900 283a8500 Header: scan: Content-Length: 3638
14:22:38.900 283a8500 Header: scan: Cache-Control: max-age=86400
14:22:38.901 283a8500 Header: scan: Expires: Mon, 24 Sep 2007 12:22:38 GMT
14:22:38.901 283a8500 Header: scan: Connection: close
14:22:38.901 283a8500 Header: scan: Content-Type: image/x-icon
14:22:38.901 283a8500 Header: Tagger 'content-type' added tag 'image/x-icon'. No action bits update necessary.
14:22:38.901 283a8500 Header: Randomizing: Last-Modified: Fri, 19 Aug 2005 07:37:30 GMT
14:22:38.901 283a8500 Header: Randomized:  Last-Modified: Wed, 29 Mar 2006 06:51:34 GMT (added 221 days 23 hours 14 minutes 4 seconds)
10.0.0.1 - - [23/Sep/2007:14:22:39 +0200] "GET http://www.apache.org/favicon.ico HTTP/1.1" 200 3638

Successful revalidation request:

14:23:04.369 283a8f00 Header: GET http://www.apache.org/favicon.ico HTTP/1.1
14:23:04.369 283a8f00 Header: Tagger 'http-method' added tag 'GET'. No action bits update necessary.
14:23:04.369 283a8f00 Header: Tagger 'client-ip-address' added tag 'IP-ADDRESS: 10.0.0.1'. No action bits update necessary.
14:23:04.369 283a8f00 Header: scan: User-Agent: curl/7.16.3 (i386-portbld-freebsd7.0) libcurl/7.16.3 OpenSSL/0.9.8e zlib/1.2.3
14:23:04.369 283a8f00 Header: Tagger 'user-agent' added tag 'User-Agent: curl/7.16.3 (i386-portbld-freebsd7.0) libcurl/7.16.3 OpenSSL/0.9.8e zlib/1.2.3'. Action bits updated accordingly.
14:23:04.370 283a8f00 Header: scan: Host: www.apache.org
14:23:04.370 283a8f00 Header: scan: Pragma: no-cache
14:23:04.370 283a8f00 Header: scan: Accept: */*
14:23:04.370 283a8f00 Header: scan: Proxy-Connection: Keep-Alive
14:23:04.371 283a8f00 Header: scan: If-Modified-Since: Wed, 29 Mar 2006 06:51:34 GMT
14:23:04.371 283a8f00 Header: Tagger 'revalidation' added tag 'REVALIDATION-REQUEST'. No action bits update necessary.
14:23:04.371 283a8f00 Header: Modified: User-Agent: Mozilla/5.0 (X11; U; FreeBSD sparc64; nl-NL; rv:1.8.1.6) Gecko/20070805 Firefox/2.0.0.6
14:23:04.371 283a8f00 Header: Randomizing: If-Modified-Since: Wed, 29 Mar 2006 06:51:34 GMT (random range: -60 minutes)
14:23:04.371 283a8f00 Header: Randomized:  If-Modified-Since: Wed, 29 Mar 2006 06:01:14 GMT (subtracted 0 hours 50 minutes 20 seconds)
14:23:04.371 283a8f00 Header: crumble crunched: Proxy-Connection: Keep-Alive!
14:23:04.381 283a8f00 Header: Adding: Connection: close
14:23:04.381 283a8f00 Connect: Overriding forwarding settings based on 'forward-socks4a tor-jail:9050 .'
14:23:04.381 283a8f00 Header: New HTTP Request-Line: GET /favicon.ico HTTP/1.1
14:23:04.381 283a8f00 Redirect: Decoding /favicon.ico if necessary.
14:23:04.381 283a8f00 Redirect: Checking /favicon.ico for redirects.
14:23:04.381 283a8f00 Request: www.apache.org/favicon.ico
14:23:04.381 283a8f00 Connect: to www.apache.org
14:23:07.005 283a8f00 Header: scan: HTTP/1.1 304 Not Modified
14:23:07.005 283a8f00 Header: scan: Date: Sun, 23 Sep 2007 12:23:05 GMT
14:23:07.005 283a8f00 Header: scan: Server: Apache/2.3.0-dev (Unix)
14:23:07.005 283a8f00 Header: scan: Connection: close
14:23:07.005 283a8f00 Header: scan: ETag: "31a763-e36-66f00280"
14:23:07.005 283a8f00 Header: scan: Expires: Mon, 24 Sep 2007 12:23:05 GMT
14:23:07.005 283a8f00 Header: scan: Cache-Control: max-age=86400
10.0.0.1 - - [23/Sep/2007:14:23:07 +0200] "GET http://www.apache.org/favicon.ico HTTP/1.1" 200 0

There may be a bunch of other severs that behave like Gatling,
but I don't think their market share comes near to the one covered
by IIS and Apache alone.

Also RFC 2616 14.25 If-Modified-Since says:

|      c) If the variant has not been modified since a valid If-
|         Modified-Since date, the server SHOULD return a 304 (Not
|         Modified) response.

and only speaks of some servers that don't do that:

|      Note: When handling an If-Modified-Since header field, some
|      servers will use an exact date comparison function, rather than a
|      less-than function, for deciding whether to send a 304 (Not
|      Modified) response. To get best results when sending an If-
|      Modified-Since header field for cache validation, clients are
|      advised to use the exact date string received in a previous Last-
|      Modified header field whenever possible.

> > Therefore it's questionable if anyone is going to bother.  
> 
> We're in violent agreement.

I'm getting used to it.

Fabian

Attachment: signature.asc
Description: PGP signature