Antwort: Re: [Mod_gzip] "mod_gzip_send_vary=Yes" disables caching on IE? ...
mod_gzip@lists.over.net
mod_gzip@lists.over.net
Mon, 9 Dec 2002 16:05:41 EST
--part1_2b.32e089f9.2b265fa5_boundary
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit
Hi Michael...
Kevin here...
Wow... good info... thanks.
You are an animal!
I just love the way you can crunch down those logs!
So... what I think you are saying is that you definitely
KNOW of the behavior that Jordan says he is seeing
ALL the time... but you haven't nailed down anything
that might be causing it.
I think we need to hear from Slava again... who I
believe is on the opposite end of the spectrum and
says he is NEVER seeing this behavior.
Jordan - Sees the problem ALL the time
Michael - Doesn't (seem to) see it right now but has seen it before.
Slava - NEVER sees the problem.
If that's the way it's shaping up then this is going to
be a pretty big mystery to solve.
I also know for a fact that Tomaz Borstnar seems to be
in the same camp as Jordan. Tomaz runs a BIG site
with 500,000 hits per day and the moment he started
using mod_gzip 1.3.26 and started sending
"Vary: Accept-Encoding" headers his
hits have DOUBLED and it looks like nothing is getting
cached anywhere.
He had to go back to mod_gzip 1.3.19 and stop sending
any "Vary:" headers.
Later...
Kevin
In a message dated 12/9/2002 1:40:29 PM Central Standard Time,
Michael.Schroepl@telekurs.de writes:
> Hi Kevin,
>
>
> > Looks like Slava is saying he's not having any problems at
> > all and that everything is working fine while (Jordan?) says
> > that MSIE isn't caching ANY of the non-compressed variants
> > if they have ANY kind of "Vary:" header.
> >
> > Can we narrow this down to the specific problem and stop
> > spinning our wheels?
> > Is this just an MSIE version specific issue?
> > What (Exact) versions and build levels of MSIE are we talking
> > about here and does one really actually WORK while another does
> > NOT?
>
> I am running Apache 1.3.26 plus mod_gzip 1.3.19.1a, but
> with my own mod_headers configuration for sending "Vary:"
> headers (as 1.3.26.1a still has a minor bug).
> This scenario is running for about half a year now, with
> customers running any type of M$IE from 5.0 to 5.5 and
> 6.0 on this server.
>
> I am running some log file analysis tool (mgzta) that is
> in a way UserAgent specific. For example, I can see the
> rate of mod_gzip compression events based on the exact
> UserAgent string, as well as the average HTTP response
> size for each UserAgent string. In some cases these va-
> lues differ from each other because certain customers
> have specific network configurations; one of these even
> asked us to turn off mod_gzip for them because they need
> to print the pages with Netscape 4.7 as top priority.
>
> My users are working on the same server most every day.
> Therefore caching is a high priority issue for me.
> Besides, I am able to trace each individual user in my
> Apache logs (because they are forced to have unique coo-
> kie values from login, and I have these in access_log).
> Therefore I can easily compute the rate of HTTP status
> codes 200, 304 and others for each user, and I can com-
> pute tables that can be sorted by each of these values.
> I run CGI scripts doing just that.
> So what are they telling me?
>
> My logs contain 720 individual users for today, using 77
> different UserAgent strings. From these, I can sense 184
> that had not a single HTTP 304 request the whole day -
> they either are caching perfectly (they are supplied with
> "Expires:" headers for images that last for 1-2 weeks, and
> most HTML pages are either "no-cache" or have "Expires:"
> intervals as well) or not at all - but would then cause an
> unusual high number of HTTP requests because the pages in
> question contain large numbers of small images. And of all
> these 184 users I can identify not a single one who obvious-
> ly has configured his browser as to not cache anything at
> all (I would then detect certain HTTP request patterns at
> first sight), and most of them are far below average of
> total HTTP requests.
> On the other end of the table, there are 114 users who do
> more than 30% of all their HTTP requests for cache valida-
> tion - these seem to have a bad caching strategy configured,
> mostly they will run "validate always". I can detect all
> types of browsers there, M$IE of all versions as well as
> different Netscape 4 variants ... given the proper working
> style with our product, this may be as bad as 81% of all
> HTTP requests being of the 304 type (lots of small images
> for ertain pages ...). There is even one Netscape7 among
> these - you can do it wrong with every UserAgent. :-(
>
> All in all I have a HTTP-304 ratio of 16,5% today, which
> is the result of sending "Expires:" and "Cache-Control:"
> headers as well as including JavaScript and CSS mostly on
> server side (so they can be compressed for Netscape 4, the
> rate of compressed responses being slightly above 80% for
> HTML files, which make up for 60% of the requests but 90%
> of the uncompressed traffic volume, even when adding esti-
> mated 1 KB of HTTP request and response headers to each
> content size; overall traffic reduction is then 56%).
> Given the best possible browser configuration a user can
> easily reach values of below 10% of HTTP-304 values - one
> third of all our users are working this range, including
> all of my colleagues from my office. But that would requi-
> re all visitors to set their browser configuration to "au-
> tomatic", thus believing the HTTP headers we send them.
> Unlike the "Accept-Encoding:" header, I cannot detect the
> browser cache configuration by some individual HTTP header,
> thus I can't show the users a warning on the login page
> (which I do when they are not using compression). I may be
> able to see whether they do a conditional GET, but I cannot
> easily find out whether they should rather have relied on
> the expiration period I told them via the corresponding
> HTTP headers.
>
> All of these responses are getting "Vary: Accept-Encoding",
> regardless whether they are compressed or not - I want to
> "play fair" to whatever proxy may be out there (and there
> are Squids of all shapes and sizes at the locations of our
> customers, back to version 2.1, as well as some other pro-
> xies).
> What I actually _don't_ see is any special behaviour of
> any special M$IE version when dealing with "Vary:" headers.
> (I don't have Apple Macs as clients, only Windows, plus
> some OS/2 with Netscape 4.61.)
>
> On the other hand I somehow _do_ know a behaviour of M$IE
> similar to the one that was described in this thread. I can
> even reproduce it against one specific Apache server in our
> office (but only with M$IE, not with any other browser).
> I have another specific Apache running the same Apache and
> mod_gzip version and the same configuration as the aformen-
> tioned one, and the effect will _not_ occur there; I can re-
> produce both within the same browser session, for each M$IE
> version we have here, 5.0 as well as 5.5 or even 6.0.
> This behaviour has started some months ago, but I don't have
> any explanation for it ... but I definitely send identical
> HTTP headers from both servers, and there is no proxy in
> between my browser and either of them ...
>
> Regards, Michael
>
>
--part1_2b.32e089f9.2b265fa5_boundary
Content-Type: text/html; charset="US-ASCII"
Content-Transfer-Encoding: 7bit
<HTML><FONT FACE=arial,helvetica><FONT SIZE=2><BR>
Hi Michael...<BR>
Kevin here...<BR>
<BR>
Wow... good info... thanks.<BR>
<BR>
You are an animal!<BR>
<BR>
I just love the way you can crunch down those logs!<BR>
<BR>
So... what I think you are saying is that you definitely<BR>
KNOW of the behavior that Jordan says he is seeing<BR>
ALL the time... but you haven't nailed down anything<BR>
that might be causing it.<BR>
<BR>
I think we need to hear from Slava again... who I <BR>
believe is on the opposite end of the spectrum and<BR>
says he is NEVER seeing this behavior.<BR>
<BR>
Jordan - Sees the problem ALL the time<BR>
Michael - Doesn't (seem to) see it right now but has seen it before.<BR>
Slava - NEVER sees the problem.<BR>
<BR>
If that's the way it's shaping up then this is going to<BR>
be a pretty big mystery to solve.<BR>
<BR>
I also know for a fact that Tomaz Borstnar seems to be<BR>
in the same camp as Jordan. Tomaz runs a BIG site<BR>
with 500,000 hits per day and the moment he started<BR>
using mod_gzip 1.3.26 and started sending <BR>
"Vary: Accept-Encoding" headers his<BR>
hits have DOUBLED and it looks like nothing is getting<BR>
cached anywhere.<BR>
<BR>
He had to go back to mod_gzip 1.3.19 and stop sending<BR>
any "Vary:" headers.<BR>
<BR>
Later...<BR>
Kevin<BR>
<BR>
<BR>
<BR>
<BR>
In a message dated 12/9/2002 1:40:29 PM Central Standard Time, Michael.Schroepl@telekurs.de writes:<BR>
<BR>
<BR>
<BLOCKQUOTE TYPE=CITE style="BORDER-LEFT: #0000ff 2px solid; MARGIN-LEFT: 5px; MARGIN-RIGHT: 0px; PADDING-LEFT: 5px">Hi Kevin,<BR>
<BR>
<BR>
> Looks like Slava is saying he's not having any problems at<BR>
> all and that everything is working fine while (Jordan?) says<BR>
> that MSIE isn't caching ANY of the non-compressed variants<BR>
> if they have ANY kind of "Vary:" header.<BR>
><BR>
> Can we narrow this down to the specific problem and stop<BR>
> spinning our wheels?<BR>
> Is this just an MSIE version specific issue?<BR>
> What (Exact) versions and build levels of MSIE are we talking<BR>
> about here and does one really actually WORK while another does<BR>
> NOT?<BR>
<BR>
I am running Apache 1.3.26 plus mod_gzip 1.3.19.1a, but<BR>
with my own mod_headers configuration for sending "Vary:"<BR>
headers (as 1.3.26.1a still has a minor bug).<BR>
This scenario is running for about half a year now, with<BR>
customers running any type of M$IE from 5.0 to 5.5 and<BR>
6.0 on this server.<BR>
<BR>
I am running some log file analysis tool (mgzta) that is<BR>
in a way UserAgent specific. For example, I can see the<BR>
rate of mod_gzip compression events based on the exact<BR>
UserAgent string, as well as the average HTTP response<BR>
size for each UserAgent string. In some cases these va-<BR>
lues differ from each other because certain customers<BR>
have specific network configurations; one of these even<BR>
asked us to turn off mod_gzip for them because they need<BR>
to print the pages with Netscape 4.7 as top priority.<BR>
<BR>
My users are working on the same server most every day.<BR>
Therefore caching is a high priority issue for me.<BR>
Besides, I am able to trace each individual user in my<BR>
Apache logs (because they are forced to have unique coo-<BR>
kie values from login, and I have these in access_log).<BR>
Therefore I can easily compute the rate of HTTP status<BR>
codes 200, 304 and others for each user, and I can com-<BR>
pute tables that can be sorted by each of these values.<BR>
I run CGI scripts doing just that.<BR>
So what are they telling me?<BR>
<BR>
My logs contain 720 individual users for today, using 77<BR>
different UserAgent strings. From these, I can sense 184<BR>
that had not a single HTTP 304 request the whole day -<BR>
they either are caching perfectly (they are supplied with<BR>
"Expires:" headers for images that last for 1-2 weeks, and<BR>
most HTML pages are either "no-cache" or have "Expires:"<BR>
intervals as well) or not at all - but would then cause an<BR>
unusual high number of HTTP requests because the pages in<BR>
question contain large numbers of small images. And of all<BR>
these 184 users I can identify not a single one who obvious-<BR>
ly has configured his browser as to not cache anything at<BR>
all (I would then detect certain HTTP request patterns at<BR>
first sight), and most of them are far below average of<BR>
total HTTP requests.<BR>
On the other end of the table, there are 114 users who do<BR>
more than 30% of all their HTTP requests for cache valida-<BR>
tion - these seem to have a bad caching strategy configured,<BR>
mostly they will run "validate always". I can detect all<BR>
types of browsers there, M$IE of all versions as well as<BR>
different Netscape 4 variants ... given the proper working<BR>
style with our product, this may be as bad as 81% of all<BR>
HTTP requests being of the 304 type (lots of small images<BR>
for ertain pages ...). There is even one Netscape7 among<BR>
these - you can do it wrong with every UserAgent. :-(<BR>
<BR>
All in all I have a HTTP-304 ratio of 16,5% today, which<BR>
is the result of sending "Expires:" and "Cache-Control:"<BR>
headers as well as including JavaScript and CSS mostly on<BR>
server side (so they can be compressed for Netscape 4, the<BR>
rate of compressed responses being slightly above 80% for<BR>
HTML files, which make up for 60% of the requests but 90%<BR>
of the uncompressed traffic volume, even when adding esti-<BR>
mated 1 KB of HTTP request and response headers to each<BR>
content size; overall traffic reduction is then 56%).<BR>
Given the best possible browser configuration a user can<BR>
easily reach values of below 10% of HTTP-304 values - one<BR>
third of all our users are working this range, including<BR>
all of my colleagues from my office. But that would requi-<BR>
re all visitors to set their browser configuration to "au-<BR>
tomatic", thus believing the HTTP headers we send them.<BR>
Unlike the "Accept-Encoding:" header, I cannot detect the<BR>
browser cache configuration by some individual HTTP header,<BR>
thus I can't show the users a warning on the login page<BR>
(which I do when they are not using compression). I may be<BR>
able to see whether they do a conditional GET, but I cannot<BR>
easily find out whether they should rather have relied on<BR>
the expiration period I told them via the corresponding<BR>
HTTP headers.<BR>
<BR>
All of these responses are getting "Vary: Accept-Encoding",<BR>
regardless whether they are compressed or not - I want to<BR>
"play fair" to whatever proxy may be out there (and there<BR>
are Squids of all shapes and sizes at the locations of our<BR>
customers, back to version 2.1, as well as some other pro-<BR>
xies).<BR>
What I actually _don't_ see is any special behaviour of<BR>
any special M$IE version when dealing with "Vary:" headers.<BR>
(I don't have Apple Macs as clients, only Windows, plus<BR>
some OS/2 with Netscape 4.61.)<BR>
<BR>
On the other hand I somehow _do_ know a behaviour of M$IE<BR>
similar to the one that was described in this thread. I can<BR>
even reproduce it against one specific Apache server in our<BR>
office (but only with M$IE, not with any other browser).<BR>
I have another specific Apache running the same Apache and<BR>
mod_gzip version and the same configuration as the aformen-<BR>
tioned one, and the effect will _not_ occur there; I can re-<BR>
produce both within the same browser session, for each M$IE<BR>
version we have here, 5.0 as well as 5.5 or even 6.0.<BR>
This behaviour has started some months ago, but I don't have<BR>
any explanation for it ... but I definitely send identical<BR>
HTTP headers from both servers, and there is no proxy in<BR>
between my browser and either of them ...<BR>
<BR>
Regards, Michael<BR>
<BR>
</BLOCKQUOTE><BR>
<BR>
</FONT></HTML>
--part1_2b.32e089f9.2b265fa5_boundary--