[Mod_gzip] Mod_gzip and location: headers
mod_gzip@lists.over.net
mod_gzip@lists.over.net
Tue, 17 Feb 2004 23:25:33 EST
--part1_1d7.19e68610.2d64433d_boundary
Content-Type: text/plain; charset="US-ASCII"
Content-Transfer-Encoding: 7bit
Hi Jann...
This is Kevin Kiley again.
Thanks for the detail.
I think I see the problem.
It's in your embedded Per(script).
> You wrote...
>
> One of the first lines in /membership/sign_in.html says
>
> if ( $req_rec->prev )
> {
> $uri = $req_rec->prev->uri. '?' .$req_rec->prev->args;
> }
> else
> {
> $uri = $fdat{destination} || '/home.html';
> }
( God, I hate perl! - ROFL )
The simple check above for a 'previous' URI is not nearly
good enough to handle all the circumstances that you might
run into when trying to use 'other' 3rd party modules ( such
as mod_gzip ). It's simply too naive about the inner workings
of Apache to work under anything but the most simple
circumstances.
What's happening here is that the authors chose to use
an 'internal redirect' but paid no attention to the fact that
the request might ALREADY be ANOTHER internal redirect.
This would be the case when mod_gzip has been told to
compress the original URI request that's now being
're-directed' all over kingdom come.
mod_gzip, itself, uses an 'internal redirect' to get the job done.
Lots of Apache modules use 'internal redirects' and I doubt
that most of them would get along with this Perl script, either.
It's just not 'smart' enough, it seems, to handle 'redirects
withing redirects'.
I'm not even sure this would work if you tried to use Apache's
own mod_proxy with it. mod_proxy uses an 'internal redirect'
as well.
Not sure what to tell you at this point but I have the
following suggestions...
1. Do something simpler. It really sounds like you are trying
to go to New York via China. Find a simpler way to do the
login. Since it's just Cookie based and nothing more compicated
than that there are about 20 other ways you could go here.
2. Try using Apache 2.0 and the new 'mod_deflate' filter.
The 'mod_deflate' filter will compress your output and it
does NOT involve an 'internal redirect'. What mod_deflate
lacks is anything near the feature set of mod_gzip but
for basic compression under simple scenarios it seems
to work fine. ( Caveat: mod_deflate is slow. It uses regular
ZLIB libraries which you have to get/install yourself
whereas mod_gzip uses it's own real-time optimized
LZ77 compression routines built right into mod_gzip ).
3. Play with the Perl yourself a little and see if you can
get it smart enough to handle 'multiple redirects'.
Don't take me to the bank on this but something like
this MIGHT make it 'do the right thing' for at least one
more level or recursion ( the mod_gzip case )...
if ( $req_rec->prev )
{
// We are on an internal redirect...
// Is there 'another' level above this one?
if ( ($req_rec->prev)->prev )
{
// Yes... this appears to be a 'redirect within a redirect'... assume
// the req record at the NEXT level has the URI we want...
$uri = ($req_rec->prev)->prev->uri. '?' .$req_rec->prev-args;
}
else $uri = $req_rec->prev->uri. '?' .$req_rec->prev->args;
}
else
{
$uri = $fdat{destination} || '/home.html';
}
Something like that.
I will bet if you just add some code to 'print out' the URI names
of ALL the request records in the '->prev' chain at the point where
this Perlscript is executing you will see at which level the
URI you REALLY WANT actually exists.
Later...
Kevin
In a message dated 2/17/2004 12:14:27 PM Central Standard Time, jann@jann.com
writes:
>
> Hi Kevin:
>
> First of all, thanks so much for the quick response.
>
> The Location header was not what the error was. (thanks, though--i
> found this out the hard way)... Let me explain quickly:
>
> Using HTML::Embperl now, we are attempting to use mod_gzip as well.
>
> i have the following setup:
>
> <IfModule mod_gzip.c>
> mod_gzip_on Yes
> mod_gzip_can_negotiate Yes
> mod_gzip_command_version '/mod_gzip_status'
> mod_gzip_temp_dir /tmp
> mod_gzip_keep_workfiles No
> mod_gzip_minimum_file_size 500
> mod_gzip_maximum_file_size 500000
> mod_gzip_maximum_inmem_size 60000
> mod_gzip_min_http 1001
> mod_gzip_handle_methods GET POST
> mod_gzip_item_exclude reqheader "User-agent: Mozilla/4.0[678]"
> mod_gzip_item_include file \.html$
> mod_gzip_item_include mime ^text/html$
> mod_gzip_item_include mime ^text/plain$
> mod_gzip_item_include mime ^httpd/unix-directory$
> mod_gzip_item_exclude mime ^image/
> #mod_gzip_item_exclude rspheader "Location: .*"
> mod_gzip_item_exclude file \.css$
> mod_gzip_item_exclude file \.js$
> mod_gzip_dechunk Yes
> LogFormat "%h %l %u %t \"%V %r\" %In:%{mod_gzip_input_size}n -<
> Out:%{mod_gzip_output_size}n =
> %{mod_gzip_compression_ratio}n pct." common_with_mod_gzip_info2
> CustomLog /usr/local/apache/logs/mod_gzip_log common_with_mod_gzip_info2
> mod_gzip_add_header_count Yes
> mod_gzip_send_vary Yes
>
>
>
>
> The issue is that for authentication on the website, we use
> Apache::AuthCookie, which does the following:
>
> When it hits: http://www1.dinnerbroker.com/my_account/this_url.html
>
> Authcookie says: Hey, that area is protected: Load up the
> corresponding login page using an internal redirect.
>
> that page is: /membership/sign_in.html
>
> THAT page submits to /LOGIN which AuthCookie uses to get the login
> info, sets a few cookies and forwards the user back to:
> /membership/sign_in.html using another internal redirect with all the
> cookies set.
>
> This happens because Apache sets $rec_req->prev to the original url
> name (in this case /my_account/this_url.html) and the second time
> through, after the valid login, /membership/sign_in.html sees that
> $rec_req->prev has a value (again, in this case:
> /my_account/this_url.html).
>
> One of the first lines in /membership/sign_in.html says:
>
> if ($req_rec->prev)
> {
> $uri = $req_rec->prev->uri. '?' .$req_rec->prev->args ;
> }
> else
> {
> $uri = $fdat{destination} || '/home.html';
> }
>
> well... $req_rec->prev never is set... (and it IS set when i do NOT use
> mod_gzip)
>
> i do not know why.
>
> (ps: $req_rec is the HTML::Embperl variable holding the passed
> request_handler object)
>
> Understand? If so, can you tell me why this works when mod_gzip is not
> involved yet does not work WITH mod_gzip?
>
> Thanks in advance for any help.
>
>
> Mr. Jann Linder
> DinnerBroker
> San Francisco, CA
> 888.432.8288 x5249
> On Feb 17, 2004, at 12:31 AM, TOKILEY@aol.com wrote:
>
> >
> > Hi Jann...
> > This is Kevin Kiley...
> >
> > > earlier it was stated that some people are having issues when you
> > use
> > > mod_gzip and some places on your site forward the user to another
> > page
> > > using a Location: header.
> >
> > Didn't see those messages but I believe you.
> >
> > > Essentially mod_gzip eats the entire request and the browser never
> > gets
> > > the Location: request.
> >
> > About the only way I can think of that this might happen is if the
> > Server is brain-dead enough to actually make it look like it's
> > going to deliver the real content but sends just a "Location:"
> > header instead of the real data.
> >
> > What I mean is... if the Server puts together an actual '200' response
> > and it also adds a valid "Content-type:" header but then just tacks
> > on a "Location:" header then ANY 'content filter' you might be running
> > is going to have a problem with that. It will think that data is
> > supposed
> > to be coming ( of 'Content-type: X' ) but nothing ever comes from
> > the Server.
> >
> > > Anyone know of a workaround or solution (short of NOT using
> > Location:
> > > headers).
> >
> > Sure...
> >
> > mod_gzip_item_exclude rspheader "Location: *"
> >
> > Just make sure mod_gzip doesn't try to compress something
> > when nothing is going to be sent by the Server.
> >
> > In all cases when 'certain' response header fields should
> > automatically
> > exclude a response from being 'touched' by mod_gzip the solution
> > is to add an 'exclusion' record that uses the 'rspheader' keyword.
> >
> > That's what it's there for.
> >
> > Later...
> > Kevin
> >
> > PS: Do you have a good capture of a response with a "Location:" header
> > in it that doesn't work? I'd be curious to see exactly what OTHER
> > fields
> > the Server is putting out along with the "Location:" header. If it's
> > a 200
> > response that also has a valid "Content-type:" and that content-type
> > is being included in the list of things to be 'compressed' then that
> > could be the problem right there.
> >
> > PPS: We ARE talking about "Location:" header and not
> > "Content-Location:",
> > right? They are 2 different animals... but even if
> > "Content-Location:" was
> > causing weirdness the solution would be the same 'exclusion' using
> > rspheader keyword.
> >
> > In a message dated 2/17/2004 12:08:05 AM Central Standard Time,
> > jann@jann.com writes:
> >
> >
> >
> > earlier it was stated that some people are having issues when you use
> > mod_gzip and some places on your site forward the user to another page
> > using a Location: header.
> >
> > Essentially mod_gzip eats the entire request and the browser never
> > gets
> > the Location: request.
> >
> > HLP!
> >
> > Anyone know of a workaround or solution (short of NOT using Location:
> > headers).
> >
> >
> >
> >
> > Mr. Jann Linder
> > DinnerBroker
> > San Francisco, CA
> > 888.432.8288 x5249
> >
> > _______________________________________________
> > mod_gzip mailing list
> > mod_gzip@lists.over.net
> > http://lists.over.net/mailman/listinfo/mod_gzip
> >
> >
> >
>
>
--part1_1d7.19e68610.2d64433d_boundary
Content-Type: text/html; charset="US-ASCII"
Content-Transfer-Encoding: quoted-printable
<HTML><FONT FACE=3Darial,helvetica><HTML><FONT SIZE=3D2 PTSIZE=3D10><BR>
Hi Jann...<BR>
This is Kevin Kiley again.<BR>
<BR>
Thanks for the detail.<BR>
I think I see the problem.<BR>
It's in your embedded Per(script).<BR>
<BR>
> You wrote...<BR>
><BR>
> One of the first lines in /membership/sign_in.html says<BR>
><BR>
> if ( $req_rec->prev )<BR>
> {<BR>
> $uri =3D $req_rec->prev->uri. '?' .$req_rec->prev->args;<BR=
>
> }<BR>
> else<BR>
> {<BR>
> $uri =3D $fdat{destination} || '/home.html';<BR>
> }<BR>
<BR>
( God, I hate perl! - ROFL )<BR>
<BR>
The simple check above for a 'previous' URI is not nearly<BR>
good enough to handle all the circumstances that you might<BR>
run into when trying to use 'other' 3rd party modules ( such<BR>
as mod_gzip ). It's simply too naive about the inner workings<BR>
of Apache to work under anything but the most simple <BR>
circumstances.<BR>
<BR>
What's happening here is that the authors chose to use <BR>
an 'internal redirect' but paid no attention to the fact that<BR>
the request might ALREADY be ANOTHER internal redirect.<BR>
<BR>
This would be the case when mod_gzip has been told to<BR>
compress the original URI request that's now being <BR>
're-directed' all over kingdom come.<BR>
<BR>
mod_gzip, itself, uses an 'internal redirect' to get the job done.<BR>
<BR>
Lots of Apache modules use 'internal redirects' and I doubt<BR>
that most of them would get along with this Perl script, either.<BR>
It's just not 'smart' enough, it seems, to handle 'redirects<BR>
withing redirects'.<BR>
<BR>
I'm not even sure this would work if you tried to use Apache's<BR>
own mod_proxy with it. mod_proxy uses an 'internal redirect'<BR>
as well.<BR>
<BR>
Not sure what to tell you at this point but I have the <BR>
following suggestions...<BR>
<BR>
1. Do something simpler. It really sounds like you are trying<BR>
to go to New York via China. Find a simpler way to do the<BR>
login. Since it's just Cookie based and nothing more compicated<BR>
than that there are about 20 other ways you could go here.<BR>
<BR>
2. Try using Apache 2.0 and the new 'mod_deflate' filter.<BR>
The 'mod_deflate' filter will compress your output and it<BR>
does NOT involve an 'internal redirect'. What mod_deflate<BR>
lacks is anything near the feature set of mod_gzip but<BR>
for basic compression under simple scenarios it seems<BR>
to work fine. ( Caveat: mod_deflate is slow. It uses regular<BR>
ZLIB libraries which you have to get/install yourself <BR>
whereas mod_gzip uses it's own real-time optimized<BR>
LZ77 compression routines built right into mod_gzip ).<BR>
<BR>
3. Play with the Perl yourself a little and see if you can<BR>
get it smart enough to handle 'multiple redirects'.<BR>
Don't take me to the bank on this but something like<BR>
this MIGHT make it 'do the right thing' for at least one<BR>
more level or recursion ( the mod_gzip case )...<BR>
<BR>
if ( $req_rec->prev )<BR>
{<BR>
// We are on an internal redirect...<BR>
// Is there 'another' level above this one?<BR>
<BR>
if ( ($req_rec->prev)->prev )<BR>
{<BR>
// Yes... this appears to be a 'redirect within a redirect'...=20=
assume<BR>
// the req record at the NEXT level has the URI we want...<BR>
<BR>
$uri =3D ($req_rec->prev)->prev->uri. '?' .$req_rec-&g=
t;prev-args;<BR>
}<BR>
else $uri =3D $req_rec->prev->uri. '?' .$req_rec->prev->args;<BR=
>
}<BR>
else<BR>
{<BR>
$uri =3D $fdat{destination} || '/home.html';<BR>
}<BR>
<BR>
Something like that.<BR>
<BR>
I will bet if you just add some code to 'print out' the URI names<BR>
of ALL the request records in the '->prev' chain at the point where<BR>
this Perlscript is executing you will see at which level the <BR>
URI you REALLY WANT actually exists.<BR>
<BR>
Later...<BR>
Kevin<BR>
<BR>
<BR>
In a message dated 2/17/2004 12:14:27 PM Central Standard Time, jann@jann.co=
m writes:<BR>
<BR>
<BR>
<BLOCKQUOTE TYPE=3DCITE style=3D"BORDER-LEFT: #0000ff 2px solid; MARGIN-LEFT=
: 5px; MARGIN-RIGHT: 0px; PADDING-LEFT: 5px"><BR>
Hi Kevin:<BR>
<BR>
First of all, thanks so much for the quick response.<BR>
<BR>
The Location header was not what the error was. (thanks, though--i <BR=
>
found this out the hard way)... Let me explain quickly:<BR>
<BR>
Using HTML::Embperl now, we are attempting to use mod_gzip as well.<BR>
<BR>
i have the following setup:<BR>
<BR>
<IfModule mod_gzip.c><BR>
mod_gzip_on Yes<BR>
mod_gzip_can_negotiate Yes<BR>
mod_gzip_command_version '/mod_gzip_status'<BR>
mod_gzip_temp_dir /tmp<BR>
mod_gzip_keep_workfiles No<BR>
mod_gzip_minimum_file_size 500<BR>
mod_gzip_maximum_file_size 500000<BR>
mod_gzip_maximum_inmem_size 60000<BR>
mod_gzip_min_http 1001<BR>
mod_gzip_handle_methods GET POST<BR>
mod_gzip_item_exclude reqheader "User-agent: Mozilla/4.0[678]"<BR>
mod_gzip_item_include file \.html$<BR>
mod_gzip_item_include mime ^text/html$<BR>
mod_gzip_item_include mime ^text/plain$<BR>
mod_gzip_item_include mime ^httpd/unix-directory$<BR>
mod_gzip_item_exclude mime ^image/<BR>
#mod_gzip_item_exclude rspheader "Location: .*"<BR>
mod_gzip_item_exclude file \.css$<BR>
mod_gzip_item_exclude file \.js$<BR>
mod_gzip_dechunk Yes<BR>
LogFormat "%h %l %u %t \"%V %r\" %<S>In:%{mod_gzip_input_size}n -< Out:%{=
mod_gzip_output_size}n =3D <BR>
%{mod_gzip_compression_ratio}n pct." common_with_mod_gzip_info2<BR>
CustomLog /usr/local/apache/logs/mod_gzip_log common_with_mod_gzip_info2<BR>
mod_gzip_add_header_count Yes<BR>
mod_gzip_send_vary Yes<BR>
<BR>
<BR>
<BR>
<BR>
The issue is that for authentication on the website, we use <BR>
Apache::AuthCookie, which does the following:<BR>
<BR>
When it hits: http://www1.dinnerbroker.com/my_account/this_url.html<BR>
<BR>
Authcookie says: Hey, that area is protected: Load up the <BR>
corresponding login page using an internal redirect.<BR>
<BR>
that page is: /membership/sign_in.html<BR>
<BR>
THAT page submits to /LOGIN which AuthCookie uses to get the login <BR>
info, sets a few cookies and forwards the user back to: <BR>
/membership/sign_in.html using another internal redirect with all the <BR>
cookies set.<BR>
<BR>
This happens because Apache sets $rec_req->prev to the original url <BR>
name (in this case /my_account/this_url.html) and the second time <BR>
through, after the valid login, /membership/sign_in.html sees that <BR>
$rec_req->prev has a value (again, in this case: <BR>
/my_account/this_url.html).<BR>
<BR>
One of the first lines in /membership/sign_in.html says:<BR>
<BR>
if ($req_rec->prev)<BR>
{<BR>
$uri =3D $req_rec->prev->uri. '?' .$req_rec->prev->args ;<BR>
}<BR>
else<BR>
{<BR>
$uri =3D $fdat{destination} || '/home.html';<BR>
}<BR>
<BR>
well... $req_rec->prev never is set... (and it IS set when i do NOT use <=
BR>
mod_gzip)<BR>
<BR>
i do not know why.<BR>
<BR>
(ps: $req_rec is the HTML::Embperl variable holding the passed <BR>
request_handler object)<BR>
<BR>
Understand? If so, can you tell me why this works when mod_gzip is not <BR>
involved yet does not work WITH mod_gzip?<BR>
<BR>
Thanks in advance for any help.<BR>
<BR>
<BR>
Mr. Jann Linder<BR>
DinnerBroker<BR>
San Francisco, CA<BR>
888.432.8288 x5249<BR>
On Feb 17, 2004, at 12:31 AM, TOKILEY@aol.com wrote:<BR>
<BR>
><BR>
> Hi Jann...<BR>
> This is Kevin Kiley...<BR>
><BR>
> > earlier it was stated that some people are having issues when you=20=
<BR>
> use<BR>
> > mod_gzip and some places on your site forward the user to another=20=
<BR>
> page<BR>
> > using a Location: header.<BR>
><BR>
> Didn't see those messages but I believe you.<BR>
><BR>
> > Essentially mod_gzip eats the entire request and the browser never=
<BR>
> gets<BR>
> > the Location: request.<BR>
><BR>
> About the only way I can think of that this might happen is if the<BR>
> Server is brain-dead enough to actually make it look like it's<BR>
> going to deliver the real content but sends just a "Location:"<BR>
> header instead of the real data.<BR>
><BR>
> What I mean is... if the Server puts together an actual '200' response<=
BR>
> and it also adds a valid "Content-type:" header but then just tacks<BR>
> on a "Location:" header then ANY 'content filter' you might be running<=
BR>
> is going to have a problem with that. It will think that data is <BR>
> supposed<BR>
> to be coming ( of 'Content-type: X' ) but nothing ever comes from<BR>
> the Server.<BR>
><BR>
> > Anyone know of a workaround or solution (short of NOT using <BR>
> Location:<BR>
> > headers).<BR>
><BR>
> Sure...<BR>
><BR>
> mod_gzip_item_exclude rspheader "Location: *"<BR>
><BR>
> Just make sure mod_gzip doesn't try to compress something<BR>
> when nothing is going to be sent by the Server.<BR>
><BR>
> In all cases when 'certain' response header fields should <BR>
> automatically<BR>
> exclude a response from being 'touched' by mod_gzip the solution<BR>
> is to add an 'exclusion' record that uses the 'rspheader' keyword.<BR>
><BR>
> That's what it's there for.<BR>
><BR>
> Later...<BR>
> Kevin<BR>
><BR>
> PS: Do you have a good capture of a response with a "Location:" header<=
BR>
> in it that doesn't work? I'd be curious to see exactly what OTHER <BR>
> fields<BR>
> the Server is putting out along with the "Location:" header. If it's <B=
R>
> a 200<BR>
> response that also has a valid "Content-type:" and that content-type<BR=
>
> is being included in the list of things to be 'compressed' then that<BR=
>
> could be the problem right there.<BR>
><BR>
> PPS: We ARE talking about "Location:" header and not <BR>
> "Content-Location:",<BR>
> right? They are 2 different animals... but even if <BR>
> "Content-Location:" was<BR>
> causing weirdness the solution would be the same 'exclusion' using<BR>
> rspheader keyword.<BR>
><BR>
> In a message dated 2/17/2004 12:08:05 AM Central Standard Time, <BR>
> jann@jann.com writes:<BR>
><BR>
><BR>
><BR>
> earlier it was stated that some people are having issues when you use<B=
R>
> mod_gzip and some places on your site forward the user to another page<=
BR>
> using a Location: header.<BR>
><BR>
> Essentially mod_gzip eats the entire request and the browser never <BR>
> gets<BR>
> the Location: request.<BR>
><BR>
> HLP!<BR>
><BR>
> Anyone know of a workaround or solution (short of NOT using Location:<B=
R>
> headers).<BR>
><BR>
><BR>
><BR>
><BR>
> Mr. Jann Linder<BR>
> DinnerBroker<BR>
> San Francisco, CA<BR>
> 888.432.8288 x5249<BR>
><BR>
> _______________________________________________<BR>
> mod_gzip mailing list<BR>
> mod_gzip@lists.over.net<BR>
> http://lists.over.net/mailman/listinfo/mod_gzip<BR>
><BR>
><BR>
><BR>
<BR>
</BLOCKQUOTE></S><BR>
<BR>
</FONT></HTML>
--part1_1d7.19e68610.2d64433d_boundary--