[Mod_gzip] Why is the module load order important?

[Kevin Kiley]

Matthew...
Kevin Kiley here...

>: Kiley wrote...
>: There IS a special load order.
>: mod_gzip has to come LAST in the list of modules mentioned
>: in httpd.conf ( which means it actually comes FIRST in the
>: order of internal modules ).
>:
>Sorry, I jumped in on this in the middle.  What happens if it doesn't come
>last?  I'm loading the following modules (in this order)
>
>        mod_php4
>        mod_fastcgi
>        mod_ssl
>        mod_roaming
>        mod_perl
>        mod_gzip
>        mod_ruby
>        mod_jk
>
>and I haven't noticed any errors so far.  I haven't tried any mod_perl
scripts
>in a long time but everything seems to work so far.  I have to admit that
>php and ssl are the most obvious ones I would have noticed, though, and
they
>are above mod_gzip

Ok... hang to on to your hat and given my time contstraint today I will
try to answer this one as best I can for you ( and others ).

mod_gzip doesn't really HAVE to be loaded 'last'... but it SHOULD be.

I've been saving this explanation because it's complicated.

This is not an easy thing that mod_gzip is doing and no Apache
module ever written has been able to do what mod_gzip can do
so the explanation involves understanding a bit ( actually, a lot )
about how Apache actually works.

I have 'renamed' this thread to 'Why is the module load order important?'
so it will stand out better on the forum. I am definitely putting all these
dribs
and drabs of explanations together into a MANUAL for mod_gzip but
until a few bugs are fixed and I am sure no new commands are needed
I am trying to just handle these one at a time.

Ok... here we go...

The NEW mod_gzip uses the Apache 'type_checker' hook.
It had to do this for basically one reason... so things would work
with everything including 'ColdFusion'. ColdFusion is doing some
whacky shit in a 'type_checker' routine of their own inside
mod_coldfusion and so the lowest common denominator for
handling mod_php, mod_perl, mod_include, mod_coldfusion,
and mod_whatever turned out to be this 'type_checker' hook.

The way Apache works... if any module returns OK from the
low level type_checker hook then no other module with a
type_checker hook gets 'called'. This is what ColdFusion is
doing. It is 'stealing away' the type_checker chain early on and
no other module gets a chance to play once that happens.

So... the only way to get control of the output from ColdFusion
backend ( and other modules ) is to 'steal from the stealer'.

Here is where the 'load order' comes into play...

Apache allows the LAST 'LoadModule' binary listed in httpd.conf
to become the FIRST module called when it starts calling all the
low level 'hooks'. It's a 'last in first called' approach since they are
simply 'pushing' module pointers onto a 'stack' internally as they
are encountered in the httpd.conf file.

( Aside: This is what 'ClearModuleList' config command is really
for and why Apache has warnings in their ./Configure script
about 'Don't change load order unless you know what you are
doing'. The 'ClearModuleList' and subsequent 'AddModule'
commands are a way to re-order the module calling order
'on the fly' even for modules compiled into the core. )

If mod_gzip is not the LAST module 'loaded' in httpd.conf then
it can still compress everything coming out of any module that
comes BEFORE it... but there is no guarantee that it can 'intercept'
the data coming out of any module that is loaded AFTER mod_gzip.

To make things more complicated, however, the above is not
always the case. It has to do with whether or not a module is using
the 'type_checker' hook. If any module loaded AFTER mod_gzip
is not using a 'type_checker' hook then the output from that module
will probably still get compressed.

If, however, you load a module like mod_coldfusion ( which has its
own 'stealing' type checker' ) AFTER mod_gzip then the output of
ColdFusion will probably never pass through mod_gzip.

Confused? You should be. It's a confusing thing, for sure, but it's
the way Apache works.

The previous version of mod_gzip did not have the 'steal from the
stealer' low level hooks so only those 'mod_xxxxx' modules that
were 'playing fair' would allow mod_gzip to 'see' the output and
compress it.

There is still a standard 'mod_gzip_handler()' routine as there was
before... but it is now receiving its marching orders from the lower
level hooks which can 'see' everything that is happening and
'steal from the stealer'.

This is NOT 'hacking around in Apache'. mod_gzip follows the
rules for modules perfectly and is doing nothing that is not 'allowed'
in the current Apache module design scheme... it's just making some
pretty sophisticated use of what is 'allowable' and ends up being
'smarter' than any other module loaded into the Server.

As 'smart' as it is, however, it still is 'playing by the rules' and that
means the order in which modules are loaded can directly affect
what it is or is not able to do.

It is possible, with Apache, to load modules in any order and then
have any one particular module make sure it is the 'top dog' but
this, itself, invloves what even I would call 'hacking around' in
Apache. To do this ( and I have in previous test versions ) you
have to rewrite Apache's internal call tables in much the same way
you would re-arrange interrupt vector tables to re-position a device
driver in the interrupt chain. I have decided NOT to do this since that
approach become Apache 'version dependent' and you have to do
things differently based on what version of Apache it really is.

As it is... mod_gzip accomplishes the 'magic' and is totally version
independent. I have tested it all the way back to Apache 1.3.6 and
it still works fine. I don't even have a version of Apache older than
that but it will probably work all the way back to 1.2.x versions, if
anyone is crazy enough to still be using that old a version of Apache.

SUMMARY

In order to be SURE that mod_gzip can compress the output from ANY
other module either compiled into the server or dynamically loaded then
it should always be the LAST one loaded in your httpd.conf file.

It will STILL WORK if it is not the 'last one' but now it gets
'conditional'.

Here is the list you gave and I have added some comments out to the
right to explain what ( might ) happen...

>        mod_php4
>        mod_fastcgi
>        mod_ssl
>        mod_roaming
>        mod_perl
>        mod_gzip    -- All output from modules ABOVE this point can
definitely be compressed
>        mod_ruby   -- If this module has a 'type_checker' hook that
'steals' the chain then it might not work
>        mod_jk       -- Ditto. Output might be able to be compressed, might
not. It depends.

I hope that helps.

Again... there is no way to really understand why the load order is
important unless
you know a least a little about how Apache 'calls' modules at run-time. It's
complicated.

Later...
Kevin Kiley