Parent Directory
| 
Revision Log
| Links to HEAD: | (view) (annotate) |
| Sticky Revision: |
core, mod_deflate, mod_sed: Reduce memory usage by reusing bucket brigades in several places
Adjust content metadata on deflate/inflate response before bailing out on a 304 response so that the metadata does not differ from 200 response.
Reimplement deflate_check_etag() so that it isn't such a pig and correctly works with weak etags. Related to PR 39727. Note that there remains an error in ap_meets_conditions because that function does not currently check for a transformed etag. I am working on that.
* Only drop the last char (the '"') and not the last one of the etag itself.
mod_deflate: Fix creation of invalid Etag headers. We now make sure that the Etag value is properly quoted when adding the gzip marker. PR 39727.
* Fix r->content_encoding for inflate_in and inflate_out filters.
* Fix r->content_encoding in deflate_out_filter if it was set before.
PR 39727: Fixup ETag handling in mod deflate (updated following extensive discussion on-list). This is not a full-and-final fix, because we don't ourselves do anything useful with these ETags. But at least we're no longer screwing up clients.
mod_deflate: Don't leave a strong ETag in place while transforming the entity. PR 39727 Comment: Another user just subscribed to this bug. We need a fix more than we need an inconclusive discussion!
mod_deflate: initialise inflate-out filter correctly when the first brigade contains no data buckets. PR 43512
Correcting defects in r563317: fix to PR 42993
Check all sources of Content-Encoding in inflate_out filter PR 42993 Reasoning: http://marc.info/?l=apache-httpd-dev&m=118643107831358&w=2
* Also unset Content-MD5 in the deflate_out_filter
* We already unset Content-Length in the inflate_out_filter. So move the comment and unsetting of Content-MD5 in the right place.
Don't try to compress/decompress where there's a Content-Range. According to RFC2616, the range would have to apply *after* applying content-encoding, so anything that's been set before running the deflate filter is going to be broken.
mod_deflate - discussion on dev@ Postpone unsetting of headers until after first get_brigade from upstream
Generalise the content encoding detection and protocol: so it aslo works in inflate out filter, as suggested by rpluem. NOTE: this fails with some generators (cgi, asis) due to a deeper bug: content-encoding is set later than mod_deflate sees it. This has always been the case, and could use a separate fix if anyone wants inflate_out other than in a proxy situation. And it's not mod_deflate's problem.
Be liberal in what we accept. Where it matters.
Be liberal and correct in what we accept
Update r560689 following rpluem's analysis thereof.
Fix protocol handling in mod_deflate input filter PR 23287
don't need the definitions for error buckets any more
* generalize from error buckets to metadata buckets * also applicable to the deflate input filter * pass along metadata buckets rather than giving up mid-stream vastly improved by: Rüdiger Plüm
with LimitRequestBody xxx and the deflate output filter configured, mod_deflate eats the 413 error bucket, a 500 error is logged, and a blank screen appears at the browser.
Housekeeping: keep track of size even in the edge-case where validation bytes trickle in over multiple buckets.
* Fix some comments. No functional change.
* Also log the presence of garbage data at the end of the stream if all validation data is available in the first round.
* Actually append new data to the validation buffer and do not overwrite old data already there by setting the correct offset for the target buffer.
* Ensure that we do not perform our final operations twice if a second EOS falls down the chain by accident. Otherwise we are likely to run in a SEGFAULT. So remove ourselves from the chain.
* Ensure that we do not try to inflate validation data or garbage data. None of this is zlib's business.
* Rework inflate out filter and give it a similar workflow as the deflate out
filter. This fixes the following bugs in the inflate out filter:
- Incorrect handling of flush buckets.
- Excessive memory usage for large compressed content (because we now
already sent parts down the chain and do not process the whole brigade
first before sending something down the chain).
- Handle the case correctly where the validation bytes at the end of
the compressed data stream are distributed across different buckets /
brigades.
- Fix a memory leak due to not cleaning up the internal structures of
libz in some error cases.
PR: 39854
* some optimizations taken from the inflate out filter
* We can ignore Z_BUF_ERROR in flush_libz_buffer because:
When we call libz_func we can assume that
- avail_in is zero (due to the surrounding code that calls
flush_libz_buffer)
- avail_out is non zero due to the fact that we just emptied
the output buffer and stored it into a brigade
So the only reason for Z_BUF_ERROR is that the internal libz
buffers are now empty and thus we called libz_func one time
too often. This does not hurt. It simply says that we are done.
* Use a define for the number of validation bytes (CRC and length) after the compressed data
* Add parameter crc to flush_libz_buffer in order to call the libz's crc32 function on the output buffer if needed. This is actually needed by the later rework of the inflate out filter.
* Rather use a pool cleanup function than calling deflateEnd before every return from function to ensure that libz's internals get cleaned up.
* Initialize zRC to avoid a compiler warning.
* This shortcut is too short. It is not up to the filters to decide whether filters down the chain can do something useful with this empty brigade.
* In preparation of the changes of the inflate out filter:
- rename flush_zlib_buffer to flush_libz_buffer (this name seems
better)
- add a parameter to tell flush_libz_buffer whether it should
deflate or inflate as this function should be also used for the
inflate out filter.
* Remove ourselves from the filter chain if we failed to init libz, as we pass data down the filter chain uncompressed afterwards.
* Fix potential memory leaks in deflate_out_filter if bailing out due to an error (either in the lower filter chain or during a libz operation). We need to call deflateEnd as it is very likely that this filter is never called again to ensures that libz's internal structures get cleaned properly.
* Fix wrong FLUSH bucket handling in deflate_out_filter: Actually the internal structures of libz never got flushed as ctx->stream.avail_in is always zero here. As the EOS and the FLUSH bucket case use the same code for flushing libz's internal buffers move this code to the new function flush_zlib_buffer.
update license header text
PR#39854 Remove bogus code that chokes on flush buckets
Update the copyright year in all .c, .h and .xml files
Correctly handle internal redirects, by testing only for r-main == NULL. ap_is_initial_req() returns false for internal redirects.
No functional Change: Removing trailing whitespace. This also means that "blank" lines consisting of just spaces or tabs are now really blank lines
- mod_deflate currently only does C-E, and not T-E. Fix the comment at the top saying it does T-E.
mod_deflate should be merging the Vary header, not Setting it, and ignoring what other modules put there. Spotted By: Rici Lake
Update copyright year to 2005 and standardize on current copyright owner line.
* modules/filters/mod_deflate.c (deflate_out_filter): Check explicitly for a 304 or 204 response rather than a brigade which begins with EOS, to correctly handle such responses when generated by a CGI script. Don't alter the headers for this case either.
general property cleanup
* modules/filters/mod_deflate.c: Remove use of zutil.h (not installed by zlib 1.2.1) and OS_CODE. (deflate_out_filter): Simplify to use an immortal brigade for the gzip header. PR: 28673
Use the ap_is_initial_req() function to test for a subrequest instead of using r->main.
Tidy up handling of empty brigades (Joe Orton); Ensure we still set headers on HEAD and 204/304; and fix inflate_out_filter handling of HEAD/etc.
Tidy up: get rid of dubious cast (nd) and initialise encoding.
Fix previous patch to deal correctly with multiple empty brigades before we know if there's any content, and not re-process the headers.
Fix headers in deflate_out_filter for the case of an empty first brigade
style
that's not really an error condition, so make the message APLOG_DEBUG.
Update inflate_out_filter to support gzip compression flags.
Don't deflate responses with zero length e.g. proxied 304's
* modules/filters/mod_deflate.c (deflate_out_filter): Destroy buckets immediately after are used so that memory consumption is not proportional to the size of the response. PR: 29318
Fix bug in mod_deflate that unconditionally sent deflate'd output even when Accept-Encoding is not present. Reported by Roy Fielding for http://cvs.apache.org/viewcvs/ with Safari.
content-encoding needs to be unset. Submitted by: Nick Kew <nick webthing.com>
remove unused variables
mod_deflate: - New option for DEFLATE output file (force-gzip), which skips checking the accept-encoding header. - New output filter 'INFLATE' for uncompressing responses. Submitted by: Nick Kew <Nick at WebThing dot com> Reviewed by: Ian Holsman
fix name of The Apache Software Foundation
fix copyright dates according to the first checkin
apply Apache License, Version 2.0
remove email address in the hopes ppl stop emailing me directly ;(
update license to 2004.
get rid of _FOREACH
here applies the same. Don't skip the \0 delimiter when searching for already applied encodings. Additionally don't compress if *any* non-identity encoding was applied before. (deflate, pkzip, whatever).
Doh. If Accept-Encoding contains no gzip token, we skip the 0 delimiter and search for "gzip" somehwere in the memory. This was originally discovered by Joe Orton. But there's more. We must skip any parameters, since these do not contain what we're looking for. PR: 21523
Fix bug in mod_deflate, which was passing calling deflate()
without checkinig first whether it has something to deflate. (currently
this causes deflate to generate a fatal error according to the zlib spec).
PR 22259.
PR:
Obtained from:
Submitted by:
Reviewed by:
Don't attempt to hold all of the response until we're done. We'll pass data on when the zlib buffer becomes full and we need to reset the buffer anyway. Also, tidy up a similar semantic when we see the EOS by returning an error if ap_pass_brigade gets an error (we'd lose it otherwise).
Check also for r->content_encoding (now we should have them all...) for already compressed content in mod_deflate. PR: 19913 Submitted by: Tsuyoshi SASAMOTO <nazonazo@super.win.ne.jp>
reflect backport of mod_deflate.c r1.31 change directive from compressionlevel to deflatecompressionlevel PR: Obtained from: Submitted by: Ian Holsman, Andre Malo Reviewed by:
Check also err_headers_out for an already set Content-Encoding: gzip header. This prevents gzip compressed content from a CGI script from being compressed once more. PR: 17797
new directive 'compressionlevel' Default compression level now changed to Zlib's default (was 'best_speed') Obtained from: Stephen Pierzchala <stephen@pierzchala.com> Michael Schroepl <Michael.Schroepl@telekurs.de> Reviewed by: Ian Holsman
Fix potential memory leaks in mod_deflate on malformed input data. PR: 16046
finished that boring job: update license to 2003. Happy New Year! ;-))
break down macro code and avoid mixed case. Thanks to Justin for his hints.
The patch allows the user to log the accurate filter input and output byte count, instead of only the rounded compression ratio. The DeflateFilterNote directive will be extended as follows: DeflateFilterNote [type] name type can be one of "input", "output" or "ratio". "ratio" is assumed if the type is omitted (backwards compatible).
Fix a memory leak in mod_deflate with dynamic content. Because the brigade was being destroyed (ie, cleanup unregistered) rather than just emptied out and then reused anyway, the last call down the filter stack would leak buckets. PR: 14321 Submitted by: Ken Franken <kfranken@decisionmark.com> Reviewed by: Jeff Trawick, Cliff Woolley
Always emit Vary header if mod_deflate is involved in the request. Submitted by: Andr��Malo <nd@perlig.de> Reviewed by: Justin Erenkrantz
actuall fix the bug
deflate's filter removal was reversed for non-html files, resulting in it never being applied for non-html files Submitted by: Kris Verbeeck <Kris.Verbeeck@ubizen.com>
mod-deflate now checks to make sure gzip-only-text-html is set to 1, not anything. This will allow things like... SetEnv gzip-only-text/html 1 BrowserMatch "GECKO" gzip-only-text/html=0 Obtained from: Andr� Schild <A.Schild@aarboard.ch> Reviewed by: Ian Holsman
Add a filter_init function to the filters so that a filter can execute arbitrary code before the handlers are invoked. This resolves an issue with incorrect 304s on If-Modified-Since mod_include requests since ap_meets_conditions() is not aware that this is a dynamic request and it is not possible to satisfy 304 for these requests (unless xbithack full is on, of course). When mod_include runs as a filter, it is too late to set any flag since the handler is responsible for calling ap_meets_conditions(), which it should do before generating any data. If a module doesn't need to run such arbitrary code, it can just pass NULL as the argument and all is well. PR: 9673 Reviewed by: Ryan Bloom and others
Clean up a signedness emit
Modify the deflate input filter so that it should always return data on a blocking read.
An important observations before some x86'er attacks this idea.
get the prototype for strcasecmp() on AIX
Oops. When deflating: only reset ctx->stream buffers when the ctx was NULL. When inflating: remember to clear the ctx->bb after we're done with it.
Why the stack buffer and the copy? This seems much more straightforward.
keep the signed-ness of char buffers consistent with what is expected by functions they are passed to (putLong() this time)
Unroll the putLong and getLong loops and correct putLong's input param type to be unsigned char* instead of char*. Submitted by: Sander Striker Reviewed by: Justin Erenkrantz
Make sure that z_RC is not Z_STREAM_END in case we get a 0-length bucket and we don't enter the while loop. Submitted by: Jeff Trawick's horde of picky compilers
get mod_deflate to compile with compilers that care about the signed-ness of char * it looks like it should work fine, but it is untested
Input filter support for mod_deflate. Highly experimental, but known to work for some tests.
Add a note indicating what it would take to get MTIME field populated. (I did this locally, but didn't figure it was worth committing, but it's still worth noting.)
Implement flushing support for mod_deflate. (Review of RFC 1952 indicates that this is indeed easily allowable.)
Allow mod_deflate to be more tolerant of multiple content-encodings. - Tighten scope of a few affected variables (accepts, token). - Look for gzip in r->headers_out (!) - if there, we've already compressed. (Previously, we'd look at r->headers_in which was bogus - we're output.) - Switch the Accept-Encoding check to be case-insensitive per RFC 2616. - Remove commented-out code. - If C-E wasn't present or it was just "identity," set C-E to "gzip", otherwise, merge "gzip" onto the C-E list. This patch was originally submitted by Ian and cleaned up and tested by Justin. Submitted by: Ian Holsman Reviewed by: Justin Erenkrantz
content with "Content-Encoding" header, content is encoded. But mod_deflate does not check it. It cause to encode content twice. This problem is reproducable by getting encoded content via mod_proxy. Patch Contributed by kaz@asada.sytes.net (ASADA Kazuhisa) Bug #9222 Thanks Kazuhisa
stop using APLOG_NOERRNO in calls to ap_log_[pr]error()
Fix mod_deflate corruption when using multiple input buckets. We should only clear *_out when we are the beginning or avail_out == 0. Justin was tracking this down before he went to WWDC and didn't finish before he left. A (virtual) beer goes to Asada who filed this PR with the right patch! PR: 9014 Submitted by: kaz@asada.sytes.net (ASADA Kazuhisa) Reviewed by: Justin Erenkrantz
Add DeflateBufferSize directive to replace hardcoded FILTER_BUFSIZE definition. Also speeds up ctx init since it takes the buffer and does a malloc rather than a calloc on it.
If we determine that we shouldn't be involved in this request, remove ourselves from the filter chain (so we don't duplicate this path each time a brigade comes through).
Move mod_deflate from experimental to filters to represent our belief in its stability.
This form allows you to request diffs between any two revisions of this file. For each of the two "sides" of the diff, enter a numeric revision.
| apache@apache.org | ViewVC Help |
| Powered by ViewVC 1.1.2 |