Log Message: |
Fix the unbounded memory usage when mod_dav_svn is paired up with
mod_headers or mod_deflate.
The problem is caused by how mod_dav passes the output filter list to its
providers. A hook receives the current head of the output filter list for
a particular request. Certain filters, such as the one in mod_headers, are
designed to perform the work only once. When the work is done, a filter
removes itself from the list. If a filter is the first in the list, this
updates the head of the linked list in request_rec (r->output_filters), but
not the ap_filter_t * pointer that was passed as an argument to mod_dav_svn.
So, mod_dav_svn continues to operate with an outdated list of filters, and
every write causes a re-execution of the already removed filter.
In case with mod_headers, this triggers an unbounded memory usage, e.g.,
responding to a GET request for a 500 MB file results in the server
consuming a large amount (~3.2 GB) of memory. Full discussion of the
mod_dav's part of the issue can be found in [1].
The idea behind the fix is that we add a new opaque type, dav_svn__output,
a small corresponding set of private API functions, and adjust the existing
output helpers so that every write would target a dav_svn__output object.
A dav_svn__output is constructed for a particular request, and internally
it ensures that the writes *always* target the actual output filter list
in r->output_filters.
[1] https://mail-archives.apache.org/mod_mbox/httpd-dev/201608.mbox/%3C20160822151917.GA22369%40redhat.com%3E
* subversion/mod_dav_svn/dav_svn.h
(dav_svn__output): New.
(dav_svn__output_create, dav_svn__output_get_bucket_alloc,
dav_svn__output_pass_brigade): New.
(dav_svn__merge_response, dav_svn__update_report,
dav_svn__log_report, dav_svn__dated_rev_report,
dav_svn__get_locations_report, dav_svn__get_location_segments_report,
dav_svn__file_revs_report, dav_svn__replay_report,
dav_svn__get_mergeinfo_report, dav_svn__get_locks_report,
dav_svn__get_deleted_rev_report, dav_svn__get_inherited_props_report,
dav_svn__post_create_txn, dav_svn__post_create_txn_with_props,
dav_svn__brigade_write, dav_svn__brigade_puts,
dav_svn__brigade_printf, dav_svn__brigade_putstrs,
dav_svn__make_base64_output_stream, dav_svn__final_flush_or_error):
Now work with a dav_svn__output, instead of an ap_filter_t.
* subversion/mod_dav_svn/util.c
(dav_svn__output): New.
(dav_svn__output_create, dav_svn__output_get_bucket_alloc,
dav_svn__output_pass_brigade): Implement new functions.
(dav_svn__brigade_write, dav_svn__brigade_puts,
dav_svn__brigade_printf, dav_svn__brigade_putstrs,
dav_svn__final_flush_or_error): Target the actual output filter list
for a particular request.
(brigade_write_baton): Store a dav_svn__output object.
(brigade_write_fn): Target the actual output filter list for a
particular request.
(dav_svn__make_base64_output_stream): Now works with a dav_svn__output,
instead of an ap_filter_t.
* subversion/mod_dav_svn/repos.c
(diff_ctx_t): Store a dav_svn__output object.
(write_to_filter): Call dav_svn__brigade_write() function.
(close_filter): Use the dav_svn__output's API. The apr_brigade_cleanup()
call is no longer required, since dav_svn__output_pass_brigade() also
cleans the bucket brigade.
(emit_collection_head, emit_collection_entry, emit_collection_tail):
Now work with a dav_svn__output, instead of an ap_filter_t.
(deliver): Create a dav_svn__output at the beginning of this function.
Use the dav_svn__output's API for brigade creation and passing to the
output filter stack.
(handle_post_request): Now works with a dav_svn__output, instead of
an ap_filter_t.
(dav_svn__method_post): Create a dav_svn__output and forward it to
handle_post_request().
* subversion/mod_dav_svn/version.c
(deliver_report): Create a dav_svn__output and forward it to the
report handlers.
(merge): Create a dav_svn__output and forward it to
dav_svn__merge_response().
* subversion/mod_dav_svn/merge.c
(send_response, do_resources, dav_svn__merge_response): Now work with
a dav_svn__output and use the new private API.
* subversion/mod_dav_svn/posts/create_txn.c
(dav_svn__post_create_txn, dav_svn__post_create_txn_with_props): Now
work with a dav_svn__output.
* subversion/mod_dav_svn/reports/dated-rev.c
(dav_svn__dated_rev_report): Work with a dav_svn__output. Use the new
private API when creating a bucket brigade.
* subversion/mod_dav_svn/reports/deleted-rev.c
(dav_svn__get_deleted_rev_report): Work with a dav_svn__output. Use
the new private API when creating a bucket brigade.
* subversion/mod_dav_svn/reports/file-revs.c
(file_rev_baton): Store a dav_svn__output object.
(dav_svn__file_revs_report): Work with a dav_svn__output. Use the new
private API when creating a bucket brigade.
* subversion/mod_dav_svn/reports/get-location-segments.c
(location_segment_baton): Store a dav_svn__output object.
(dav_svn__get_location_segments_report): Work with a dav_svn__output.
Use the new private API when creating a bucket brigade.
* subversion/mod_dav_svn/reports/get-locations.c
(send_get_locations_report, dav_svn__get_locations_report): Work with a
dav_svn__output. Use the new private API when creating a bucket brigade.
* subversion/mod_dav_svn/reports/get-locks.c
(send_get_lock_response, dav_svn__get_locks_report): Work with a
dav_svn__output. Use the new private API when creating a bucket brigade.
* subversion/mod_dav_svn/reports/inherited-props.c
(dav_svn__get_inherited_props_report): Work with a dav_svn__output.
Use the new private API when creating a bucket brigade.
* subversion/mod_dav_svn/reports/log.c
(log_receiver_baton): Store a dav_svn__output object.
(log_revision_receiver): Rewrite the forced flush by creating and sending
a flush bucket to the output filters (that's what ap_fflush() does
internally). Drop the aborted connection check, since it's now a
part of dav_svn__output_pass_brigade().
(dav_svn__log_report): Work with a dav_svn__output. Use the new private
API when creating a bucket brigade.
* subversion/mod_dav_svn/reports/mergeinfo.c
(dav_svn__get_mergeinfo_report): Work with a dav_svn__output. Use the
new private API when creating a bucket brigade.
* subversion/mod_dav_svn/reports/replay.c
(edit_baton_t): Store a dav_svn__output object.
(make_editor, dav_svn__replay_report): Work with a dav_svn__output.
Use the new private API when creating a bucket brigade.
* subversion/mod_dav_svn/reports/update.c
(update_ctx_t): Store a dav_svn__output object.
(dav_svn__update_report): Work with a dav_svn__output. Use the new
private API when creating a bucket brigade.
|