/[Apache-SVN]
ViewVC logotype

Revision 1758224


Jump to revision: Previous Next
Author: kotkov
Date: Mon Aug 29 13:41:11 2016 UTC (7 years, 7 months ago)
Changed paths: 17
Log Message:
Fix the unbounded memory usage when mod_dav_svn is paired up with
mod_headers or mod_deflate.

The problem is caused by how mod_dav passes the output filter list to its
providers.  A hook receives the current head of the output filter list for
a particular request.  Certain filters, such as the one in mod_headers, are
designed to perform the work only once.  When the work is done, a filter
removes itself from the list.  If a filter is the first in the list, this
updates the head of the linked list in request_rec (r->output_filters), but
not the ap_filter_t * pointer that was passed as an argument to mod_dav_svn.
So, mod_dav_svn continues to operate with an outdated list of filters, and
every write causes a re-execution of the already removed filter.

In case with mod_headers, this triggers an unbounded memory usage, e.g.,
responding to a GET request for a 500 MB file results in the server
consuming a large amount (~3.2 GB) of memory.  Full discussion of the
mod_dav's part of the issue can be found in [1].

The idea behind the fix is that we add a new opaque type, dav_svn__output,
a small corresponding set of private API functions, and adjust the existing
output helpers so that every write would target a dav_svn__output object.
A dav_svn__output is constructed for a particular request, and internally
it ensures that the writes *always* target the actual output filter list
in r->output_filters.

[1] https://mail-archives.apache.org/mod_mbox/httpd-dev/201608.mbox/%3C20160822151917.GA22369%40redhat.com%3E

* subversion/mod_dav_svn/dav_svn.h
  (dav_svn__output): New.
  (dav_svn__output_create, dav_svn__output_get_bucket_alloc,
   dav_svn__output_pass_brigade): New.
  (dav_svn__merge_response, dav_svn__update_report,
   dav_svn__log_report, dav_svn__dated_rev_report,
   dav_svn__get_locations_report, dav_svn__get_location_segments_report,
   dav_svn__file_revs_report, dav_svn__replay_report,
   dav_svn__get_mergeinfo_report, dav_svn__get_locks_report,
   dav_svn__get_deleted_rev_report, dav_svn__get_inherited_props_report,
   dav_svn__post_create_txn, dav_svn__post_create_txn_with_props,
   dav_svn__brigade_write, dav_svn__brigade_puts,
   dav_svn__brigade_printf, dav_svn__brigade_putstrs,
   dav_svn__make_base64_output_stream, dav_svn__final_flush_or_error):
   Now work with a dav_svn__output, instead of an ap_filter_t.

* subversion/mod_dav_svn/util.c
  (dav_svn__output): New.
  (dav_svn__output_create, dav_svn__output_get_bucket_alloc,
   dav_svn__output_pass_brigade): Implement new functions.
  (dav_svn__brigade_write, dav_svn__brigade_puts,
   dav_svn__brigade_printf, dav_svn__brigade_putstrs,
   dav_svn__final_flush_or_error): Target the actual output filter list
   for a particular request.
  (brigade_write_baton): Store a dav_svn__output object.
  (brigade_write_fn): Target the actual output filter list for a
   particular request.
  (dav_svn__make_base64_output_stream): Now works with a dav_svn__output,
   instead of an ap_filter_t.

* subversion/mod_dav_svn/repos.c
  (diff_ctx_t): Store a dav_svn__output object.
  (write_to_filter): Call dav_svn__brigade_write() function.
  (close_filter): Use the dav_svn__output's API.  The apr_brigade_cleanup()
   call is no longer required, since dav_svn__output_pass_brigade() also
   cleans the bucket brigade.
  (emit_collection_head, emit_collection_entry, emit_collection_tail):
   Now work with a dav_svn__output, instead of an ap_filter_t.
  (deliver): Create a dav_svn__output at the beginning of this function.
   Use the dav_svn__output's API for brigade creation and passing to the
   output filter stack.
  (handle_post_request): Now works with a dav_svn__output, instead of
   an ap_filter_t.
  (dav_svn__method_post): Create a dav_svn__output and forward it to
   handle_post_request().

* subversion/mod_dav_svn/version.c
  (deliver_report): Create a dav_svn__output and forward it to the
   report handlers.
  (merge): Create a dav_svn__output and forward it to
   dav_svn__merge_response().

* subversion/mod_dav_svn/merge.c
  (send_response, do_resources, dav_svn__merge_response): Now work with
   a dav_svn__output and use the new private API.

* subversion/mod_dav_svn/posts/create_txn.c
  (dav_svn__post_create_txn, dav_svn__post_create_txn_with_props): Now
   work with a dav_svn__output.

* subversion/mod_dav_svn/reports/dated-rev.c
  (dav_svn__dated_rev_report): Work with a dav_svn__output.  Use the new
   private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/deleted-rev.c
  (dav_svn__get_deleted_rev_report): Work with a dav_svn__output.  Use
   the new private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/file-revs.c
  (file_rev_baton): Store a dav_svn__output object.
  (dav_svn__file_revs_report): Work with a dav_svn__output.  Use the new
   private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/get-location-segments.c
  (location_segment_baton): Store a dav_svn__output object.
  (dav_svn__get_location_segments_report): Work with a dav_svn__output.
   Use the new private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/get-locations.c
  (send_get_locations_report, dav_svn__get_locations_report): Work with a
   dav_svn__output.  Use the new private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/get-locks.c
  (send_get_lock_response, dav_svn__get_locks_report): Work with a
   dav_svn__output. Use the new private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/inherited-props.c
  (dav_svn__get_inherited_props_report): Work with a dav_svn__output.
   Use the new private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/log.c
  (log_receiver_baton): Store a dav_svn__output object.
  (log_revision_receiver): Rewrite the forced flush by creating and sending
   a flush bucket to the output filters (that's what ap_fflush() does
   internally).  Drop the aborted connection check, since it's now a
   part of dav_svn__output_pass_brigade().
  (dav_svn__log_report): Work with a dav_svn__output.  Use the new private
   API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/mergeinfo.c
  (dav_svn__get_mergeinfo_report): Work with a dav_svn__output.  Use the
   new private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/replay.c
  (edit_baton_t): Store a dav_svn__output object.
  (make_editor, dav_svn__replay_report): Work with a dav_svn__output.
   Use the new private API when creating a bucket brigade.

* subversion/mod_dav_svn/reports/update.c
  (update_ctx_t): Store a dav_svn__output object.
  (dav_svn__update_report): Work with a dav_svn__output.  Use the new
   private API when creating a bucket brigade.


Changed paths

Path Details
Directorysubversion/trunk/subversion/mod_dav_svn/dav_svn.h modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/merge.c modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/posts/create_txn.c modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/reports/dated-rev.c modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/reports/deleted-rev.c modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/reports/file-revs.c modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/reports/get-location-segments.c modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/reports/get-locations.c modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/reports/get-locks.c modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/reports/inherited-props.c modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/reports/log.c modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/reports/mergeinfo.c modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/reports/replay.c modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/reports/update.c modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/repos.c modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/util.c modified , text changed
Directorysubversion/trunk/subversion/mod_dav_svn/version.c modified , text changed

infrastructure at apache.org
ViewVC Help
Powered by ViewVC 1.1.26