/[Apache-SVN]
ViewVC logotype

Revision 1866950


Jump to revision: Previous Next
Author: kotkov
Date: Sat Sep 14 16:45:47 2019 UTC (4 years, 7 months ago)
Changed paths: 3
Log Message:
Fix an issue with the readline implementation for file streams that could
cause excessive memory usage for inputs containing one or multiple \0 bytes.

This is the likely cause of the OOM reported in
  https://lists.apache.org/thread.html/c96eb5618ac0bf6e083345e0fdcdcf834e30913f26eabe6ada7bab62@%3Cusers.subversion.apache.org%3E
(I think that the problem itself is a regression introduced by me in 1.10.)

Note: one thing I noticed while preparing the fix is that our `readline_fn`
functions for different streams have inconsistent behavior if the input data
contains \0 bytes.  More specifically, they may return different `line` values,
that may either be truncated at \0 or actually contain the whole data between
EOLs, including \0 bytes.  For now, this patch only fixes the excessive memory
usage problem, and I noted this related problem in the test and left it for
future work.

* subversion/libsvn_subr/stream.c
  (readline_apr_lf, readline_apr_generic): Reallocate the buffer based on its
   current size, instead of calculating the new size based on the already
   prealloc'd size.  There are no actual benefits in reallocating based on
   `blocksize`, and in the described case with \0 bytes doing so also backfires
   and may cause excessive allocations due to the actual size of the string
   being less than we expect it to.  A degenerate case of the erroneous
   behavior is ...

* subversion/tests/libsvn_subr/stream-test.c
  (test_stream_readline_file_nul): ...exploited in this new test.
  (test_funcs): Run new test.

* subversion/tests/libsvn_subr
  (): Adjust svn:ignore.


Changed paths

Path Details
Directorysubversion/trunk/subversion/libsvn_subr/stream.c modified , text changed
Directorysubversion/trunk/subversion/tests/libsvn_subr/ modified , props changed
Directorysubversion/trunk/subversion/tests/libsvn_subr/stream-test.c modified , text changed

infrastructure at apache.org
ViewVC Help
Powered by ViewVC 1.1.26