Revision 1546928

On the fsfs-improvements branch:  Merge all changes from the 
log-addressing branch into this one.  There were no conflicts.

This makes fsfs-improvements the canonical branch for the FSFS
format 7 changes and the former sub-branch may be deleted.

For easier review, the following is a summary of the changes.
Please refer to either the in-source comments or the original
commit messages for details.

BRANCH-README contains a feature overview.  The structure and
structure-indexes files contain a detailed description of the
new addressing concepts and their external representation.

* BRANCH-README
  (): Replace the old description with that for FSFS f7.

* subversion/libsvn_fs_fs/structure
  (): Add format 7 to format list and add index files as new
      elements.  Describe the concept of addressing modes,
      "item index" and "item type" in detail.  Explain the
      intention behind the new pack code.

* subversion/libsvn_fs_fs/structure-indexes
  (): New file.
      Detailed description of the two-way mapping between
      logical and physical addresses.  Also describes the
      binary format used to store the index files on disk.

* subversion/libsvn_fs_fs/index.*
  (): New files.
      Add a FSFS-private API for both indexes providing the
      mapping functionality as defined in structure-indexes.

* subversion/libsvn_fs_fs/rev_file.*
  (): New files.
      Provide a FSFS-private API that combines the 3 physical
      files (rev/pack, 2 x index) into a common data structure
      and ensures that they are being accessed consistently wrt.
      pack state.  This is necessary when there is an svnadmin
      pack running in the background while we read the repo.

* subversion/libsvn_fs_fs/id.*
  (): Replace OFFSET with ITEM_INDEX.
  (svn_fs_fs__id_create_root): New utility function enabled by
                               all root nodes having the same
                               item index.

* subversion/libsvn_fs_fs/util.*
  (): Add new path constructors for index files.  Move item
      to offset mapping and rev / pack file open functions
      to their respective new API files.

* subversion/libsvn_fs_fs/low_level.*
  (): Replace OFFSET with ITEM_INDEX.

* subversion/libsvn_fs_fs/fs.h
  (PATH_*): Path constants for the new index files.
  (CONFIG_*): New fsfs.conf configuration options.
  (SVN_FS_FS__FORMAT_NUMBER): Bump format number.
  (*_t): Replace OFFSET with ITEM_INDEX.
  (fs_fs_data_t): Add config settings and caches for the various
                  index elements.

* subversion/libsvn_fs_fs/caching.c
  (svn_fs_fs__initialize_caches): Initialize new caches.

* subversion/libsvn_fs_fs/fs.c
  (initialize_fs_struct): Default to physical addressing.

* subversion/libsvn_fs_fs/fs_fs.c
  (): Handle new addressing format info and config file options.
      Use a different revision 0 template for format 7.

* subversion/libsvn_fs_fs/cached_data.c
  (): Switch file access function to the new rev_file API.
      Follow the OFFSET to ITEM_INDEX rename.
      Implement the "block read" feature and make all data read
      functionality use the block_read() function.
  (dbg_log_access): Produce a more detailed access log in logical
                    addressing mode.

* subversion/libsvn_fs_fs/transaction.c
  (): Provide generic item_index allocator and index writer
      functions that work in both addressing modes.  Also, add
      wrappers streams that calculate the new low-level FVN-1a
      checksums for everything we write to disk.  Invoke all
      of this when writing data to the proto-rev file.  Finally,
      predict the addressing mode of the future revision and
      upgrade transactions from physical to log addressing mode
      upon misprediction or for old txns.

* subversion/libsvn_fs_fs/pack.h
  (svn_fs_fs__order_dir_entries): In mixed addressing mode repos,
                                  the optimal read order depends
                                  on the revision number.

* subversion/libsvn_fs_fs/pack.c
  (): Implement the new reorder-on-pack feature that is a core
      element of f7.  Details of it are described at the top of
      this file.
  (svn_fs_fs__order_dir_entries): Select the optimal order depending
                                  on the addressing mode.

* subversion/libsvn_fs_fs/tree.c
  (fs_dir_optimal_order): Adapt to pack API change.

* subversion/libsvn_fs_fs/verify.c
  (): In log. addressing mode, verify the forward and backward
      indexes are consistent with one another.  Verify that rev /
      pack file size match the index values and that the low-level
      checksums for every item in the rev / pack file match the
      index data as well.

* subversion/libsvn_fs_fs/hotcopy.c
  (): Copy the new index files as well, if the respective revs
      use logical addressing.  Also, clean them up when a shard
      got packed since the last incremental hotcopy.

* subversion/libsvn_fs_fs/recovery.c
  (): Adapt to new rev_file API and ID API changes.

* subversion/libsvn_fs_fs/rep-cache.c
  (): Follow the OFFSET to ITEM_INDEX rename.

* subversion/tests/cmdline/svnadmin_tests.py
  (): Add very limited f7 index read functionality - just enough
      to manipulate the changed paths lists in f7 as well.  Update
      expected output patterns for f7.  Restrict verify_non_utf8_paths
      to pre-f7 repos as manipulating f7 repos is harder and not
      relevant for the code / test coverage here.

* subversion/tests/cmdline/svntest/main.py
  (is_fs_log_addressing): FSFS f7 also uses log addressing.

* subversion/tests/libsvn_fs_fs
  (): svn:ignore test repositories

* subversion/tests/libsvn_fs_fs/fs-fs-pack-test.c
  (): Write correct addressing mode info and check for the right
      files in packed f7 repos.  Add tests TXN format upgrade tests.

* tools/server-side/svn-rep-sharing-stats.c
  (record): Follow the OFFSET to ITEM_INDEX rename.