Client
******

The Subversion client is built on three libraries.  One operates strictly on the working copy and does not talk to the repository. Another talks to the repository but never changes the working copy.  The last library uses the first two to provide operations such as commit and update, that both talk to the repository and change the working copy.

The initial client is a Unix-style command-line tool (like standard CVS), but it should be easy to write a GUI client as well, based on the same libraries.  The libraries capture the core Subversion functionality, segregating it from user interface concerns.

This chapter describes the libraries, and the physical layout of working copies.

Deltas
======

A delta is a data structure that compactly describes the differences between two arbitrary directory trees.  Deltas can also carry ancestry information, indicating how the files in one tree are related to files in the other tree, if this is available.  Finally, deltas can describe changes to file meta-information, like permission bits, creation dates, and so on.  Both the working copy and repository access libraries operate on deltas.

Deltas use three distinct structures:

  * text deltas, which describe changes to a string of bytes, like the contents of a file,

  * property deltas, which describe changes to a list of named properties, perhaps using text deltas, and

  * tree deltas proper, which describe changes to directory trees.

The term delta without qualification generally means a tree delta, unless some other meaning is clear from context.

Text Deltas
-----------

A text delta describes the difference between two strings of bytes, the source string and the target string.  Given a source string and a target string, we can compute a text delta; given a source string and a delta, we can reconstruct the target string.  However, note that deltas are not invertible: you cannot always reconstruct the source string given the target string and delta.

The standard Unix ``diff'' format is one possible representation for text deltas; however, diffs are not ideal for internal use by a version control system, for several reasons:

  * Diffs are line-oriented, which makes them human-readable, but sometimes makes them perform poorly on binary files.

  * Diffs represent a series of replacements, exchanging selected ranges of the old text with new text; again, this is easy for humans to read, but it is more expensive to compute and less compact than some alternatives.

Instead, Subversion uses the VDelta format, as described in Hunt, J. J., Vo, K.-P., and Tichy, W. F.  An empirical study of delta algorithms.  Lecture Notes in Computer Science 1167 (July 1996), 49-66. David Korn and Kiem-Phong Vo have submitted an Internet Draft, titled ``The VCDIFF Generic Differencing and Compression Data Format,'' describing an improvement to VDelta which we may adopt.

Property Deltas
---------------

A property delta describes changes to a property list, of the sort associated with files, directories, and directory entries, and version numbers.  A property delta can record creating, deleting, and changing the text of any number of properties.

A property delta is an unordered set of name/change pairs.  No two pairs within a given property delta have the same name.  A pair's name indicates the property affected, and the change indicates what happens to its value.  There are three kinds of changes:

set VALUE
     Change the value of the named property to the byte string VALUE. If there is no property with the given name, one is added to the property list.

delete
     Remove the named property from the property list.

At the moment, the set command can either create or change a property value.  However, this simplification means that the server cannot distinguish between a client which believes it is creating a value afresh, and a client which believes it is changing the value of an existing property.  It may simplify conflict detection to divide set into two separate add and change operations.

In the future, we may add a vdelta change, which specifies a change to an existing property's value as a VDelta-format text delta. This would give us a compact way to describe small changes to large property values.

The concrete form of a property delta, used for transmitting such deltas over the net, is the concatenation of the following items:

  * a non-negative ASCII decimal number S, followed by an ASCII newline character, and

  * a series of name/change pairs, S bytes long in total.

Each name/change pair is represented by the concatenation of the following items:

  * a non-negative ASCII decimal number N, followed by a space,

  * the property name, as a Unicode UTF-8 string N bytes long, canonically decomposed and ordered, followed by a newline, and

  * a set or delete item, followed by a newline.

A set item has the following form:

  * the word set, encoded in lower-case ASCII characters, followed by a space,

  * a non-negative ASCII decimal number P, followed by a space, and

  * the property value, as a byte string P bytes long.

A delete item has the following form:

  * the word delete, encoded in lower-case ASCII characters.

Tree Deltas
-----------

A tree delta describes changes between two directory trees, the source tree and the target tree.  Tree deltas can describe copies, renames, and deletions of files and directories, changes to file contents, and changes to property lists.  A tree delta can also carry information about how the files in the target tree are derived from the files in the source tree, if this information is available.

The format for tree deltas described here is easy to compute from a Subversion working directory, and easy to apply to a Subversion repository.  Furthermore, the size of a tree delta in this format is independent of the commands used to produce the target tree --- it depends only on the degree of difference between the source and target trees.

A tree delta is interpreted in the context of three parameters:

  * SOURCE-ROOT, the name of the directory to which this complete tree delta applies,

  * SOURCE-DIR, which is a directory in the source tree that we are currently modifying to yield ...

  * ... target-dir --- the directory we're constructing.

When we start interpreting a tree delta, all three parameters are equal. As we walk the tree delta, TARGET-DIR walks the tree we are constructing, and SOURCE-DIR walks the corresponding portion of the source tree, which we use as the original.  SOURCE-ROOT remains constant as we walk the delta; we may use it to choose new source trees.

A tree delta is a list of changes, (tdelta CHANGE ...), describing how to edit the contents of SOURCE-DIR to yield TARGET-DIR.  There are three kinds of changes:

(delete "NAME")
     SOURCE-DIR has an entry named NAME, which is not present in TARGET-DIR.

(new "NAME" CONTENT)
     TARGET-DIR has an entry named NAME, which is not present in SOURCE-DIR; CONTENT describes the file or directory to which the new directory entry refers.

(replace "NAME" CONTENT)
     Both SOURCE-DIR and TARGET-DIR have an entry named NAME, which has changed; CONTENT describes the new file or directory.

Any entries in SOURCE-DIR whose names aren't mentioned are assumed to appear unchanged in TARGET-DIR.  Thus, an empty change list indicates that a directory is unchanged.

In the change descriptions above, each CONTENT takes one of the following forms:

file ANCESTOR TEXT-DELTA
     The given TARGET-DIR entry refers to a file, F. ANCESTOR indicates which directory in the source tree F is derived from, if any, and TEXT-DELTA describes how to construct F from that ancestor.

directory ANCESTOR TREE-DELTA
     The given TARGET-DIR entry refers to a subdirectory, SUB. ANCESTOR indicates which directory in the source tree SUB is derived from, if any.

     The tree delta TREE-DELTA describes how to construct SUB from that ancestor.  It should be interpreted with the SOURCE-DIR given by ANCESTOR, and a new TARGET-DIR of TARGET-DIR/NAME.

     Since TREE-DELTA is itself a complete tree delta structure, tree deltas are trees.  The structure of a tree delta is a subgraph of the target tree.

In the CONTENT descriptions above, each ANCESTOR takes one of the following forms:

"PATH"
     The ancestor of the new or changed file or directory is SOURCE-ROOT/PATH.  When followed by a TEXT-DELTA, that text delta should be applied to SOURCE-ROOT/PATH.  When followed by a TREE-DELTA, SOURCE-ROOT/PATH should be the new SOURCE-DIR for interpreting that tree delta.

same
     This is a shortcut for a PATH of SOURCE-DIR/NAME. This makes the common case --- files or directories modified in place --- more compact.

none
     This indicates that the file or directory has no ancestor in the source tree.  When followed by a TEXT-DELTA, that delta should be applied to the empty file to yield the new text.  When followed by a TREE-DELTA, that delta should be evaluated as if SOURCE-DIR were an imaginary empty directory.

Suppose we have the following source tree:

     /dir1/file1
           file2
           dir2/file3
                file4
           dir3/file5
                file6

If we edit the contents of /dir1/file1, we can describe the effect on the tree with the following tree delta, to be applied to the root:

     (tdelta
       (replace "dir1" directory same
         (tdelta
           (replace "file1" file same TEXT-DELTA))))

The outer tdelta form describes the changes made to the root directory.  Within the root directory, there are changes in dir1, described by the nested tdelta.  Within /dir1, there are changes in file1, described by the TEXT-DELTA.

If we had edited both /dir1/file1 and /dir1/file2, then there would simply be two entries in the inner tdelta.

As another example, starting from the same source tree, suppose we rename /dir1/file1 to /dir1/file8:

     (tdelta
       (replace "dir1" directory same
         (tdelta
           (delete "file1")
           (new "file8" file "/dir1/file1" NULL-DELTA))))

As above, the inner tdelta describes how /dir1 has changed: the entry for /dir1/file1 has disappeared, but there is a new entry, /dir1/file8, which is derived from and textually identical to /dir1/file1 in the source directory.  This is just an indirect way of describing the rename.

Why is it necessary to be so indirect?  Consider the delta representing the result of:

 1. renaming /dir1/file1 to /dir1/tmp,

 2. renaming /dir1/file2 to /dir1/file1, and

 3. renaming /dir1/tmp to /dir1/file2

(in other words, exchanging file1 and file2):

     (tdelta
       (replace "dir1" directory same
         (tdelta
           (replace "file1" file "/dir1/file2" NULL-DELTA)
           (replace "file2" file "/dir1/file1" NULL-DELTA))))

The indirectness allows the tree delta to capture an arbitrary rearrangement without resorting to temporary filenames.

Another example, starting from the same source tree:

 1. rename /dir1/dir2 to /dir1/dir4,

 2. rename /dir1/dir3 to /dir1/dir2, and

 3. move file3 from /DIR1/DIR4 to /DIR1/DIR2.

Note that file3's path has remained the same, even though the directories around it have changed.  Here is the tree delta:

     (tdelta
       (replace "dir1" directory same
         (tdelta
           (replace "dir2" directory "/dir1/dir3"
             (tdelta
               (new "file3" file "/dir1/dir2/file3" NULL-DELTA)))
           (delete "dir3")
           (new "dir4" directory "/dir1/dir2"
             (tdelta
               (delete "file3"))))))

In other words:

  * /dir1 has changed;

  * the new directory /dir1/dir2 is derived from the old /dir1/dir3, and contains a new entry file3, derived from the old /dir1/dir2/file3;

  * there is no longer any /dir1/dir3; and

  * the new directory /dir1/dir4 is derived from the old /dir1/dir2, except that its entry for file3 is now gone.

Some more possible maneuvers, left as exercises for the reader:

  * Delete dir2, and then create a file named dir2.

  * Rename /dir1/dir2 to /dir1/dir4; move file2 into /dir1/dir4; and move file3 into /DIR1/DIR3.

  * Move dir2 into dir3, and move dir3 into /.

Skeltas
-------

It is sometimes useful to represent a set of changes to a tree, just as a tree delta does, without providing any text deltas.  Text deltas are often large and expensive to compute, and tree deltas can be useful without them.  For example, one can detect whether two changes conflict --- whether they change the same file, for example --- without knowing exactly how the conflicting files changed.

A Skelta is a simply tree delta in which all text deltas are omitted --- it is a ``skeleton delta''.  The svn commit command uses skeltas to provide early notice of conflicts: first, the client transmits a skelta to the server, which can check for textual conflicts and reject the user's commit, before the client takes the time to transmit the (possibly large) textual changes.

Working copies and the working copy library
===========================================

Working copies are client-side directory trees containing both versioned data and Subversion administrative files.  The functions in the working copy management library are the only functions in Subversion which operate on these trees.

The layout of working copies
----------------------------

This section gives an overview of how working copies are arranged physically, but is not a full specification of working copy layout.

As with CVS, Subversion working copies are simply directory trees with special administrative subdirectories, in this case named "SVN" instead of "CVS":

                                  myproj
                                  / | \
                    _____________/  |  \______________
                   /                |                 \
                 SVN               src                doc
             ___/ | \___           /|\             ___/ \___
            |     |     |         / | \           |         |
           base  ...   ...       /  |  \     myproj.texi   SVN
                                /   |   \              ___/ | \___
                           ____/    |    \____        |     |     |
                          |         |         |      base  ...   ...
                         SVN      foo.c     bar.c     |
                     ___/ | \___                      |
                    |     |     |                     |
                  base   ...   ...               myproj.texi
               ___/ \___
              |         |
            foo.c     bar.c

Each dir/SVN/ directory records the files in dir, their version numbers and property lists, pristine versions of all the files (for client-side delta generation), the repository from which dir came, and any local changes (such as uncommitted adds, deletes, and renames) that affect dir.

Although often it would often be possible to deduce certain information (such as the origin repository) by examining parent directories, this is avoided in favor of making each directory be as much a self-contained unit as possible.

For example, immediately after a checkout the administrative information for the entire working tree could be stored in one top-level file.  But subdirectories instead keep track of their own version information.  This would be necessary anyway once the user starts committing new versions for particular files, and it also makes it easier for the user to prune a big, complete tree into a small subtree and still have a valid working copy.

The SVN subdir contains:

  * A base directory, containing the pristine repository versions of the files in the corresponding working directory.

  * A versions file, the first entry of which records the version number of this directory (and, by implication, the version number of any files not otherwise mentioned).  The remaining entries are for files at other versions.

     It may help to think of this file as the functional equivalent of the CVS/Entries file.

  * A properties file, recording properties for this directory and all directory entries and files it contains.  Although this information could be stored in the versions file, it is sufficiently separable to warrant its own file; very often one wants to look up a version without looking up a property, and vice-versa.

  * A changes file, recording uncommitted changes to and from this directory (adds, removes, renames).

  * A lock file, whose presence implies that some client is currently operating on the adminstrative area.

  * A tmp directory.

The formats of these files are not specified yet.

The working copy management library
-----------------------------------

  * Requires:

       * a working copy

  * Provides:

       * ability to manipulate the working copy's versioned data

       * ability to manipulate the working copy's administrative files

This library performs "offline" operations on the working copy.

Note that the interface as given here is incomplete.  This section concentrates on functions meant to be used in conjunction with the repository access library, such as routines to produce deltas and make committable changes.  But functions used most often within the working copy library itself, and only rarely outside it, are left unspecified here.  For example, there is a ver_t wc_base_version (path) function, and a ver_t wc_lock (path) function, but they are not part of the conceptual interface this section describes.

In the function names and descriptions below, the term node refers to any kind of object in a Subversion tree.  At the moment, Subversion trees can only contain files and directories, but we will eventually add symlinks and other interesting creatures; these are all nodes.

skelta_t wc_make_skelta (pathlist)
     Returns a skelta -- a delta object describing the changes to paths but not including the actual content of the changes (i.e., the vdeltas).  In other words, this is how you get a list of all local modifications.

     A pathlist contains directories or files; directories may be entered recursively.  Multiple paths must be support because of scenarios like the following:

             $ ls
             src/  foo/  other/  bar/  baz/  README  Makefile
             $ svn commit foo/ bar/ baz/qux.c

     The commit is atomic, and covers some but not all of the subdirectories available.  Therefore, it must be possible to request a delta for any subset of the working tree.  Although some files and directories outside that subset might also have been modified, the delta will not include those changes.

delta_t wc_fill_skelta (skelta)

     Turns skelta into a delta by generating the appropriate vdiffs and pdiffs, and placing them into the skelta.

delta_t wc_make_delta (pathlist)
     Equivalent to

             wc_fill_skelta (make_skelta (pathlist))

bool wc_apply_delta (delta)

     Applies delta to the working copy.

bool wc_add (path)

     Add the file path (i.e., tweak the administrative files appropriately).

bool wc_delete (path)

     Remove this file.

bool wc_rename (old_path, new_path)

     Move/rename this file (within the working copy, of course).

bool wc_copy (src_path, dest_path)

     Copy this file (within the working copy).

str_t wc_get_node_prop (path, propname)

     Return local value of propname for the file or directory path.

str_t wc_get_dirent_prop (path, propname)

     Return local value of propname for the directory entry path.

proplist_t wc_get_node_proplist (path)

     Return all properties (names and values) of file or directory path, in a hash table.

proplist_t wc_get_dirent_proplist (path)

     Return all properties (names and values) of directory entry path, in a hash table.

proplist_t wc_get_node_propnames (path)

     Return all property names for file or directory path.

proplist_t wc_get_dirent_propnames (path)

     Return all property names for a directory entry.

The repository access library
=============================

  * Requires:

       * network access to a Subversion server

  * Provides:

       * the ability to interact with a repository

This library performs operations involving communication with the repository.

Reading History
---------------

ver_t ra_latest (repos, user)
     Report the latest version number for repos.

prop_t ra_get_ver_prop (repos, user, ver, propname)
     Return the value of propname on ver.

proplist_t ra_get_ver_proplist (repos, user, ver)
     Return all properties (names and values) of ver.

proplist_t ra_get_ver_propnames (repos, user, ver)
     Return all property keys for a ver.

Reading Nodes
-------------

node_t ra_read (repos, user, ver, path)
     Return a node from the repository. If path is a file, returns full contents of the file. If path is a directory, returns a list of dir_entries.

str_t ra_get_node_prop (repos, user, ver, path, propname)
     Return value of propname for path in repos.

str_t ra_get_dirent_prop (repos, user, ver, path, propname)
     Return value of propname for the directory entry path in repos.

proplist_t ra_get_node_proplist (repos, ver, path)
     Return all properties (names and values) of path in repos.

proplist_t ra_get_dirent_proplist (repos, user, ver, path)
     Return all properties (names and values) of directory entry path in repos.

proplist_t ra_get_node_propnames (repos, user, ver, path)
     Return all property names for path in repos.

proplist_t ra_get_dirent_propnames (repos, user, ver, path)
     Return all property names for directory entry path in repos

Difference Queries
------------------

skelta_t ra_get_status (repos, user, skelta)
     Return status (up-to-date or not) for the files mentioned in skelta.

delta_t ra_get_update (repos, user, skelta)
     Return changes for files mentioned in skelta.  Like ra_get_status, but returns filled delta instead of a skelta.

delta_t ra_get_delta (repos, user, ver1, path1, ver2, path2)
     Return delta between two trees in repos.

diff_t ra_get_diff (repos, user, ver1, path1, ver2, path2)
     Return a text diff between versions in repos. (This must be done server side, the client doesn't have enough information to make the diff.)

Writing
-------

token_t ra_submit (repos, user, skelta)
     Submit skelta for approval (see Locking).

ver_t ra_write (repos, user, delta, token)
     Write previously-approved delta into the repository.

     Returns the repos's new version number.

     This delta and token must correspond to the skelta and token of a previous submit() call.

bool ra_abandon (repos, user, token)
     Abandon the commit identified by token.  For example, a user might interrupt the client between calls to submit() and write(), and the server would want to know the commit has been abandoned so it can clear out its pool of approved changes.

The client operation library
============================

  * Requires:

       * the working copy management library

       * the repository access library

  * Provides:

       * all client-side Subversion commands

These functions correspond to user-level client commands.

They are listed here to show how it is possible to implement them using the libraries given above, not to explain what they do.  Their behavior corresponds to the CVS operations of the same names.

bool add (path)
     Passes path to wc_add().

bool checkout (repos, ver, path)
     todo: discuss serialized delta vs lazy reading

bool commit (pathlist, log_msg)
     Make a skelta with wc_make_skelta(), ra_submit() it to the repository for approval.  If an approval token is forthcoming, use wc_fill_skelta() to produce a delta, along with the log_msg as a property change on the version, and send it and the token back to the repository via ra_write().  Else if no approval, then abandon() the commit.

bool diff (pathlist)
     A Subversion diff, like a CVS diff, can involve information available only to the server, or client, or both:

      1. diff locally modified file against its base version
          Just run diff locally.

      2. diff local file against a repository-only version
          Call ra_get_delta(), then wc_apply_delta() to the pristine base version to get the new file (result probably stored in SVN/tmp/, of course), then run diff on the local file and the new file.

      3. diff two repository-only versions
          Call ra_get_diff().

bool import (pathlist)
     A special case of add(), which see.

bool log (ver)
     Call ra_get_ver_prop() with the appropriate property name.

bool rename (old_path, new_path)
     Pass through to wc_rename().

bool remove (pathlist)
     Pass through to wc_delete().

bool branch (branchname, pathlist)
     Uses wc_copy() (see Model).

bool tag (tagname, pathlist)
     Same as branch(), and may also set properties marking this as a read-only clone, so it behaves like a CVS tag.

bool status (pathlist)
     Call ra_get_status().

bool update (pathlist)
     Call ra_get_update(), passing result to wc_apply_delta().  (There may be streamy variants of these so the application of a delta doesn't need to wait for the entire delta to have arrived.)