Add more documentation.
Change-Id: I82159ee4741ed3a2997f09aeb520ee1efe239319
diff --git a/README.md b/README.md
index 2aefb36..7ad8800 100644
--- a/README.md
+++ b/README.md
@@ -1,6 +1,7 @@
-This is a FUSE filesystem that provides light-weight, lazily downloaded,
-read-only checkouts of Git repositories. It is intended for use with Android.
+SlothFS is a FUSE filesystem that provides light-weight, lazily downloaded,
+read-only checkouts of manifest-based Git projects. It is intended for use with
+Android.
How to use
@@ -10,7 +11,7 @@
go install github.com/google/slothfs/cmd/slothfs-multifs
mkdir /tmp/mnt
- slothfs-multifs -gitiles https://android.googlesource.com/ /tmp/mnt &
+ slothfs-multifs -gitiles https://android.googlesource.com/ /tmp/mnt &
To create a workspace "ws" corresponding to the latest manifest version
@@ -19,47 +20,7 @@
> /tmp/m.xml &&
ln -s /tmp/m.xml /tmp/mnt/config/ws
-To populate a checkout
-
- go install github.com/google/slothfs/cmd/slothfs-populate
- mkdir -p checkout/frameworks
- cd checkout/frameworks
- git clone https://android.googlesource.com/platform/frameworks/base
- cd ../
- slothfs-populate -ro /tmp/mnt/ws .
-
-The filesystem daemon uses an on-disk cache, which by default is stored under
-~/.cache/slothfs
-
-
-Configuring
-===========
-
-The FUSE file system clones repositories on-demand. You can avoid cloning
-altogether for repositories you know you don't need. This is configured through
-a JSON file.
-
-For example, if you work on Android, and build on a Linux machine, you will
-never need the Darwin related prebuilts. You can avoid a costly clone for those
-by doing:
-
- {"Repo": ".*darwin.*", "Clone": false}
-
-Similarly, the build system system will read files (typically called '*.mk')
-across the entire tree. When any .mk file is opened, this should not trigger a
-clone. This is achieved with the following entry
-
- {"File": ".*mk$", "Clone": false}
-
-Together, the following `config.json` file is a good start for working on
-android:
-
- [{"Repo": ".*darwin.*", "Clone": false},
- {"File": ".*mk$", "Clone": false}]
-
-A more elaborate configuration file is included as `android.json`.
-
-By default, slothfs loads the configuration from =$HOME/.config/slothfs/clone.json=.
+More details can be found in the [manual](docs/manual.md).
DISCLAIMER
diff --git a/design.md b/docs/design.md
similarity index 100%
rename from design.md
rename to docs/design.md
diff --git a/docs/manual.md b/docs/manual.md
new file mode 100644
index 0000000..8ec7e2c
--- /dev/null
+++ b/docs/manual.md
@@ -0,0 +1,202 @@
+
+Building Android with SlothFS
+=============================
+
+[Slothfs](https://github.com/google/slothfs) is a FUSE file system that offers a
+read-only view of a Git tree. It downloads files lazily, so it takes up less
+diskspace than a full checkout based on `repo`. Once its caches are seeded,
+creating fresh workspaces or syncing should take on the order of seconds.
+
+It is developed as open source, and can be used by anyone that uses
+Gerrit/Gitiles based Git hosting. The [design doc](design.md) explains the
+background behind its design choices.
+
+It has been tested on Linux, but we expect it works on OSX too.
+
+For the remainder of this document, we will assume that you are trying to
+compile some flavor of Android.
+
+
+Installing
+==========
+
+Run `all.bash` to compile, test and install all components. The rest of this
+document assumes this has been done, and `$GOPATH/bin/` is in your `$PATH`.
+
+In addition, install the standard Android `clone.json` to avoid unnecessary git
+clones
+
+ mkdir -p $HOME/.config/slothfs
+ curl https://raw.githubusercontent.com/google/slothfs/master/android.json \
+ > $HOME/.config/slothfs/clone.json
+
+
+Selecting the host
+==================
+
+The defaults of slothfs are for the public version of Android at
+`android.googlesource.com`. Set the `-gitiles_url` option to change against
+which version of Android you want to run.
+
+
+Dereferencing a manifest
+========================
+
+The manifest describes which repositories go into an Android checkout. To make a
+file system out of this, we must decide at which exact revision each repository
+should be offered.
+
+This can be done with `slothfs-expand-manifest`, eg.
+
+ slothfs-expand-manifest > /tmp/m.xml
+
+
+Mounting the filesystem
+=======================
+
+First, create a mountpoint:
+
+ sudo mkdir -p /slothfs && sudo chown $USER /slothfs
+
+Then, to mount the file system, run
+
+ slothfs-multifs /slothfs
+
+
+Configuring a workspace
+=======================
+
+A workspace can be created by a symlinking a deferenced manifest field into the
+`config` directory, i.e.
+
+ ln -s /tmp/m.xml /slothfs/config/my-workspace
+
+This should create a directory `/slothfs/my-workspace` holding the tree
+described in `/tmp/m.xml`.
+
+
+Using a workspace
+=================
+
+To use the read-only tree, create a writable checkout. Let's assume we want to
+change the `art` repo:
+
+ mkdir -p checkout ; cd checkout
+ git clone https://android.googlesource.com/platform/art
+
+To make `checkout` into a full-fledged checkout, run
+
+ slothfs-populate -ro /slothfs/my-workspace .
+
+This populate the `checkout` directory with symlinks into `/slothfs`, yielding a
+full check out.
+
+
+Syncing
+=======
+
+Advancing your checkout to a different timestamp uses the same commands. To sync
+to the current state of the Android tree, do the following
+
+ SYNC=$(date -Iminutes)
+ slothfs-expand-manifest > /tmp/${SYNC}.xml
+ ln -s /tmp/${SYNC}.xml /slothfs/config/${SYNC}
+ slothfs-populate -ro /slothfs/${SYNC} .
+
+this will reroute symlinks in your check to your newly created
+workspace. `slothfs-populate` will also update timestamps so incremental builds
+keep working.
+
+
+Removing a workspace
+====================
+
+Remove a workspace by removing its symlink configuration entry, eg.
+
+ rm /slothfs/config/my-workspace
+
+Unmounting slothfs
+==================
+
+A SlothFS daemon can be unmounted with `fusermount`
+
+ fusermount -u /slothfs
+
+If this reports `device busy`, check if there any processes holding open files
+(eg. Jack compilation servers.)
+
+
+Metadata
+========
+
+The file system offers the following metadata files:
+
+ workspace/.slothfs/manifest.xml - manifest XML
+
+ workspace/path/to/repo/.slothfs/tree.json - tree listing of this repository
+ workspace/path/to/repo/.slothfs/treeID - hex tree ID of the this repository
+
+In addition, each blob has the `user.gitsha1` extended attribute that surfaces
+the blob's git SHA1 checksum.
+
+
+Configuring
+===========
+
+SlothFS clones repositories on-demand, as soon as a file in the repository is
+opened. You can avoid cloning altogether for repositories you know you don't
+need; the files will then be fetched over HTTP.
+
+Details of which repositories and files trigger cloning are configured through a
+JSON file.
+
+For example, if you work on Android, and build on a Linux machine, you will
+never need the Darwin related prebuilts. You can avoid a costly clone for those
+by doing:
+
+ {"Repo": ".*darwin.*", "Clone": false}
+
+Similarly, the build system system will read files (typically called '*.mk')
+across the entire tree. When any .mk file is opened, this should not trigger a
+clone. This is achieved with the following entry
+
+ {"File": ".*mk$", "Clone": false}
+
+Together, the following `config.json` file is a good start for working on
+android:
+
+ [{"Repo": ".*darwin.*", "Clone": false},
+ {"File": ".*mk$", "Clone": false}]
+
+A more elaborate configuration file is included as `android.json`.
+
+
+File layout
+-----------
+
+SlothFS loads the configuration data from a directory that can be ste with the
+`-config` flag. The following configuration is available
+
+ $HOME/.config/clone.json # clone configuration
+ $HOME/.config/manifests/ # configured workspaces
+
+SlothFS caches data in a directory which can be set with `-cache` flag.
+The following data are cached:
+
+ $HOME/.cache/slothfs/tree # trees
+ $HOME/.cache/slothfs/git # bare git repositories
+ $HOME/.cache/slothfs/blob # blobs
+
+
+Caveats: timestamps
+-------------------
+
+When syncing a checkout with `slothfs-populate`, an attempt is made to update
+timestamps of the blobs, so incremental builds will work as expected. However,
+since blobs are shared between different workspaces, a sync in one workspace may
+cause spurious rebuilds in other workspaces.
+
+Similarly, interrupting `slothfs-populate` and then syncing to another workspace
+may yield unpredictable results.
+
+When the slothfs FUSE daemon is restarted, all timestamp information is lost.
