project: initial separation of shared project objects

For now, this is opt-in via environment variables:
  - export REPO_USE_ALTERNATES=1

The shared project logic that shares the internal .git/objects/ dir
directly between multiple projects via the project-objects/ tree has
a lot of KI with random corruption.  It all boils down to projects
sharing objects/ but not refs/.  Git operations that use refs to see
what objects are reachable and discard the rest can easily discard
objects that are used by other projects.

Consider this project layout:
<show fs layout>

There are unique refs in each of these trees that are not visible in
the others.  This means it's not safe to run basic operations like
git prune or git gc.

Since we can't share refs (each project needs to have unique refs
like HEAD in order to function), let's change how we share objects.
The old way involved symlinking .git/objects/ to the project-objects
tree.  The new way shares objects using git's info/alternates.

This means project-objects/ will only contain objects that exist in
the remote project.  Local per-project objects (like when creating
branches and making changes) will never be shared.  When running a
prune or gc operation in the per-project state, it will only ever
repack or discard those per-project objects.  The common shared
objects would only be cleaned up when running a common operation
(i.e. by repo itself).

One downside to this for users is if they try blending unrelated
upstream projects.  For example, in CrOS we have multiple kernel
projects (for diff versions) checked out.  If a dev fetched the
upstream Linus tree into one of them, the objects & tags would
not be shared with the others, so they would have to fetch the
upstream state for each project.  Annoying, but better than the
current corruption situation we're in now.

Also if the dev runs a manual `git fetch` in the per-project to
sync it up to newer state than the last `repo sync` they ran,
the objects would get duplicated.  However, git operations later
on should eventually dedupe this.

Bug: https://crbug.com/gerrit/15553
Change-Id: I313a9b8962f9d439ef98ac0ed37ecfb9e0b3864e
Reviewed-on: https://gerrit-review.googlesource.com/c/git-repo/+/328101
Reviewed-by: Mike Frysinger <vapier@google.com>
Tested-by: LaMont Jones <lamontjones@google.com>
2 files changed
tree: 7804e4ba1a12a0a25bbff7360d69010e64464c72
  1. .github/
  2. docs/
  3. hooks/
  4. man/
  5. release/
  6. subcmds/
  7. tests/
  8. .flake8
  9. .gitattributes
  10. .gitignore
  11. .gitreview
  12. .mailmap
  13. .project
  14. .pydevproject
  15. color.py
  16. command.py
  17. completion.bash
  18. editor.py
  19. error.py
  20. event_log.py
  21. fetch.py
  22. git_command.py
  23. git_config.py
  24. git_refs.py
  25. git_ssh
  26. git_superproject.py
  27. git_trace2_event_log.py
  28. gitc_utils.py
  29. hooks.py
  30. LICENSE
  31. main.py
  32. MANIFEST.in
  33. manifest_xml.py
  34. pager.py
  35. platform_utils.py
  36. platform_utils_win32.py
  37. progress.py
  38. project.py
  39. README.md
  40. repo
  41. repo_trace.py
  42. requirements.json
  43. run_tests
  44. setup.py
  45. ssh.py
  46. SUBMITTING_PATCHES.md
  47. tox.ini
  48. wrapper.py
README.md

repo

Repo is a tool built on top of Git. Repo helps manage many Git repositories, does the uploads to revision control systems, and automates parts of the development workflow. Repo is not meant to replace Git, only to make it easier to work with Git. The repo command is an executable Python script that you can put anywhere in your path.

Contact

Please use the repo-discuss mailing list or issue tracker for questions.

You can file a new bug report under the “repo” component.

Please do not e-mail individual developers for support. They do not have the bandwidth for it, and often times questions have already been asked on repo-discuss or bugs posted to the issue tracker. So please search those sites first.

Install

Many distros include repo, so you might be able to install from there.

# Debian/Ubuntu.
$ sudo apt-get install repo

# Gentoo.
$ sudo emerge dev-vcs/repo

You can install it manually as well as it's a single script.

$ mkdir -p ~/.bin
$ PATH="${HOME}/.bin:${PATH}"
$ curl https://storage.googleapis.com/git-repo-downloads/repo > ~/.bin/repo
$ chmod a+rx ~/.bin/repo