commit | dcb02654360e7617380361a3b755dc380c239edf | [log] [tgz] |
---|---|---|
author | Yunjie Li <yunjieli@google.com> | Tue Feb 11 11:06:33 2020 -0800 |
committer | Yunjie Li <yunjieli@google.com> | Tue May 12 17:32:15 2020 -0700 |
tree | 5b8d0bece1ac610da5506faf29ccf5fd793e4f02 | |
parent | 067d946090c4db43134687c93d30192ad1314bec [diff] |
PackBitmapIndex: Reduce memory usage in GC Currently, the garbage collection is consistently failing for some large repositories in the building bitmap phase, e.g.Linux-MSM project: https://source.codeaurora.org/quic/la/kernel/msm-3.18 Historically, bitmap index creation happened in 3 phases: 1. Select the commits to which bitmaps should be attached. 2. Create all bitmaps for these commits, stored in uncompressed format in the PackBitmapIndexBuilder. 3. Deltify the bitmaps and write them to disk. We investigated the process. For phase 2 it's most efficient to create bitmaps starting with oldest commit and moving to the newest commit, because the newer commits are able to reuse the work for the old ones. But for bitmap deltification in phase 3, it's better when a newer commit's bitmap is the base, and the current disk format writes bitmaps out for the newest commits first. This change introduces a new collection to hold the deltified and compressed representations of the bitmaps, keeping a smaller subset of commits in the PackBitmapIndexBuilder to help make the bitmap index creation more memory efficient. And in this commit, we're setting DISTANCE_THRESHOLD to 0 in the PackWriterBitmapPreparer, which means the garbage collection will not have much behavoir change and will still use as much memory as before. Change-Id: I6ec2c3e8dde11805af47874d67d33cf1ef83660e Signed-off-by: Yunjie Li <yunjieli@google.com>
An implementation of the Git version control system in pure Java.
This project is licensed under the EDL (Eclipse Distribution License).
JGit can be imported straight into Eclipse and built and tested from there. It can be built from the command line using Maven or Bazel. The CI builds use Maven and run on Jenkins.
org.eclipse.jgit
A pure Java library capable of being run standalone, with no additional support libraries. It provides classes to read and write a Git repository and operate on a working directory.
All portions of JGit are covered by the EDL. Absolutely no GPL, LGPL or EPL contributions are accepted within this package.
org.eclipse.jgit.ant
Ant tasks based on JGit.
org.eclipse.jgit.archive
Support for exporting to various archive formats (zip etc).
org.eclipse.jgit.http.apache
Apache httpclient support.
org.eclipse.jgit.http.server
Server for the smart and dumb Git HTTP protocol.
org.eclipse.jgit.lfs
Support for LFS (Large File Storage).
org.eclipse.jgit.lfs.server
Basic LFS server support.
org.eclipse.jgit.packaging
Production of Eclipse features and p2 repository for JGit. See the JGit Wiki on why and how to use this module.
org.eclipse.jgit.pgm
Command-line interface Git commands implemented using JGit (“pgm” stands for program).
org.eclipse.jgit.ssh.apache
Client support for the ssh protocol based on Apache Mina sshd.
org.eclipse.jgit.ui
Simple UI for displaying git log.
Native symbolic links are supported, provided the file system supports them. For Windows you must use a non-administrator account and have the SeCreateSymbolicLinkPrivilege.
Only the timestamp of the index is used by JGit if the index is dirty.
JGit requires at least a Java 8 JDK.
CRLF conversion is performed depending on the core.autocrlf
setting, however Git for Windows by default stores that setting during installation in the “system wide” configuration file. If Git is not installed, use the global or repository configuration for the core.autocrlf setting.
The system wide configuration file is located relative to where C Git is installed. Make sure Git can be found via the PATH environment variable. When installing Git for Windows check the “Run Git from the Windows Command Prompt” option. There are other options like Eclipse settings that can be used for pointing out where C Git is installed. Modifying PATH is the recommended option if C Git is installed.
We try to use the same notation of $HOME
as C Git does. On Windows this is often not the same value as the user.home
system property.
org.eclipse.jgit
Read loose and packed commits, trees, blobs, including deltafied objects.
Read objects from shared repositories
Write loose commits, trees, blobs.
Write blobs from local files or Java InputStreams.
Read blobs as Java InputStreams.
Copy trees to local directory, or local directory to a tree.
Lazily loads objects as necessary.
Read and write .git/config files.
Create a new repository.
Read and write refs, including walking through symrefs.
Read, update and write the Git index.
Checkout in dirty working directory if trivial.
Walk the history from a given set of commits looking for commits introducing changes in files under a specified path.
Object transport
Fetch via ssh, git, http, Amazon S3 and bundles. Push via ssh, git and Amazon S3. JGit does not yet deltify the pushed packs so they may be a lot larger than C Git packs.
Garbage collection
Merge
Rebase
And much more
org.eclipse.jgit.pgm
org.eclipse.jgit.ant
org.eclipse.jgit.archive
org.eclipse.http
There are some missing features:
Post questions, comments or discussions to the jgit-dev@eclipse.org mailing list. You need to be subscribed to post. File bugs and enhancement requests in Bugzilla.
See the EGit Contributor Guide.
More information about Git, its repository format, and the canonical C based implementation can be obtained from the Git website.