Google Git
Sign in
gerrit / zoekt / 88436ac05ffdb586e65d18461c7c13c4bc33de9b
commit88436ac05ffdb586e65d18461c7c13c4bc33de9b[log] [tgz]
authorKeegan Carruthers-Smith <keegan.csmith@gmail.com>Thu Aug 12 11:41:39 2021 +0200
committerKeegan Carruthers-Smith <keegan.csmith@gmail.com>Tue Aug 17 10:31:50 2021 +0200
tree476cc59a489926a860918e184bf2d136c5e91836
parentb214ed5aae784a99b695a076dd4db493f95d28ed [diff]
shards: log a summary of shards in toLoad

We don't log every shard we load due to logspam on startup. However,
while zoekt-webserver is running it is useful to see more detailed
information since we are normally only loading a small number of shards
at a time.

This commit strikes a balance by only logging 5 of the shard names in
the list of shards to load.

Additionally for the unloading log line we only mention the base of the
path since it should always be in the indexdir.

Change-Id: I444a594cf5ea3359772b5f42651ee860b2a0fc1f
2 files changed
tree: 476cc59a489926a860918e184bf2d136c5e91836
  1. build/
  2. cmd/
  3. ctags/
  4. doc/
  5. gitindex/
  6. query/
  7. shards/
  8. web/
  9. .gitignore
  10. all.bash
  11. api.go
  12. bits.go
  13. bits_test.go
  14. build-deploy.sh
  15. contentprovider.go
  16. CONTRIBUTING
  17. eval.go
  18. eval_test.go
  19. go.mod
  20. go.sum
  21. hititer.go
  22. hititer_test.go
  23. index_test.go
  24. indexbuilder.go
  25. indexdata.go
  26. indexfile_other.go
  27. indexfile_unix.go
  28. LICENSE
  29. matchiter.go
  30. matchtree.go
  31. matchtree_test.go
  32. read.go
  33. read_test.go
  34. README.md
  35. section.go
  36. section_test.go
  37. toc.go
  38. write.go
README.md
"Zoekt, en gij zult spinazie eten" - Jan Eertink

("seek, and ye shall eat spinach" - My primary school teacher)

This is a fast text search engine, intended for use with source code. (Pronunciation: roughly as you would pronounce “zooked” in English)

INSTRUCTIONS

Downloading:

go get github.com/google/zoekt/

Indexing:

go install github.com/google/zoekt/cmd/zoekt-index
$GOPATH/bin/zoekt-index .

Searching

go install github.com/google/zoekt/cmd/zoekt
$GOPATH/bin/zoekt 'ngram f:READ'

Indexing git repositories:

go install github.com/google/zoekt/cmd/zoekt-git-index
$GOPATH/bin/zoekt-git-index -branches master,stable-1.4 -prefix origin/ .

Indexing repo repositories:

go install github.com/google/zoekt/cmd/zoekt-{repo-index,mirror-gitiles}
zoekt-mirror-gitiles -dest ~/repos/ https://gfiber.googlesource.com
zoekt-repo-index \
   -name gfiber \
   -base_url https://gfiber.googlesource.com/ \
   -manifest_repo ~/repos/gfiber.googlesource.com/manifests.git \
   -repo_cache ~/repos \
   -manifest_rev_prefix=refs/heads/ --rev_prefix= \
   master:default_unrestricted.xml

Starting the web interface

go install github.com/google/zoekt/cmd/zoekt-webserver
$GOPATH/bin/zoekt-webserver -listen :6070

A more organized installation on a Linux server should use a systemd unit file, eg.

[Unit]
Description=zoekt webserver

[Service]
ExecStart=/zoekt/bin/zoekt-webserver -index /zoekt/index -listen :443  --ssl_cert /zoekt/etc/cert.pem   --ssl_key /zoekt/etc/key.pem
Restart=always

[Install]
WantedBy=default.target

SEARCH SERVICE

Zoekt comes with a small service management program:

go install github.com/google/zoekt/cmd/zoekt-indexserver

cat << EOF > config.json
[{"GithubUser": "username"},
 {"GithubOrg": "org"},
 {"GitilesURL": "https://gerrit.googlesource.com", "Name": "zoekt" }
]
EOF

$GOPATH/bin/zoekt-server -mirror_config config.json

This will mirror all repos under ‘github.com/username’, ‘github.com/org’, as well as the ‘zoekt’ repository. It will index the repositories.

It takes care of fetching and indexing new data and cleaning up logfiles.

The webserver can be started from a standard service management framework, such as systemd.

SYMBOL SEARCH

It is recommended to install Universal ctags to improve ranking. See here for more information.

ACKNOWLEDGEMENTS

Thanks to Alexander Neubeck for coming up with this idea, and helping me flesh it out.

DISCLAIMER

This is not an official Google product