toc: modify disk format to include names for backwards/forwards compatiblity.

Including section names in the table of contents permits simpler
forwards and backwards compatibility. Instead of having to bump
the entire IndexFormatVersion when a new section is added, there is
simply a new section present in the table of contents.

Older versions can read TOCs with unknown sections and skip over them
with a warning. This is useful to permit downgrades without always
requiring a reindex.

Newer versions can read TOCs from older version with missing sections
and handle them gracefully, by checking for empty sections when loading
an index file and implementing whatever fallback code is necessary.

Section evolution is possible by having a new name for a tagged section,
adding the old section to the CompatibilityList, and writing the
conversion code when loading the file, or modifying the users of the
section to use whichever one is loaded.

Change-Id: I9aa05f29eb9d64fd0fff218f008d2031f1a15c8c
7 files changed
tree: 4e8e8bd242c88d4fddc3a6f2521789314b0ce239
  1. build/
  2. cmd/
  3. ctags/
  4. doc/
  5. gitindex/
  6. query/
  7. shards/
  8. testdata/
  9. web/
  10. .gitignore
  11. all.bash
  12. api.go
  13. bits.go
  14. bits_test.go
  16. contentprovider.go
  18. eval.go
  19. eval_test.go
  20. go.mod
  21. go.sum
  22. hititer.go
  23. hititer_test.go
  24. index_test.go
  25. indexbuilder.go
  26. indexdata.go
  27. indexfile_other.go
  28. indexfile_unix.go
  30. matchiter.go
  31. matchtree.go
  32. matchtree_test.go
  33. read.go
  34. read_test.go
  36. section.go
  37. section_test.go
  38. toc.go
  39. write.go
"Zoekt, en gij zult spinazie eten" - Jan Eertink

("seek, and ye shall eat spinach" - My primary school teacher)

This is a fast text search engine, intended for use with source code. (Pronunciation: roughly as you would pronounce “zooked” in English)



go get


go install
$GOPATH/bin/zoekt-index .


go install
$GOPATH/bin/zoekt 'ngram f:READ'

Indexing git repositories:

go install
$GOPATH/bin/zoekt-git-index -branches master,stable-1.4 -prefix origin/ .

Indexing repo repositories:

go install{repo-index,mirror-gitiles}
zoekt-mirror-gitiles -dest ~/repos/
zoekt-repo-index \
   -name gfiber \
   -base_url \
   -manifest_repo ~/repos/ \
   -repo_cache ~/repos \
   -manifest_rev_prefix=refs/heads/ --rev_prefix= \

Starting the web interface

go install
$GOPATH/bin/zoekt-webserver -listen :6070

A more organized installation on a Linux server should use a systemd unit file, eg.

Description=zoekt webserver

ExecStart=/zoekt/bin/zoekt-webserver -index /zoekt/index -listen :443  --ssl_cert /zoekt/etc/cert.pem   --ssl_key /zoekt/etc/key.pem



Zoekt comes with a small service management program:

go install

cat << EOF > config.json
[{"GithubUser": "username"},
 {"GithubOrg": "org"},
 {"GitilesURL": "", "Name": "zoekt" }

$GOPATH/bin/zoekt-server -mirror_config config.json

This will mirror all repos under ‘’, ‘’, as well as the ‘zoekt’ repository. It will index the repositories.

It takes care of fetching and indexing new data and cleaning up logfiles.

The webserver can be started from a standard service management framework, such as systemd.


It is recommended to install Universal ctags to improve ranking. See here for more information.


Thanks to Alexander Neubeck for coming up with this idea, and helping me flesh it out.


This is not an official Google product