tree 3c50665241a44ec5ff806deffeed512760a31f4b
parent 68ffcf060d8ab341ad5d7e6e3eef0a3961073ab0
author Keegan Carruthers-Smith <keegan.csmith@gmail.com> 1622040206 +0200
committer Keegan Carruthers-Smith <keegan.csmith@gmail.com> 1626166774 +0200

index: read in ngrams in own function

Our heap profiles are dominated by readIndexData. By moving some of the
work into its own functions, the heap profiles will tell us where in
particular data is allocated from. We suspect most data is from
readNgrams, but want to confirm emperically.

Co-authored-by: Stefan Hengl <stefan@sourcegraph.com>
Change-Id: I85cb05f27ac2c2d9135c816173a8a97cea6a706b