tree fa1adb429488862206aac2fcb576c7332ed28d1b
parent cdb8434c50fb59e4143dd983de7b9561f471c93f
author Ole Rehmsen <oler@google.com> 1627636016 +0200
committer Ole Rehmsen <oler@google.com> 1627636016 +0200

Do not consider `-` part of tokens to highlight

This is a trade-off:

In most programming languages, `-` cannot be part of an identifier. On
the contrary, it is often used in operators, such as for subtraction or
as part of `->` it C++ pointer dereferencing.

In other languages such as CSS - can be part of an identifier
(e.g. 'some-css-class).

I am argueing that `-` being part of a word/token is rarer: Regexp \w
does not contain `-` for example, and if you double click on a -
separated word in the browser, it will only select the segment until the
-.

In the long term, we might make this dependent on the input language,
until then I think that not including - is better than incuding it.

Change-Id: Ic843c87db5fb75aecceb7581654ef2b1a6c1793b