Turns out that GitHub (unlike VS Code's default theme) highlights the token "support.type" with a very similar shade of blue as the value tokens

For that reason, I've added a formatting step in the syncing process to convert the scope "support.type" to "entity.name.tag" in order to get the same color as keys in JSON files. Just need to wait another 3 months and JSONC will finally look good!

@DecimalTurn In the future, you might want to consult this file to judge the similarity between different scopes. In addition, this is worth a read if you're unfamiliar with how TextMate scopes differ to CSS classes.

@Alhadis Thanks for sharing, very useful!

VSCode does their own reformatting aswell https://github.com/microsoft/vscode/blob/main/extensions/json/build/update-grammars.js

{
  "key": "value"
}

"key": "value"

VSCode does their own reformatting aswell

Yeah, but I'm using the final product, so this shouldn't be the reason for the discrepencies.

And even with the 9.2.0 release, the problem with coloring wasn't fixed 😞.

Could it be that PrettyLights gives precedence to the first token here?
"name": "string.json.comments entity.name.tag.property-name.json.comments",

That would mean that the string token is used instead of entity which would explain why the keys have the same color as value-strings.

@Alhadis has a page/gist with how prettylights renders various parts of grammars but I can't remember its location.

found it https://github.com/Alhadis/Atom-GitHubSyntax?tab=readme-ov-file#textmate-vs-css

and heres the supported scopes https://github.com/Alhadis/language-etc/blob/master/samples/lists/scope-previews.nanorc

does https://novalightshow.netlify.app/ correctly show Githubs shortfall?

found it https://github.com/Alhadis/Atom-GitHubSyntax?tab=readme-ov-file#textmate-vs-css

Ironically, this was already linked in this thread, but I evidently didn't read it properly as it clearly answers my previous question 😅.

does https://novalightshow.netlify.app/ correctly show Githubs shortfall?

Unfortunately, no. The behavior varies between the two.

Could it be that PrettyLights gives precedence to the first token here?
"name": "string.json.comments entity.name.tag.property-name.json.comments",

The space in that scope's name is ambiguous. Did you mean to tokenise code like this…?

<token scope="string.json.comments">
	<token scope="entity.name.tag.property-name.json.comments">
</token>

Or this…?

<token scope="string.json.comments entity.name.tag.property-name.json.comments">

(Both of the above code-blocks are using pseudocode to communicate the intended token structure).

If it's the latter, and you really did mean to include a space in a scope-name, then it's malformed. Whitespace will be treated as-is by the TextMate engine, and will be matched only by a selector that specifies a scope that's literally named comments entity. The CSS equivalent of this is .comments\ entity, which attempts to specify a class-name that includes whitespace, which is impossible:

document.body.classList.toggle("comments entity");
// Uncaught DOMException: DOMTokenList.toggle: The token can not contain whitespace.

The other problem is that TextMate selectors are order-sensitive, a matter that's discussed in detail here. Basically, string.json.comment won't be highlighted like string.comment because the second scope is json instead of comment.

By the same token (no pun intended), you can inhibit highlighting of desirable scopes that have undesirable appearances on GitHub; i.e, variable.language has more generic colouring than variable does, but if you want your grammar to include a scope-list of variable.language.readonly for editor-specific reasons, you can instead write variable.readonly.language instead. I believe VSCode treats scopes more akin to CSS classes, so the order of scopes here is irrelevant.

VSCode splits the scopeNames on spaces
so string markup.italic will apply both the scopes string and markup.italic
making it both brown and italic (in VSCode)

I don't know why VSCode chose to change it
microsoft/vscode-textmate@8e4fb9a
maybe @alexdima knows

VSCode treats scopes more akin to CSS classes, so the order of scopes here is irrelevant.

VSCode is order sensitive just like TextMate

VSCode is order sensitive just like TextMate

Shit, I wasn't aware of that. It's been a while since I've even touched the editor, but I faintly recall CSS being used at some point… (which might've been a vague assumption based solely on VS Code being an Electron app, and therefore implemented with a CSS/HTML-based interface).

I stand corrected regarding VSCode's TextMate implementation, then.

Did you mean [...]?

@Alhadis, I can't really answer questions with regards to the original intentions because that grammar is mostly an exact copy of the grammar for JSONC used by VS Code internally (from here). My goal was to get the same behavior as what is seen in VS Code with as few changes as possible.

To solve this latest tokenization isssue, I've changed the order to be : "name": "entity.name.tag.property-name.json.comments string.json.comments",. I understand that this won't apply both tokens and just entity.name should be recognized, but that's the outcome I'm looking for in order to get the desired coloring.

@DecimalTurn TextMate 2.0 doesn't like spaces in it scopeNames
comment.comment comment.comment will not apply token comment
comment.comment comment.comment will apply token comment
comment comment will not apply token comment
maybe Github is different
but it definitely isn't compatible with TextMate

comment.comment comment.comment will not apply token comment

Really? This screenshot from Textmate produced with this version (that I just created to test this) shows that the string token was applied to the keys even if there was a space:

comment.comment comment.comment will not apply token comment

Really? This screenshot from Textmate produced with [this version](https://github.com/DecimalTurn/vscode-jsonc-syntax-highlighting/commit/750f857e58aca204a3e3dab588b40bc2ce964aa5) (that I just created to test this) shows that the `string` token was applied to the keys even if there was a space:

@DecimalTurn I believe @RedCMD intended to write:

comment.comment comment.comment will not apply token comment comment

The space doesn't invalidate the entire scope-list, only those that come after it. The string scope in your above example works because it precedes the use of whitespace.

Ok, that was my understanding as well. Thanks for clarifying.
I'm satisfied with the re-ordered version from my previous comment. I'll stick with that until the next release 🤞.

my bad
did more testing
and it seems using a space causes weird behaviours
it only works if the first scopeName contains a period . (but not at the start)
and it only applies the very first scopePart
eg
string.regexp support.function will only apply token string; not string.regexp OR string.regexp support.function
string support.function will not apply token string
string. support.function will only apply token string

support dark purple
support.function dark yellow
string green
string.regexp orange yellow

entity.name.tag.property-name.json.comments string.json.comments happens to work because entity is a valid theme token and has a period after it

Very interesting findings @RedCMD, thanks for sharing. This means that if PrettyLights behaves the same as TextMate, we should see that JSONC keys highlighted with purple if I follow the logic correctly. That's still better than blue, so I'm willing to let it as is just to test things out (for science!).

LightShow link