Fix: allow dots in GitHub repository names (closes #7776) #7934

pinakiz · 2025-08-27T17:32:41Z

Fix: allow . in github-identifier-pattern

This updates the regex in the schema so that repository identifiers
can include the . character. Previously, such identifiers were rejected.

Changelog:

Updated github-identifier-pattern regex to include . .

Github Issue: Fixes #7776

Added a new file under changelog/ to document the fix for taskcluster#7776.

for more information, see https://pre-commit.ci

petemoore

Many thanks @pinakiz!

Unfortunately the fix will need to be a bit more sophisticated. See line 5 and 6 above your change, it says

Specifically, the length limitation and the fact that identifiers can't contain dots . is critical.

The issue is that github repositories appear in AMQP routing keys, for example in Pull Request exchanges.

You see the github repository is the third (index 2) entry in the routing key, so having a . in the name would break the message binding.

In order to fix that we would need to change the routing key to be an encoded form of the github repository name, for example, which could also break existing clients and software that relies on the raw (unencoded) repository name being used in the routing key.

Your fix would have been fine, if it wasn't for this, but this rather complicates matters. Probably the real solution is for all routing key entries to escape the . somehow, but that would be a major change.

However, we appreciate your offer of a contribution here, and sorry that it did not work out this time!

Many thanks again.

pinakiz · 2025-08-28T08:49:29Z

@petemoore Thanks a lot for the detailed explanation 🙏. Makes perfect sense about the routing key issue , I definitely learned something new here! Looking forward to contributing again in the future.

lotas · 2025-08-28T09:09:12Z

hey @pinakiz I checked code as well, and found two places that say that dots are actually might be fine:

taskcluster/services/github/src/exchanges.js

Lines 16 to 32 in 28d3aa0

    
           }, { 
        
             name: 'organization', 
        
             summary: 'The GitHub `organization` which had an event. ' + 
        
                               'All periods have been replaced by % - such that ' + 
        
                               'foo.bar becomes foo%bar - and all other special ' + 
        
                               'characters aside from - and _ have been stripped.', 
        
             maxSize: 100, 
        
             required: true, 
        
           }, { 
        
             name: 'repository', 
        
             summary: 'The GitHub `repository` which had an event.' + 
        
                               'All periods have been replaced by % - such that ' + 
        
                               'foo.bar becomes foo%bar - and all other special ' + 
        
                               'characters aside from - and _ have been stripped.', 
        
             maxSize: 100, 
        
             required: true, 
        
           },

And there's a sanitization that is being used in API:

taskcluster/services/github/src/api.js

Lines 21 to 25 in 28d3aa0

    
           // Strips/replaces undesirable characters which GitHub allows in 
        
           // repository/organization names (notably .) 
        
           function sanitizeGitHubField(field) { 
        
             return field.replace(/[^a-zA-Z0-9-_\.]/gi, '').replace(/\./g, '%'); 
        
           }

So it might be that this restriction on the schema level can be lifted, but the challenge here would be to actually test it, and see if existing tests are covering all cases where this might happen.

Maybe we can reopen this and make sure sanitization is used everywhere

petemoore · 2025-08-28T10:24:13Z

Ah, I just saw in the description of repository in my link earlier:

The GitHub repository which had an event. All periods have been replaced by % - such that foo.bar becomes foo%bar - and all other special characters aside from - and _ have been stripped.

So indeed, tests here are key, to ensure the dots are replaced as desired, and anything consuming messages from the message bus are listening for the converted repository name, rather than the raw name.

An orthogonal concern from this comment above (not from the change in this PR) is about two different repositories sharing the same identifier (e.g. repositories foo and foo^ both being indexed under foo). If the repository name is used as a key, and isn't guaranteed to be unique, this could potentially be exploited, or just cause corrupted data (where records overwrite each other).

lotas · 2025-09-01T09:59:24Z

services/github/schemas/constants.yml

 # all common identifiers. It's not personal, it's just that without these
 # limitation, the identifiers won't be useful as routing keys in RabbitMQ
 # topic exchanges. Specifically, the length limitation and the fact that
 # identifiers can't contain dots `.` is critical.


we should update this comment to tell that github service is doing sanitization and all dots are being replaced

lotas · 2025-09-01T09:59:49Z

services/github/schemas/constants.yml

 # topic exchanges. Specifically, the length limitation and the fact that
 # identifiers can't contain dots `.` is critical.
-github-identifier-pattern:     "^([a-zA-Z0-9-_%]*)$"
+github-identifier-pattern:     "^([a-zA-Z0-9-_.%]*)$"


can you please run yarn generate to update all autogenerated schemas as well? Or just let me know, I can run it

Also, I didn't spot any tests that would cover such checks. Would you be comfortable checking if you could add some new tests for repositories with dots?
Might be tricky to setup data for the tests though, I suggest starting with api_test.js

Thanks

Should I allow '%' ? Since ' . ' is later transformed into '%', allowing '%' could create ambiguity.

looks like github is already strict with repository names, I wonder how '%' got there in the first place

The git_Identifier_pattern also allows whitespace-only strings. Is this something we should be concerned about?"

Fix: allow dots in GitHub repository names (closes taskcluster#7776)

39cbf62

pinakiz requested a review from a team as a code owner August 27, 2025 17:32

pinakiz requested review from lotas, petemoore and matt-boris and removed request for a team August 27, 2025 17:32

chore(changelog): add changelog entry for issue taskcluster#7776

5d9f355

Added a new file under changelog/ to document the fix for taskcluster#7776.

pinakiz force-pushed the fix-#7776 branch from 86ca501 to 5d9f355 Compare August 28, 2025 05:31

[pre-commit.ci] auto fixes from pre-commit.com hooks

c222b17

for more information, see https://pre-commit.ci

petemoore requested changes Aug 28, 2025

View reviewed changes

pinakiz closed this Aug 28, 2025

pinakiz reopened this Aug 28, 2025

lotas reviewed Sep 1, 2025

View reviewed changes

pinakiz and others added 2 commits September 1, 2025 21:57

Done: added tests and improved comment.

f2db39f

Delete ca.pem

d641cd9

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix: allow dots in GitHub repository names (closes #7776) #7934

Fix: allow dots in GitHub repository names (closes #7776) #7934

pinakiz commented Aug 27, 2025

Uh oh!

petemoore left a comment •

edited

Loading

Uh oh!

pinakiz commented Aug 28, 2025

Uh oh!

lotas commented Aug 28, 2025 •

edited

Loading

Uh oh!

petemoore commented Aug 28, 2025

Uh oh!

lotas Sep 1, 2025

Uh oh!

lotas Sep 1, 2025

Uh oh!

lotas Sep 1, 2025

Uh oh!

pinakiz Sep 1, 2025

Uh oh!

lotas Sep 1, 2025

Uh oh!

pinakiz Sep 1, 2025

Uh oh!

Uh oh!

Fix: allow dots in GitHub repository names (closes #7776) #7934

Are you sure you want to change the base?

Fix: allow dots in GitHub repository names (closes #7776) #7934

Conversation

pinakiz commented Aug 27, 2025

Uh oh!

petemoore left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

pinakiz commented Aug 28, 2025

Uh oh!

lotas commented Aug 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

petemoore commented Aug 28, 2025

Uh oh!

lotas Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

lotas Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

lotas Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

pinakiz Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

lotas Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

pinakiz Sep 1, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

petemoore left a comment •

edited

Loading

lotas commented Aug 28, 2025 •

edited

Loading