Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@cmelone
Copy link
Member

@cmelone cmelone commented Jun 16, 2025

This PR enables syncing commits (and pipeline statuses) between repositories on two Gitlab instances.

Modifications were made to several parts of the codebase:

  • global config options are more verbose to accommodate Gitlab and Github both being options for the source
  • there are now two Gitlab client classes (for dest and src). the logic is different and consolidating them would have a negative effect on readability, imo (open to disagreement though)
  • there are also two Gitlab handler classes for similar reasons
  • many new route handlers in src/hubcast/web/gitlab/routes.py

Feedback would be helpful on software arch decisions and potential consolidation of functionality.

closes #115

outstanding issues (see comments below):

  • can't post pipeline status to forked repos
  • bot user for source gitlab instance?
  • naming of handler and client classes for src/dest

TODOs after this PR:

  • bot/commenting functionality

@cmelone cmelone changed the title WIP: support arbitrary git service for source repo WIP: support gitlab instance for source repo Jul 8, 2025
@cmelone
Copy link
Member Author

cmelone commented Jul 8, 2025

We use pipeline/commit statuses in gitlab to communicate the state of an external pipeline. However, if Hubcast is handling a merge request sync from a fork, it cannot POST any commit status, as it would require write permission to the fork's API endpoint.

See this issue: https://gitlab.com/gitlab-org/gitlab/-/issues/374133

Alternatives:

  • write the external pipeline status to a comment within an MR and update it with changes

let me know if you know of others

edit: we can use this feature: https://docs.gitlab.com/user/project/merge_requests/status_checks in combination with commit statuses (if we have permissions)

edit2: nevermind

Status checks fail if they stay in the pending state for more than two minutes.

@cmelone cmelone force-pushed the add/arbitrary-src-repo branch from 3de8a84 to 36b1dbd Compare July 8, 2025 22:08
cmelone added a commit to cmelone/hubcast that referenced this pull request Jul 8, 2025
Rather than fetching base64-encoded json and decoding it ourselves,
request the raw text of the config file available in the github
repository. This aligns with the approach used in llnl#145.
alecbcs pushed a commit that referenced this pull request Jul 8, 2025
* Fetch raw github repository config

Rather than fetching base64-encoded json and decoding it ourselves,
request the raw text of the config file available in the github
repository. This aligns with the approach used in #145.

* clarify comment

* optional json serialization is not needed
@cmelone
Copy link
Member Author

cmelone commented Jul 11, 2025

re: bot setup if using gitlab as the source repo

Gitlab does not provide a similar mechanism to Github apps, where a collection of permissions/webhooks can be "installed" to a repo.

From what I've seen in other examples (codecov, jenkins), the workaround is:

Instruct Hubcast users to create a "bot" account on the source Github instance. The account should have access at the Developer role.

  • An access token with api and read_repository access would be provided as an env variable to hubcast along with the bot username.
  • The bot user could be given access to many Gitlab groups or individual repositories by the administrators of those entities.
  • The bot would set the status of pipelines and comment on PRs.

Downsides:

  • each repository would need to configure a webhook to create events upon pushes, comments, and merge request changes
    • similar to the way we handle this for the destination, we could handle the creation of this webhook upon setup/register of the repository somehow
  • how would we handle multiple webhooks? they would have the same URL of course, but should they also share webhook secrets?

@cmelone cmelone changed the title WIP: support gitlab instance for source repo Support Gitlab instances for source repositories Jul 11, 2025
@cmelone cmelone force-pushed the add/arbitrary-src-repo branch 2 times, most recently from 6af1d46 to 73a1f07 Compare July 11, 2025 23:45
@cmelone
Copy link
Member Author

cmelone commented Jul 12, 2025

the handlers and clients have names like GitLabSrcClientFactory and GitLabSrcHandler. I haven't changed the destination equivalents yet, or GitHub classes if there's a more attractive option available

@cmelone cmelone marked this pull request as ready for review July 12, 2025 00:23
@cmelone cmelone requested a review from alecbcs July 12, 2025 00:23
@github-actions github-actions bot added the docs Improvements or additions to documentation label Jul 22, 2025
- checks if a source repository is private and authenticates if
  necessary
- private forks are skipped as we cannot access them
- see llnl#189 for the github equivalent
  implementation
@cmelone cmelone force-pushed the add/arbitrary-src-repo branch from 6f7a75c to cbf8809 Compare December 12, 2025 23:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

docs Improvements or additions to documentation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Support Gitlab to Gitlab topology

2 participants