Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

directhex
Copy link
Contributor

This is a prerequisite for inclusion on source.dot.net.

It seems a first attempt was made to add sdk.git to source.dot.net in early 2021, but it was added incorrectly and not fixed at any point in the interim.

Generally speaking, "V2" repositories (data sent in by individual repo CI) are more robust for inclusion than the old V1 repos (repo data generated during source-indexer CI runs) - as such we aren't really interested in adding or fixing any V1 repos, and would like to move to V2.

This move also allows the repo owners who know their code best to fine-tune their build flags and environment far more easily than for V1 repos.

Please note that there is an issue under active investigation where V2 repository publishing can occasionally fail due to obscure permissions errors from Azure since we switched uploads from SAS to Managed Identity (currently about 6 failures per day total across all repositories)

@directhex
Copy link
Contributor Author

- ${{ if eq(variables.enableSourceIndex, 'true') }}:
- template: /eng/common/templates-official/job/source-index-stage1.yml@self
parameters:
sourceIndexBuildCommand: powershell -NoLogo -NoProfile -ExecutionPolicy Bypass -Command "eng/common/build.ps1 -restore -build -ci -nativeToolsOnMachine -configuration $(buildConfiguration)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need to specify the binlog parameter here, or does the script always produce it to artifacts/log/$(buildConfiguration)/Build.binlog?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for Eric's comment. Also, I don't like the idea of copying the build process manually into a parameter, that then runs PowerShell, that then runs the build. Instead, it should use our build template, sdk-build.yml, to do a build. That template can do a basic build with no tests (and will produce a binlog) as long as the right parameters are provided to the template.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need a way to specify the binlog path, because there is no consistent buildConfiguration variable across all repos we can rely on. If we could unify what we call our configuration, then great! But that's a lot of repos.

As-is, the template has a default of artifacts/log/Debug/Build.binlog

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@MiYanni okay, I've done some experimental redesign on the netsourceindex Arcade integration (using SDK as my test repo), and I'd appreciate some design feedback before I start trying to get it into Arcade.

Flow wise, it's not possible for netsourceindex's template to ingest sdk-build.yml, because both define jobs: (and jobs: cannot be nested within a job). So I've moved all the steps: from netsourceindex into a separate template which the existing job template imports (for backward compatibility) - and the new steps template can be imported by any existing (or new) job defined by the target repo as they see fit.

AS AN EXAMPLE, NOT A RECOMMENDATION I added it here on a not-yet-pushed-to-github internal copy of branch this. A build looks something like https://dev.azure.com/dnceng/internal/_build/results?buildId=2524323&view=logs&j=fa4e7bdb-0412-58a3-8823-7a72e2d8c65d&t=5cdd6b27-41ac-530b-e1af-013abe4bc807

I think adding a new managed-only Windows build leg would be best, but the template design allows that kind of decision to be made by the repo owner to best fit their needs.

If the idiom seems sound to you, then I can look at sending a PR to Arcade with the required prerequisite changes for adoption.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@directhex That looks much better as it doesn't require reconstitution of the build process by the repo. It being a drop-in template like this works much better, IMO. For our repo, if this process only needs to run once on the Windows job, you should do it like this:

  1. Add a new variable named uploadSourceIndex under here and set it to true.
  2. Add the same uploadSourceIndex variable under here and set it to false.
  3. Since it looks like you want the build outputs included in the source index, the template should be added under here under that "Publish Logs" task.
- ${{ if eq(parameters.uploadSourceIndex, true) }}:
  - template: /eng/common/core-templates/steps/source-index-stage1-publish.yml
    parameters:
      binLogPath: artifacts\log\$(buildConfiguration)\Build.binlog

This way, it only runs once on the official Windows job and no where else.

@ericstj ericstj requested review from marcpopMSFT and MiYanni July 18, 2024 18:50
- ${{ if eq(variables.enableSourceIndex, 'true') }}:
- template: /eng/common/templates-official/job/source-index-stage1.yml@self
parameters:
sourceIndexBuildCommand: powershell -NoLogo -NoProfile -ExecutionPolicy Bypass -Command "eng/common/build.ps1 -restore -build -ci -nativeToolsOnMachine -configuration $(buildConfiguration)"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 for Eric's comment. Also, I don't like the idea of copying the build process manually into a parameter, that then runs PowerShell, that then runs the build. Instead, it should use our build template, sdk-build.yml, to do a build. That template can do a basic build with no tests (and will produce a binlog) as long as the right parameters are provided to the template.

Copy link
Contributor

Due to lack of recent activity, this PR has been labeled as 'Stale'. It will be closed if no further activity occurs within 7 more days. Any new comment will remove the label.

@github-actions github-actions bot added the stale label Sep 22, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area-Infrastructure stale untriaged Request triage from a team member
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants