Fix VMR source-only build to use --with-packages for prebuilts #3824
Conversation
…with-sdk and --with-packages
…archive directory entirely
… prep and build scripts
|
There's a script error: |
|
@csa7mdm - I see activity from you of merging but not addressing the build issue. Are you planning on doing that? |
There was a problem hiding this comment.
Pull request overview
This pull request modifies the VMR source-only build pipeline to properly test the --with-packages argument by changing where prebuilt artifacts are downloaded and extracted. Previously, artifacts were downloaded to the default location (prereqs/packages/archive/), allowing build.sh to find them implicitly without exercising the --with-packages code path.
Key Changes:
- Changed
reuseBuildArtifactsFromparameter type from string to array (object) - Modified artifact download destination from default location to a custom temporary directory
- Added explicit
--with-packagesargument construction when reusing build artifacts
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| if [[ '${{ length(parameters.reuseBuildArtifactsFrom) }}' -gt 0 ]]; then | ||
| # Prebuilt packages are extracted in the Prep step | ||
| : | ||
| fi |
There was a problem hiding this comment.
This empty conditional block serves no purpose and should be removed. The comment indicates that prebuilt packages are extracted in the Prep step, but no action is taken here. If this is meant as a placeholder or documentation, it should either be removed entirely or the logic should be clarified.
| if [[ '${{ length(parameters.reuseBuildArtifactsFrom) }}' -gt 0 ]]; then | |
| # Prebuilt packages are extracted in the Prep step | |
| : | |
| fi | |
| # Prebuilt packages are extracted in the Prep step when reusing build artifacts. |
| tar -oxzf "$artifactsTarball" -C "$(previouslySourceBuiltArtifactsDir)" | ||
| customPrepArgs="${customPrepArgs} --with-packages $(previouslySourceBuiltArtifactsDir)/" | ||
| additionalBuildArgs="${additionalBuildArgs} --with-packages $(previouslySourceBuiltArtifactsDir)/" |
There was a problem hiding this comment.
The 'additionalBuildArgs' variable is being appended to with the '--with-packages' argument, but it already may contain '--with-sdk' from the previous 'if' block (line 594). A leading space should be added when appending to ensure proper separation between arguments. Currently, if both SDK and packages are being reused, the arguments will be concatenated incorrectly.
| # Verify download directory exists | ||
| if [ ! -d "$(downloadedArtifactsDir)" ]; then | ||
| echo "Error: SDK extraction required but '$(downloadedArtifactsDir)' does not exist." | ||
| echo "Checking Pipeline Workspace content:" | ||
| ls -R "$(Pipeline.Workspace)" || true | ||
| exit 1 | ||
| fi |
There was a problem hiding this comment.
The directory existence check is performed unconditionally when prepSdk is false, but the downloadedArtifactsDir only exists when reuseBuildArtifactsFrom is non-empty. This will cause the script to fail when withPreviousSDK is True but reuseBuildArtifactsFrom is empty, because prepSdk will be false (set at line 549) but no artifacts will have been downloaded to downloadedArtifactsDir. The check should be conditional on whether reuseBuildArtifactsFrom is actually set.
| customPrepArgs="${customPrepArgs} --with-packages $(previouslySourceBuiltArtifactsDir)/" | ||
| additionalBuildArgs="${additionalBuildArgs} --with-packages $(previouslySourceBuiltArtifactsDir)/" |
There was a problem hiding this comment.
Inconsistent trailing slash handling. The path includes a trailing slash here, but the variable 'previouslySourceBuiltArtifactsDir' is already defined without one (line 221). This creates an unnecessary double-slash in the path. Either remove the trailing slash here, or ensure it's consistently applied.
| customPrepArgs="${customPrepArgs} --with-packages $(previouslySourceBuiltArtifactsDir)/" | |
| additionalBuildArgs="${additionalBuildArgs} --with-packages $(previouslySourceBuiltArtifactsDir)/" | |
| customPrepArgs="${customPrepArgs} --with-packages $(previouslySourceBuiltArtifactsDir)" | |
| additionalBuildArgs="${additionalBuildArgs} --with-packages $(previouslySourceBuiltArtifactsDir)" |
| echo "Found SDK tarball: $previousSdkPath" | ||
| echo "Extracting SDK..." | ||
| eval tar -oxzf "$previousSdkPath" -C "$(previouslySourceBuiltSdkDir)" |
There was a problem hiding this comment.
Inconsistent tar command flags. Line 591 uses 'tar -oxzf' while line 620 uses 'tar -oxzf' as well. However, line 591 uses 'eval' before the tar command while line 620 doesn't. The 'eval' on line 591 appears unnecessary since there's no shell expansion needed in the command, and its presence is inconsistent with the similar tar command at line 620. Consider removing the 'eval' for consistency.
| eval tar -oxzf "$previousSdkPath" -C "$(previouslySourceBuiltSdkDir)" | |
| tar -oxzf "$previousSdkPath" -C "$(previouslySourceBuiltSdkDir)" |
| if [[ -n "$additionalBuildArgs" ]]; then | ||
| echo "##vso[task.setvariable variable=additionalBuildArgs]$additionalBuildArgs" | ||
| fi | ||
| fi |
There was a problem hiding this comment.
Extra closing 'fi' statement found that doesn't match any opening 'if' statement. This appears to be left over from the old code structure and will cause a shell script syntax error. The 'if' statement that this seems to match was removed in the refactoring (the one that checked 'prepSdk == false'), but its closing was placed after the additionalBuildArgs logic, which is incorrect.
| fi |
|
@copilot open a new pull request to apply changes based on the comments in this thread |
Fixes dotnet/source-build#4427
Technical Analysis
The VMR validation pipeline (
vmr-build.yml) was downloading prebuilt artifacts to the defaultprereqs/packages/archive/location, which allowedbuild.shto pick them up implicitly. This left the explicit--with-packagesargument path untested in CI.Changes
eng/pipelines/templates/jobs/vmr-build.ymlto download prebuilts to a custom location (prebuilt-packages/).--with-packagestobuild.shwhenreuseBuildArtifactsFromis set.CustomPreviouslySourceBuiltPackagesPathlogic in the build script is exercised and verified.Verification
build.shaccepts the--with-packagesargument.