Fix miscellaneous ci issues #78

DeeDeeG · 2020-08-04T16:33:36Z

Requirements for Contributing a Bug Fix (from template)

Fill out the template below. Any pull request that does not include enough information to be reviewed in a timely manner may be closed at the maintainers' discretion.
The pull request must only fix an existing bug. To contribute other changes, you must use a different template. You can see all templates at https://github.com/atom/.github/tree/master/.github/PULL_REQUEST_TEMPLATE.
The pull request must update the test suite to demonstrate the changed functionality. For guidance, please see https://flight-manual.atom.io/hacking-atom/sections/writing-specs/.
After you create the pull request, all status checks must be pass before a maintainer reviews your contribution. For more details, please see https://github.com/atom/.github/tree/master/CONTRIBUTING.md#pull-requests.

Identify the Bug

Fixes most of #70

Description of the Change

Update Windows job names.

Windows --> Windows_build makes it more obvious this job simply builds Windows, nothing more.
- Windows_RendererTests now depend on Windows_build, not Windows.
- Release now depends on Windows_RendererTests, not Windows.

Delete a comment about containers in Linux preparation

We don't run the Linux build in an Ubuntu 14.04 "Trusty Tahr" Docker container anymore like upstream does. We run directly in a virtualized Ubuntu 16.04 "Xenial Xerus" image for now.

Set fallback/default values for the REPO metadata variables: REPO_OWNER --> 'atom'; MAIN_REPO --> 'atom'; NIGHTLY_RELEASE_REPO --> 'atom-nightly-releases'

We can override these to match our own repos.

Delete an old workaround that's no longer applicable, with npm install in a scrfipt block, immefiately followed by another script block

This no longer "inexplicably exits after npm install", so just combine the two adjacent script blocks.
Delete the comment explaining the workaround, since it's no longer true, AND because this PR drops the workaround altogether.

Alternate Designs

Not fix these (admittedly minor) issues?

Possible Drawbacks

None that I know of. Watch to make sure that deleting cd script/vsts && npm install on Windows doesn't somehow cause problems.

Watch to make sure I didn't typo the job names under Windows.

Verification Process

CI should pass. Reviewing the diff should confirm there are no typos in the PR.

Release Notes

N/A

DeeDeeG · 2020-08-04T18:36:08Z

@aminya I can move the REPO_OWNER, MAIN_REPO and NIGHTLY_RELEASE_REPO var handling changes to another pull request if you prefer.

aminya · 2020-08-04T18:49:33Z

Thank you! will check this later today.

script/vsts/get-release-version.js

script/vsts/lib/release-notes.js

aminya · 2020-08-05T04:19:37Z

script/vsts/get-release-version.js

+  ? process.env.NIGHTLY_RELEASE_REPO
+  : 'atom-nightly-releases';

 const yargs = require('yargs');


Is there a reason that MAIN_REPO is missing here? Just want to check if we haven't missed it.

The script authors didn't use "the name of the main repo" anywhere in the script. I suppose they had no need for it.

(Where they use the string 'atom', they mean "the repo owner"; the org/user whose account the [nightly] repo is under.)

And the linter gets upset if you define a variable and don't use it.

https://github.com/atom-ide-community/atom/blob/23ad7fe8575e8419e08ede6e2ebe110e8708e725/script/vsts/get-release-version.js#L32

This is the only place in the file any of these vars is needed.

By the way, from reading that line, it appears if our Nightly repo goes stale, and lacks recent tags, our auto drafts of releases (if we use that feature) will not advance their version number. So we would perhaps have to manually fix the suggested version number?

Something to keep in mind. Some of Atom's release CI steps are very interconnected with the Nightly repo.

aminya · 2020-08-05T04:20:13Z

script/vsts/platforms/templates/cache.yml

    displayName: Cache node_modules
    inputs:
-      key: 'npm | "$(Agent.OS)" | "$(BUILD_ARCH)" | package.json, package-lock.json, script/vsts/platforms/${{ parameters.OS }}.yml'
+      key: 'npm_pr_78 | "$(Agent.OS)" | "$(BUILD_ARCH)" | package.json, package-lock.json, script/vsts/platforms/${{ parameters.OS }}.yml'


What is this?

That should be removed before this would be merged.

That's setting the cache aside for this PR so it doesn't pull anything in from master branch, or even earlier commits of the PR. Nothing else has that cache key value, so there is no existing cache to hit.

I was seeing weird errors, so rather than spend time debugging them, I wanted to see if they went away when cache restoring was disabled.

Next step is to remove that and see if it passes CI when restoring cache from the general pool (e.g. from master) is enabled again.

Please revert this. If there is a problem with caching we need to fix that instead of monkey patching.

aminya · 2020-08-05T04:20:44Z

script/vsts/platforms/templates/preparation.yml

      npm install
    displayName: Install Windows build dependencies
-    condition: eq(variables['Agent.OS'], 'Windows_NT')
+    condition: and(eq(variables['Agent.OS'], 'Windows_NT'), eq(variables['buildArch'], 'x86'))


This is incorrect.

I would like us to consolidate these variables.

buildArch is set and then BUILD_ARCH is set equal to buildArch. As far as I can tell, this is unnecessary; Only variables set from Azure DevOps UI need to be UPPERCASE_WITH_UNDERSCORES. Since these are set in yaml, they may be camelCase.

As far as I can tell:

buildArch and BUILD_ARCH are only set on Windows (x86 and x64)

buildArch and BUILD_ARCH are always available on Windows, whether as Azure DevOps variables, or as environment variables

buildArch and BUILD_ARCH hold the same value at any given moment

Therefore, I'd like us to pick one and settle on it so we can be consistent and make the CI less confusing. No need to have two different variables for the exact same value if one is good enough.

One thing to note: script/vsts/windows-run.js looks for BUILD_ARCH (capital case with underscores), so we need to standardize on that, OR edit the script to look for process.env.buildArch instead of process.env.BUILD_ARCH.

It would be a big priority for me to consolidate these due to the unnecessary complexity and confusion.

And in case that's not what you mean, how is it incorrect?

One of my goals in this PR was to make sure the script/vsts/node_modules dependencies, and windows-run.js, are totally not needed for Windows x64.

(Just like they're not needed on macOS or Linux. It should only be needed on Windows x86, to support windows-run.js -- and even then, probably the native Node API is enough to accomplish what is needed in windows-run.js if we refactor it a bit.)

If I recall correctly, the only thing still not working is the "Upload crash reports" under Windos x64, which I suppose is still a blocker, because I don't think upstream will merge if that's broken.

If need be, I can give up on eliminating windows-run.js on Windows x64. It did turn out to be more complicated than expected.

In that case, we should still switch cd script/vsts && npm install to cd script/vsts && npm ci to be slightly faster. (We don't cache those dependencies, so there is no downside to ci vs install).

If this step is only needed for x86 that makes sense. You should rename the display name and comment as well then to be specific to x86

aminya · 2020-08-05T04:22:06Z

script/vsts/platforms/windows.yml

+  - job: Windows_build
+    displayName: Windows build


Please revert this. Make it Windows to keep it simple. It is still called Windows on upstream.

I was copying the macOS build job label that upstream uses for macOS, where the CI is split up between a "build" job and a "parallel test" job. Just like we introduced for Windows at this fork.

So maybe upstream will eventually take this? I do think they want parallel testing on Windows x64. (I think they might be more likely to take it if it is separated out from the "move lots of stuff to templates" changes. But I'm not them so I can't read their minds about that. It would be most welcome if they would comment on our PR.)

This job is called on other files too like Release job. After all, it is just a name, I don't think it is important. 😄 If you want to change it you change it everywhere.

aminya · 2020-08-05T04:22:27Z

script/vsts/platforms/windows.yml

+    displayName: Windows
+    dependsOn: Windows_build


Please revert this.

script/vsts/upload-artifacts.js

DeeDeeG · 2020-08-06T03:29:11Z

I will take your feedback into consideration and try to decide what to do. Some things are subjective, like job names, but still, good decisions make the CI nicer to use and work with.

Strongly considering moving the "no install script/vsts/node_modules on Windows 64-bit" stuff out into another PR. I'm very close to getting that done, but part of it still needs work (uploading crash reports), it's somewhat experimental, and the rest of the PR is lower risk. Might be good to separate the concerns here.

That said, future me may have a different opinion on this... Just laying out my thoughts while I have enough time to think about this but not act on it just yet.

aminya · 2020-08-12T07:38:34Z

script/vsts/platforms/templates/cache.yml

    displayName: Cache node_modules
    inputs:
-      key: 'npm | "$(Agent.OS)" | "$(BUILD_ARCH)" | package.json, package-lock.json, script/vsts/platforms/${{ parameters.OS }}.yml'
+      key: 'npm_pr_78 | "$(Agent.OS)" | "$(BUILD_ARCH)" | package.json, package-lock.json, script/vsts/platforms/${{ parameters.OS }}.yml'


Please revert this. If there is a problem with caching we need to fix that instead of monkey patching.

aminya

Once you applied the requested changes, I will merge this into our upstream PR (cross-platform CI).

DeeDeeG · 2020-08-12T23:43:43Z

This needs a closer look on the Windows x86 stuff. Last I recall I was going to push that out to a separate PR. I also noticed the "script step inexplicably exits after npm ci" comment is no longer true, so we can delete the comment and combine two adjacent script tasks in a few places. I can leave that for another PR if you don't want it here.

There might be a couple other things I notice here and there that can be tidied up over time even after this PR is merged.

aminya · 2020-08-12T23:57:18Z

I prefer that you revert skipping install windows build deps on x64 and take that to another PR.

DeeDeeG · 2020-08-13T00:03:06Z

So I don't lose them, the Windows x86 related commits I dropped from this PR are:

DeeDeeG · 2020-08-13T00:04:13Z

Getting rid of ALL_CAPS BUILD_ARCH in favor of just consistently using buildArch would be amazing, but I'm going to consider that a to-do for outside of this PR also.

DeeDeeG · 2020-08-13T00:30:39Z

I'd also like to get rid of the explicit Python 3 install in Windows (because it wastes time), instead relying on the already-installed Python versions.

We pinned npm to an older version that can't use Python 3 anyhow, during the Electron 6 PR. 😢 Not a big deal though.

aminya · 2020-08-13T01:04:21Z

I'd also like to get rid of the explicit Python 3 install in Windows (because it wastes time), instead relying on the already-installed Python versions.

This should be a separate PR. Please keep this just a bug fix.

Also make sure "Release" job depends on successful Windows tests, not just on the Windows build job, before/regardless of testing.

We no-longer run the Linux build in an old Ubuntu 14.04 "Trusty Tahr" Docker container. We run builds directly in a virtualized Ubuntu 16.04 "Xenial Xerus" image now.

Using the `env:` yaml key is not actually needed unless the variables contain secret/sensitive information. (These varibales won't have sensitive content, just a repo owner and some repo names). These environment variables would expand to literally `${VAR_NAME}`, rather than to an empty string, if not explicitly set in Azure DevOps. This would make consuming the variables in JavaScript unnecessarily awkward. Instead, we can set reasonable defaults if the variables are falsy (i.e. unset or null in our case), and otherwise override with what's manually set in the environment variables.

REPO_OWNER, MAIN_REPO, and NIGHTLY_RELEASE_REPO now default to 'atom', 'atom' and 'atom-nightly-releases', respectively.

Co-authored-by: Amin Yahyaabadi <[email protected]>

This old bug in the CI inexplicably went away at some point.

DeeDeeG · 2020-08-13T01:57:05Z

Force pushed to update the first commit, where I had missed the Release job of the nightly pipeline yml, dependsOn: windows (should be dependsOn: windows_build now).

aminya · 2020-08-13T02:07:56Z

That's why I said keep the names as before. Now you need to change dependsOn: Windows everywhere.

DeeDeeG · 2020-08-13T02:19:13Z

I fixed it. All of them are properly updated.

Here's a commit (which I haven't added to this PR) that I mocked up to make the Windows yml more similar to the macOS one. If you like it I'll push it here, but I left it out for now.

DeeDeeG@0ba2f4a

I like that it's consistent, so you don't have to memorize two different sets of naming conventions and ordering of the yaml. Reading one shows you mostly how the other one is set up.

Edit to add: The "windows" display name for the Windows tests, as I added in this PR, doesn't make sense. It should at least have "test" in the name. The commit I linked in this comment has "test" in the name for the Windows test job.

See this CI run, with reasonable/human-friendly display names for all the jobs: https://dev.azure.com/DeeDeeG/b/_build/results?buildId=260&view=results

aminya · 2020-08-14T05:19:13Z

Upstreamable commits:

cfd4a0b should be merged to CI: Faster "GetReleaseVersion" "Release", and "Upload Artifacts" jobs atom/atom#21147
I will do the Windows job name changes manually for Parallelize Tests in CI (+ Refactor and Improvements) atom/atom#21109
We have not made a pull request regarding REPO Name stuff. So a separate pull request should be made for that. But I think it is better to make that once Parallelize Tests in CI (+ Refactor and Improvements) atom/atom#21109 is merged to prevent conflicts

DeeDeeG · 2020-08-14T05:46:08Z

cfd4a0b does make GetReleaseVersion fail to print the version it finds sometimes. That must be what the old error message was about.

I have only seen this on the Pull Requests pipeline, where GetReleaseVersion is not important. And I have a suspicion that the script may actually be running and its output just isn't being displayed, as an Azure DevOps bug.

In any case, I have seen no issues in those cases where we can't see any output of GetReleaseVersion. Let's look at that carefully before we submit it to upstream. If there are problems, we may need to revert.

I never saw it skip outputting a version on several runs of the Nightly Release pipeline, nor during the Release Branch Build pipeline. Still something to look out for in case it does happen and I just didn't do it enough times to reproduce it yet.

DeeDeeG · 2020-08-14T05:51:16Z

There's more to do on the REPO vars, so I'd like a chance to do that and we can submit it all in one go?

I opened #83 for this. I don't think it's hard; predominantly just sifting through the code for hard-coded strings and replacing them with the right const assigned value.

aminya added the CI label Aug 4, 2020

DeeDeeG force-pushed the fix-miscellaneous-CI-issues branch 2 times, most recently from f0ab74d to 6235a14 Compare August 4, 2020 21:38