More strict native override checking

Problem

Currently, anything in the native overrides, without a directive saying otherwise, overrides anything that has the same identifiers in the same package. The Go repo has strict rules about when an how exported types, variable, const, methods, etc are modified, however, non-exported methods may be changed each revision. Since many of the overrides are for non-exported methods, it would be helpful to write warnings when native overrides are out-of-date. This issue proposes a new set of directives that will help contributors keep track of changes in Go.

This proposed change would be most helpful when bumping the versions of GopherJS to a new version of Go. The goal is to help save time and produce better results when doing version bumps. This change would also help a little when doing updates within a revision for any update which changes the native overrides. This change will also help contributors know what the prior contributor intended to be overriding code verse helper code.

The drawback to this change is we'd have to go through all the native overrides and apply compiler directives. This cost should be a onetime effort to implement the strict checking and very minimum overhead otherwise.

Proposed Solution

I propose a tool that can be run on demand that walks through every package in the Go repo and runs the build process on it. This will allow us to check every package in the native overrides for GopherJS repo. The builder should have a strictOverrides flag added to it that defaults to false. The warnings described below will be outputted when the strictOverrides flag is set to true. When this new tool is run, strictOverrides will be set to true, so that these new warnings are only outputted when a contributor is working on a version bump. This tool could also be run as a GHA CI job.

A warning should be outputted for any packages found in the overrides which does not exist in the Go repo. This indicates a package was deleted or moved and that override code should also be deleted or moved. The warning could be something like "warning: the package at [package path] no longer exists in the Go repo".

To make this work, we'll need two new directives: replace and new. These names are just preliminary names for this proposal. These two directives output warnings (or errors) when specific conditions aren't met. Since a whole package may be moved, deleted, refactored, etc. we should only output the first N (probably 5-10) warnings from any package.

replace directive

The replace directive (//gopherjs:replace) would tell the builder that it must find a matching identifier in the same package that is being overridden, otherwise a warning should be outputted. The replace directive checks for overridden code being renamed, migrated, or removed from the code base. The outputted warning could be something like "warning: the native override for [identifier] in [package path] was not found: [override file]:[lineNo]".

Additionally, any overrides of exported types, variables, constants, functions, etc could be checked to ensure the signature (type params, params, returns) match. If the signatures don't match, a warning should be outputted. The warning could be something like "warning: the native override for [identifier] in [package path] has a different signature than found in [native file]:[lineNo]: [override file]:[lineNo]".

Lastly, the existing override-signature, purge, and keepOriginal should also check that a native code exists with the same identifier. If no matching identifier is found, then a warning should be outputted. The outputted warning could be the same as for an replace.

new directive

The new directive (//gopherjs:new) would tell the builder that the given methods must not find a matching identifier in the same package it is override, otherwise a warning should be outputted. The new directive checks for code which was not intended to be overriding code suddenly starts overriding code. This could happen something is renamed, migrated, or added that just so happens to have the same name as a helper function in the overrides. The outputted warning could be something like "warning: non-overriding code in the native overrides for [identifier] in [package path] matches native code in [native file]:[lineNo]: [override file]:[lineNo]".

Additionally, any new code could be checked to ensure that it isn't exported when not in an internal folder. If it is exported, then a warning is outputted. The outputted warning could be something like "warning: non-overriding code for [identifier] in [package path] is exported: [override file]:[lineNo]". If we run into cases were we want to export something in packages to be used by another package, it should be in an internal folder. If we need to allow more new code than that we could add a new-exported directive which acts like new but skips this check.

Lastly, we could check for new code that isn't used in a package. Any new code should be used somewhere in the override code. However, checking for used/unused code is not be easy so this could always be added later if we find we need it. This is to remove any left over code that was being used by an override, the override was updated to not use the new code, but the new code was not removed. This could output a warning like "warning: the non-overriding code in the native overrides for [identifier] in [package path] is unused: [override file]:[lineNo]".

Undefined overrides

With these new directives all code in the native overrides (except for imports) should have a directive on it. If it doesn't then the override is considered an "undefined" type of override. An undefined type will continue to function as overrides do today, where if there is a matching identifier, it replaces that code, otherwise it is considered new. When the other directives have been added to nearly all of the overrides, a warning can be added to indicate any undefined overrides. An undefined override message could be something like "warning: no specified override directive on [identifier] in [package path]: [override file]:[lineNo]"

Proposed Implementation Steps

1. Add the strictOverrides flag to the builder.
2. Create a tool that runs the build overrides for every package in the Go repo with the strictOverrides flag set to true.
3. Add the replace and new directives to astutil.go and pragma.md
4. Update the overrideInfo struct in builder to add states and metadata to the overrides. The states indicates if the identifier for the override was found or not. The metadata could indicate undefined, purge, keepOriginal, override-signature, new, or replace. Also the metadata could store the token.Pos for an override.
5. Update the builder to check for new being matched and (if the strict flag is set) output warning when it does.
6. Update the builder to check for replace, keepOriginal, purge, and override-signature not being matched and (if the strict flag is set) output warnings when they aren't.
7. Update the builder to check for any undefined overrides and output warnings for any that exist.
8. This step will probably take several PRs to finish. Add override directives to the native overrides until there are no more warnings. This may include deleting unused packages and unused new code.

This is just something I've been thinking about during the last two version bumps but I haven't determined how beneficial it would be. After seeing the changes in #1327, where a package was moved and the only indication was slower tests, I decided it was something I should propose. If we plan on doing this work, I would be willing to do most of the work implementing it, when I have the time. Also, feel free to update the proposal if you see spelling and grammar mistakes, since I tend to make a bunch of them.