Separate Parsing and Compiling #1361
Conversation
1a4dcc2 to
963a05a
Compare
4f862b6 to
a594ec1
Compare
There was a problem hiding this comment.
Read through it all, and it looks good to me. But happy to have @nevkontakte look at it too.
|
I was continuing onto the compile break up and realized that However, if you'd like, I could just make that change in this PR instead of having another one. |
There was a problem hiding this comment.
Not a specific request, but something to consider and/or use as a reference:
https://pkg.go.dev/golang.org/x/tools/go/packages#Load both loads package metadata and (optionally) parses AST files there too. So, at a first glance, adding parsed sources to PackageData is not a bad idea.
More broadly, I suspect that we could simplify a lot of what build package does today by leaning more on go/packages. My idealized build process would look something like:
- Use https://pkg.go.dev/golang.org/x/tools/go/packages#Load to load the entire program. Use the Confg.ParseFile hook to augment standard library on the fly [1].
- Perform GopherJS-specific analysis.
- Perform compilation.
I think a lot of custom BuildContext stuff would simply disappear after that. We don't have to do that right now, but I thought I'd mention it in case it turns out to be a more expedient way to achieve what we need to support generics.
[1]: Looks like this Config.ParseFile hook is new? I don't remember seeing it before, anyway. Its existence may make #1021 obsolete, or at least not a requirement for GopherJS to adopt go/packages.
|
@nevkontakte (in response to this comment), I had a WIP (this closed PR) where I was attempting to use golang.org/x/tools/go/packages#Load but was having trouble figuring out how to perform the augmentation during loading so that we could have The main issue I was having seemed to be that Another thing I wanted to try, if I couldn't figure out the above, was to just load the packages as if not in GopherJS with Anyway, because I was having issues getting it to work, I decided to try to finish that later, if no one else has figured it out. However, I feel you are right, a lot of the Config stuff and stuff getting up to the |
|
@grantnelson-wf thanks for the details! That all sounds reasonable. If I have some spare time one of the weekends, I may try to experiment with it myself. Perhaps I will be able to find a working solution, or perhaps prove definitively that it's not viable :) |
This is the start of several changes to get generic information propagated across package lines.
This changes how packages are loaded so that they are not converted into
Archives right away. This does mean thatArchivecaching has been disabled. With theArchives creation delayed, we have two phases:BuildPackages to determine the files that are needed to be parsed for a package and determine the imports that are required for a package. The result of the parse phase will beParsedPackages. All theParsedPackages needed for a project will be gotten prior to moving onto the compile phase.ParsedPackageand start compiling it, compiling any importedParsedPackageas it goes. Each compile of aParsedPackageresults in anArchive. Once theArchives have all been created, GopherJS continues as usual.The point of this change is to prepare for following PR's. In the following PR's we can break up phase 2 into smaller steps. Right now the compile phase does Type Checking, code simplifying, link finding, anaylizing, then converting into JS. The analyzing includes determining flattening, blocking, etc information for the methods. At the point I left the
TODObetween phase 1 and 2. We can move several of those parts out of compile and do them to the whole project at once. We can run type checking and analyzing all at once to help propagate information across package lines. Once all the preliminary information has been gathered, then we output theArchives.This is related to #1013 and #1270