Thanks to visit codestin.com
Credit goes to github.com

Skip to content

module-aware building #855

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
hajimehoshi opened this issue Aug 26, 2018 · 42 comments Β· Fixed by #1042
Closed

module-aware building #855

hajimehoshi opened this issue Aug 26, 2018 · 42 comments Β· Fixed by #1042
Assignees
Labels
enhancement gopherjs-tool Related to the gopherjs tool or its build system (but not the compiler itself).

Comments

@hajimehoshi
Copy link
Member

hajimehoshi commented Aug 26, 2018

Now Go 1.11 introduces module-get, I suggest to change gopherjs build or other commands to be module-aware. For example, I'd want the below code to be workable:

mkdir foo
cd foo
go mod init example.com/m
go get github.com/gopherjs/gopherjs
gopherjs build -tags=example github.com/hajimehoshi/ebiten/examples/sprites@master
@myitcv
Copy link
Member

myitcv commented Sep 7, 2018

FWIW I have module-aware building in branch go1.11 of my fork at https://github.com/myitcv/gopherjs

@dmitshur dmitshur added enhancement gopherjs-tool Related to the gopherjs tool or its build system (but not the compiler itself). labels Sep 9, 2018
@hajimehoshi
Copy link
Member Author

@myitcv I discussed with @dmitshur , and we agreed that it is fine to merge your module-aware building. I'd be happy if you could make a PR. Thanks!

@myitcv
Copy link
Member

myitcv commented Oct 26, 2018

@hajimehoshi back in April I raised #799. As a follow up to that, and to discuss my open PRs, @dmitshur and I had a call in which Dmitri said that his plan was to "sunset" GopherJS with the arrival of experimental WASM support in Go 1.11. Hence he did not want to grant access to additional people or accept any more PRs, other than a PR he was working on for Go 1.11 support (and potentially in other exceptional circumstances). Hence Dmitri's suggestion was to do things in a fork, which is how https://github.com/myitcv/gopherjs came about.

Has that position now changed?

As I suggested to Dmitri on our original call, perhaps we could also get some clarification on the current position (with respect to the https://github.com/myitcv/gopherjs fork) added to the main README so anyone else considering contributing a PR is clear.

Thanks

@hajimehoshi
Copy link
Member Author

Has that position now changed?

Thank you for elaborating and sorry for my lack of explanation. This position is not changed basically, but I asked @dmitshur that I am eager to have module support of GopherJS. @dmitshur permitted me that under the condition of my review and taking care of. But right, this change seemed to be conflicting with the current policy of GopherJS. We need to be on the same page.

IMHO, GopherJS is still useful for a while even after Wasm comes since Wasm still has several problems (e.g. OOM at mobile browsers golang/go#27462, bigger binary size than GopherJS, etc.) so Wasm cannot become a complete substitute of GopherJS so far. Actually I cannot finish to switch to Wasm for my libs yet.

As I suggested to Dmitri on our original call, perhaps we could also get some clarification on the current position (with respect to the https://github.com/myitcv/gopherjs fork) added to the main README so anyone else considering contributing a PR is clear.

+1.

@myitcv
Copy link
Member

myitcv commented Oct 28, 2018

This position is not changed basically

Thanks for clarifying @hajimehoshi. Personally I think it's a bit of a shame that various of my own and indeed a number of others' contributions have been left hanging for so long.

In #799 (back in April) I offered to help in a more formal capacity on the back of a number of issues, PRs and Slack contributions I had worked on/made in the past. Again I find it a bit unfortunate that an offer like this has been ignored and kicked into the long grass.

IMHO, GopherJS is still useful for a while even after Wasm comes since Wasm still has several problems (e.g. OOM at mobile browsers golang/go#27462, bigger binary size than GopherJS, etc.) so Wasm cannot become a complete substitute of GopherJS so far. Actually I cannot finish to switch to Wasm for my libs yet.

This was and remains my position too. But Dmitri wanted to "sunset" GopherJS. Hence he did not want to grant access to additional people or accept any more PRs, other than a PR he was working on for Go 1.11 support (and potentially in other exceptional circumstances). Hence he suggested that I work in a fork.

Therefore, if it's all the same with you, I will continue to offer what module support I can in my fork.

@mvdan
Copy link

mvdan commented Nov 5, 2018

I think some action needs to happen here. Option A is to not stop project development and welcome new (old?) contributors.

Option B is to actually bring the project to a halt, making that clear in the README and linking to active forks that may be interesting to the reader. The same would apply if the project was kept in maintenance mode, only to add support for Go 1.12, 1.13, etc.

I think leaving the project in this limbo state is unhealthy for both developers and users. Developers have no idea if they should send PRs or otherwise contribute to the project, and users are left requesting features in the issue tracker when it's very unlikely any of the developers will read them.

Please note that this is not a demand nor going against anyone in particular. I just wish that working on and with GopherJS wasn't this difficult.

@flimzy
Copy link
Member

flimzy commented Nov 5, 2018

Option A is to not stop project development and welcome new (old?) contributors.

Is this somehow different than what we have at the moment?

I don't see a problem to be solved, or a proposed solution here.

@mvdan
Copy link

mvdan commented Nov 5, 2018

Paul's links above are a good example: #799 https://github.com/gopherjs/gopherjs/pulls/myitcv

Solution A is to approve #799, let the PRs that make sense be merged, and close the others. As the project stands now, it has over 20 open PRs, most of which have had zero attention in the past year. To me, that says that sending a PR to this project is just not worth anyone's time.

I can understand if current maintainers don't have the time to keep up with incoming issues and PRs. This is precisely why new maintainers should be welcome on board, as long as design decisions are still made with consensus.

@dmitshur
Copy link
Member

dmitshur commented Nov 7, 2018

I think leaving the project in this limbo state is unhealthy for both developers and users. Developers have no idea if they should send PRs or otherwise contribute to the project, and users are left requesting features in the issue tracker when it's very unlikely any of the developers will read them.

This is true, and very unfortunate.

It has taken a lot of my time maintaining and developing GopherJS the last year, when I was on a sabbatical and working on open source full time. I now have a full time job (which involved going through the process of relocation), and have much less free time to spend on contributing to GopherJS.

I haven’t done a great job of finding a way for this open-source project to continue to be healthy.

I think some action needs to happen here.

I agree. I’m doing my best to figure out how to move forward. I’ve made some progress recently, and I’ll share updates soon. I just wanted to say I’m fully aware of the problem and am trying to resolve it.

@hajimehoshi
Copy link
Member Author

I agree. I’m doing my best to figure out how to move forward. I’ve made some progress recently, and I’ll share updates soon. I just wanted to say I’m fully aware of the problem and am trying to resolve it.

Now @myitcv has permission to modify this repository as we agreed. Would it be possible to update the policy now? IMO merging @myitcv's fork to this repository would be the best since it includes not only module-aware feature but also bug fixes...

@myitcv
Copy link
Member

myitcv commented Nov 28, 2018

@hajimehoshi I think it's a sensible goal to try and merge the module changes back to this repo, yes. I've firmed up the module support in my fork with a fairly extensive suite of tests covering both module mode and GOPATH mode. So I think a sensible next step would be to create a number of PRs in this repo to bring this repo up-to-date.

@myitcv
Copy link
Member

myitcv commented Dec 16, 2018

@dmitshur - per our discussion, I'm holding off creating any PRs etc until you raise your issue about the future of GopherJS. Do you still plan to raise that issue?

@hajimehoshi
Copy link
Member Author

What's going on this? @dmitshur

@dmitshur
Copy link
Member

dmitshur commented Jan 6, 2019

I do plan to raise that issue. I’ve already written a draft and got Paul’s initial feedback on it. What’s left for me to do is work through the feedback and make the issue. Paul has suggested I also share the draft with a few more active GopherJS contributors. I will do that too.

@myitcv
Copy link
Member

myitcv commented Jan 20, 2019

@dmitshur - any update on this?

I think given concerns/comments like hexops/vecty#215 (comment) it's important to clarify where things stand (cc @slimsag). The team at 1Password will also be keen to see a statement on plans.

When we last spoke you said you wanted to get the issue about the future of GopherJS raised before creating/merging and PRs from https://github.com/myitcv/gopherjs back into this main repo. Given the time that has elapsed since then, does it make sense to remove that constraint and get started on that work, as an indication of the future direction of the project?

@dmitshur
Copy link
Member

dmitshur commented Jan 20, 2019

@dmitshur - any update on this?

I have an update. I'm working on getting it done during this long weekend.

When we last spoke you said you wanted to get the issue about the future of GopherJS raised before creating/merging and PRs from https://github.com/myitcv/gopherjs back into this main repo.

I thought we agreed that doing that was a good plan, but it wasn't me who said that.

Given the time that has elapsed since then, does it make sense to remove that constraint and get started on that work, as an indication of the future direction of the project?

I will be spending time on this today, but if we want to get the issue reviewed by additional people before posting it, that may take additional time that's out of my hands. In general, I'd prefer to post it sooner in a draft form to get the discussion started, rather than trying to perfect it before posting.

In any case, I don't think the issue should block maintenance work that is definitely in scope, such as modules and Go 1.12 support, from being started. The goal of the issue being made is to help the project, not impede progress.

@hajimehoshi
Copy link
Member Author

hajimehoshi commented Jan 20, 2019

When we last spoke you said you wanted to get the issue about the future of GopherJS raised before creating/merging and PRs from https://github.com/myitcv/gopherjs back into this main repo.

I thought we agreed that doing that was a good plan, but it wasn't me who said that.

I think I might have said, and actually we have chatted that the direction would be good in private rooms (e.g. DM at Slack). However, all we (@dmitshur, @myitcv and me) have not made a clear and public decision as far as I understand.

@myitcv
Copy link
Member

myitcv commented Jan 20, 2019

I thought we agreed that doing that was a good plan, but it wasn't me who said that.

Agreed, we both said it was a good plan. Apologies, I must be misremembering whether it was you or me who suggested it. The point however is that we agreed the issue would come first (before/after your blog post), that you would post that issue (I sent feedback on your draft on 15 Nov 2018) then PRs for modules support, etc would follow.

The only point I'm making is that, depending on when you think you'll be in a position to raise that issue, it might make sense to try and get those PRs kicked off now, rather than waiting any longer.

@dmitshur
Copy link
Member

dmitshur commented Jan 20, 2019

The only point I'm making is that, depending on when you think you'll be in a position to raise that issue, it might make sense to try and get those PRs kicked off now, rather than waiting any longer.

Makes sense.

I hope for the issue to be posted by the end of this weekend. If it isn't, then I think it should no longer block PRs. Given that it might be posted very soon, it's probably okay to wait 1-2 more days, but no longer.

@paralin
Copy link
Contributor

paralin commented Jan 31, 2019

@myitcv In your branch, why did you have to change the signature of NewSession, and hide the code to generate the imports argument in a private func next to the tool? Before I could invoke GopherJS without importing any of the tool code.

@myitcv
Copy link
Member

myitcv commented Jan 31, 2019

@paralin - the focus of the changes in my fork has been to support modules where GopherJS is used as a CLI. Module support in this form requires using the go tool under the cover, hence there have been a number of changes to github.com/gopherjs/gopherjs/build.

I'm guessing you're looking to use github.com/gopherjs/gopherjs/build as a library? That case I have not covered thus far.

What we could look to do is configure github.com/gopherjs/gopherjs/build to work with a driver of some sort, the go tool would be one implementation. But I haven't spent any time looking at that.

@paralin
Copy link
Contributor

paralin commented Feb 1, 2019

@myitcv Why do you accept imports and the other parameter as new params to build, and put most of the modules logic and go tool logic in the "main" package? I don't see why you couldn't just put all of that code in "build", or not quite as good but better, put it in some other package than main so that I can import it and replicate what happens in the CLI.

@myitcv
Copy link
Member

myitcv commented Feb 1, 2019

@paralin - the implementation in my first cut was a rough first cut. It's highly likely there are bits that need improving/fixing etc. Shall we move discussion to an issue in https://github.com/myitcv/gopherjs if you'd like to discuss further?

@paralin
Copy link
Contributor

paralin commented Feb 1, 2019

@myitcv Certainly, apologies for being a pain over rough draft code. It's just I did attempt to switch to your branch and failed because of the code namespacing :)

@myitcv
Copy link
Member

myitcv commented Feb 1, 2019

failed because of the code namespacing

Can you clarify what you mean here?

emidoots pushed a commit to hexops/vecty that referenced this issue Jul 1, 2019
Using blackfriday v2 with GopherJS and WebAssembly is not currently easily possible because
with Go modules v2 must be imported as:

```
github.com/russross/blackfriday/v2
```

But `gopherjs serve` cannot use that versioned import path, since it is not module-ware ([open issue](gopherjs/gopherjs#855)).

So we end up stuck in a catch-22: Go 1.12 WebAssembly can't build `gopkg.in/russross/blackfriday.v2` anymore
(import path must be suffixed with `/v2`) and GopherJS cannot build `github.com/russross/blackfriday/v2`.

Initially I just tried downgrading to blackfriday v1 at `github.com/russross/blackfriday` but unfortunately
that appears to have a different problem where module-aware builders like Go 1.12 are forced to use v2 but
non-module-aware work fine, I suspect due to the `go.mod` comitted to that repo currently.

This works for now, and the long term fix is GopherJS getting module-aware building.
@leaanthony
Copy link

@myitcv just wondering what the status of this is. Thanks!

@myitcv
Copy link
Member

myitcv commented Jul 8, 2019

@leaanthony - no update unfortunately. I haven't had a chance to port the changes from https://github.com/myitcv/gopherjs, so https://github.com/myitcv/gopherjs remains the most current version of gopherjs that supports modules.

@leaanthony
Copy link

Thanks for the update πŸ‘

@nathanhack
Copy link

Hi @myitcv, I was just wondering if there's any update on this?

@myitcv
Copy link
Member

myitcv commented Sep 3, 2019

@nathanhack - unfortunately not, the current update is still per #855 (comment)

@nathanhack
Copy link

No worries, thanks for letting me know!

@flimzy
Copy link
Member

flimzy commented Feb 15, 2020

I have found that using the standard Go toolchain's module support to handle vendoring my GopherJS dependencies isn't very difficult. I decided to create a brief tutorial on the topic, as supplementary documentation for one of my open-source libraries with GopherJS support, but others may find it valuable, too.

https://jhall.io/posts/gopherjs-with-modules/

@nevkontakte
Copy link
Member

I did some research on this and there are some good and bad findings:

  • The good: go/build actually gained basic modules support, although golang.org/x/tools/go/packages remains the recommended option going forward.
  • The bad: with either package, module support hinges on the ability to run go list, so go/build module support is automatically disabled if we try to override file system access. The other issue is that build context ReleaseTags must match that of the default build context, but that is a lesser issue.

We could potentially solve that by reimplementing missing bits of module support ourselves (following in the footsteps of what @visualfc did in their fork), but this is additional maintenance burden I'm not looking forward to. The alternative is to simplify our build process by separately materializing our augmented GOROOT on disk and then either go/packages or go/build will work for us no problem. I've created #1021 to solicit some feedback on the latter option.

@paralin
Copy link
Contributor

paralin commented Apr 13, 2021

Right now I make a fake GOROOT and symlink the project root into it, and then "go mod vendor" to ensure vendor/ is there, and it works.

@nevkontakte
Copy link
Member

Yeah, that would work, but I think we'd all agree this is by no means the ideal workflow :) Now that I'm done with Go 1.16 support, I'm looking to streamline module support, especially considering that Go 1.17 was going to drop GOPATH support.

@benc-uk
Copy link

benc-uk commented May 7, 2021

Can someone post step by step what the latest workaround is ?

Kinda shocking this has been open for 3 years

@flimzy
Copy link
Member

flimzy commented May 7, 2021

Can someone post step by step what the latest workaround is ?

Kinda shocking this has been open for 3 years

Please see my link in the comments above. It provides step by step directions.

Also keep in mind that this is entirety a volunteer project, with very few active contributors. If you're eager to see this issue addressed, we would gladly welcome any PRs to that effect.

@benc-uk
Copy link

benc-uk commented May 7, 2021

Thanks @flimzy I've followed the steps to the letter

Installed the 1.12.3 version of GopherJS and Go 1.12 and I get errors about cannot find package "github.com/agext/levenshtein"
I've tried setting GOROOT, vendoring everything, but no luck :(

EDIT: FIXED IT, moving my project into GOPATH/src seemed to work for me,
I'm unable to reproduce the steps but I have a index.js built so I can move on

@nevkontakte
Copy link
Member

Hi folks!

I've opened #1042 which brings proper Modules support to GopherJS. However, since Modules are complicated and full of gremlins, I would appreciate if people could try using the version from the PR on their projects and reporting back any errors encountered. Reporting successes would be also nice πŸ™‚

@paralin
Copy link
Contributor

paralin commented Jul 18, 2021

@nevkontakte thanks, will test it out :)

@flimzy flimzy mentioned this issue Jul 23, 2021
14 tasks
@nevkontakte nevkontakte unpinned this issue Sep 18, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement gopherjs-tool Related to the gopherjs tool or its build system (but not the compiler itself).
Projects
None yet
Development

Successfully merging a pull request may close this issue.

10 participants