-
Notifications
You must be signed in to change notification settings - Fork 11.8k
Bug: Random compilation error when using multiple jobs (make -j8) #8776
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Comments
Nice sleuthing -- this sounds like a very plausible cause! |
I've tried to reproduce on two systems (Ubuntu 20.04 and OSX) and thus far I'm unable to reproduce on either. Best I can tell, I'm using the same version of make and gcc that you are.
That said, if you're willing to test, I've opened up #8781 if you're able to reproduce it more consistently than me. Thank you! |
This may be hard to reproduce, as it depends on many factors like CPU speed, IO performance etc. But when I added more "legacy" targets compiled from |
* Fix potential race condition as pointed out by @fairydreaming in #8776 * Reference the .o rather than rebuilding every time. * Adding in CXXFLAGS and LDFLAGS * Removing unnecessary linker flags.
* Fix potential race condition as pointed out by @fairydreaming in ggml-org#8776 * Reference the .o rather than rebuilding every time. * Adding in CXXFLAGS and LDFLAGS * Removing unnecessary linker flags.
* Fix potential race condition as pointed out by @fairydreaming in ggml-org#8776 * Reference the .o rather than rebuilding every time. * Adding in CXXFLAGS and LDFLAGS * Removing unnecessary linker flags.
What happened?
llama.cpp source code freshly cloned from github fails to compile when multiple jobs are used (
make -j8
).This happened on Ubuntu 22.04 with GNU Make 4.3 and gcc (Ubuntu 11.4.0-1ubuntu1~22.04) 11.4.0
It does not happen deterministically. Running
make -j8
a second time results in a correct compilation.My theory is that since legacy binaries
main quantize perplexity embedding server
are all compiled from the same source fileexamples/deprecation-warning/deprecation-warning.cpp
then during compilation with multiple jobs used they may overwrite each other deprecation-warning.o object file resulting in random compilation failures.I think it would be better to create a single target that compiles
examples/deprecation-warning/deprecation-warning.cpp
and use the resulting object file to create legacy binaries.Name and Version
Most recent commit in the cloned source code tree is 7e72aa7
What operating system are you seeing the problem on?
Linux
Relevant log output
The text was updated successfully, but these errors were encountered: