Note (more details on the rationale): to achieve parallelism some developers rely on a master-script using curl to execute worker-scripts via http calls to localhost. This is imho bad practice as it forces the php.ini used for web purpose to have exceedingly high memory/timeout issues. It also keeps web worker processes occupied for a very long time

👍

Well, time to get back from the grave I guess, 3 plusses mean that I have to turn this into a full-fledged Pull Request...

@gggeek maybe you should wait a bit more. Votes from the community are important, but the decisive votes are only the ones from the Symfony Core. So far none of them have voted on this.

@javiereguiluz ok then

I think that's a good idea. Having such a manager would allow to get rid of a lot of boilerplate code you would have to write in your own application otherwise.

You mean this: https://github.com/kriswallsmith/spork ?

@mvrhov this class is not about managing Process instances

maybe this https://github.com/liuggio/fastest/tree/master/src/Process will help you?

About spork: it seems that it can indeed be used to achieve the same goal, or almost.
Not documented clearly in its readme, but in this post: https://blog.vandenbrand.org/2013/03/17/speed-up-your-data-migration-with-spork/

spork uses fork model (which means that only PHP processes can be parallelized), while symfony/process can achieve the pretty same goals with just proc_open. Also it can be used only in unix-based environments.

I currently working on PoC implementation of process manager for symfony/process with some IPC support via pipes (to add concurrency to Behat).

It would be great to have multiple strategies to achieve the goal. Like using pthread, fork, pipe, msg, whatever is available on env. And if there aren't any - fallback to running things in sequence, using same interface.

@keradus the thing is that pthreads, fork, proc_open are for different goals. For example pthreads + coroutine could be used to achieve micro-threads functionality, fork can be used for speed-up workers startup and proc_open for anything else.

Currently there is no way to use symfony/process with long-lived processes. See #14482 for details. After process is started the only way to pass messages to it is selectable streams (php://memory isn't selectable so passing messages to processes will cause additional I/O overhead since we should pass this messages via file system).

I think that what should be done, is simple process manager for allowing to start multiple processes via proc_open. This will cover most of use cases and doesn't require any additional extensions to be used (also this is the only cross platform way to do this).

For example I have this use cases:
I work with some webservice which should generate images via wkhtmltoimage. For one request it generates 10 images. Right now this operation takes ~5 seconds. I could start several instances of wkhtmltoimage in parallel in speed up things. This is possible only via proc_open. You can't do this with just exec, or fork or via pthreads... well you can do it via pthreads but this won't be efficient.

It is tricky, but workers+jobs strategy may be done with threads and processes too.
But I agree, there are big differences, so creating an layer to make them work in similar way is tricky

creating an layer to make them work in similar way is tricky

I think that this doesn't bring much benefits. proc_open covers most of use cases + it available out-of-box for both unix and windows. But process manager totally should be implemented (or process pool).

See #18513, finally

Since this was scheduled for 3.1 which is out already and PR #18776 mentiones it does it mean this issue is resolved?

@martinsik tbh I am not sure. I looked at #18513, and all I can see is changes in the way process IO is handled, which allows to set up chained processes.
It might be beneficial to the development of a Process Manager, but it does not seem that it includes one.

Otoh I have found this extremely simple Process Manager that seems to fit the bill for my original usecase: https://github.com/jagandecapri/symfony-parallel-process

@gggeek Do you want to provide a PR for this ?

To anyone wondering about this request, don't miss the comments in #23596

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?

I'm going to close this issue as it has no activity for 3 years now.
Feel free to reopen or submit a PR if you can.
Thanks