Random failure of the Process tests #9182

fabpot · 2013-10-01T07:54:47Z

Since a while now, the Process test suite randomly fails with error messages like the following one:

1) Symfony\Component\Process\Tests\SigchildDisabledProcessTest::testProcessPipes with data set #8 ('fwrite(STDOUT, $in = file_get_contents(\'php://stdin\')); fwrite(STDERR, $in);', 4096)
Failed asserting that 4145152 matches expected 4194305.
/home/travis/build/symfony/symfony/src/Symfony/Component/Process/Tests/AbstractProcessTest.php:113

ping @romainneutron

The text was updated successfully, but these errors were encountered:

romainneutron · 2013-10-04T13:34:53Z

I actually thought it was just happening on windows platform when using file handles (see https://bugs.php.net/bug.php?id=65650 that produces the same test failure)

I'm gonna dig this, any help would be very appreciated

romainneutron · 2013-10-09T10:53:33Z

Okay, I've run the test suite in 2.2 / 2.3 / master on my computer (OSX) and a desktop ubuntu intensively. I got everytime the same results (many many many tries) :

$ ~ phpunit --filter=testProcessPipes --repeat=200 Tests/SimpleProcessTest.php
PHPUnit 3.8-g55a6dd0 by Sebastian Bergmann.

Configuration read from /Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/phpunit.xml.dist

The Xdebug extension is not loaded. No code coverage will be generated.

.............................................................   61 / 2000 (  3%)
.............................................................  122 / 2000 (  6%)
.............................................................  183 / 2000 (  9%)
.............................................................  244 / 2000 ( 12%)
.............................................................  305 / 2000 ( 15%)
.............................................................  366 / 2000 ( 18%)
.............................................................  427 / 2000 ( 21%)
.............................................................  488 / 2000 ( 24%)
.............................................................  549 / 2000 ( 27%)
.............................................................  610 / 2000 ( 30%)
.............................................................  671 / 2000 ( 33%)
.............................................................  732 / 2000 ( 36%)
.............................................................  793 / 2000 ( 39%)
.............................................................  854 / 2000 ( 42%)
.............................................................  915 / 2000 ( 45%)
.............................................................  976 / 2000 ( 48%)
............................................................. 1037 / 2000 ( 51%)
............................................................. 1098 / 2000 ( 54%)
............................................................. 1159 / 2000 ( 57%)
............................................................. 1220 / 2000 ( 61%)
............................................................. 1281 / 2000 ( 64%)
............................................................. 1342 / 2000 ( 67%)
............................................................. 1403 / 2000 ( 70%)
............................................................. 1464 / 2000 ( 73%)
............................................................. 1525 / 2000 ( 76%)
............................................................. 1586 / 2000 ( 79%)
............................................................. 1647 / 2000 ( 82%)
............................................................. 1708 / 2000 ( 85%)
............................................................. 1769 / 2000 ( 88%)
............................................................. 1830 / 2000 ( 91%)
............................................................. 1891 / 2000 ( 94%)
............................................................. 1952 / 2000 ( 97%)
................................................

Time: 7.46 minutes, Memory: 4531.75Mb

Tests are okay. Please note the amount of memory used (thats another issue).

I started to become desperate when I got these failures :

$ ~ phpunit --filter=testProcessPipes --repeat=200 Tests/SimpleProcessTest.php
PHPUnit 3.8-g55a6dd0 by Sebastian Bergmann.

Configuration read from /Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/phpunit.xml.dist

The Xdebug extension is not loaded. No code coverage will be generated.

.............................................................   61 / 2000 (  3%)
.............................................................  122 / 2000 (  6%)
.............................................................  183 / 2000 (  9%)
.............................................................  244 / 2000 ( 12%)
.............................................................  305 / 2000 ( 15%)
.............................................................  366 / 2000 ( 18%)
.............................................................  427 / 2000 ( 21%)
.............................................................  488 / 2000 ( 24%)
...........................F.................................  549 / 2000 ( 27%)
.............................................................  610 / 2000 ( 30%)
......................F......................................  671 / 2000 ( 33%)
.............................................................  732 / 2000 ( 36%)
.............................................................  793 / 2000 ( 39%)
.............................................................  854 / 2000 ( 42%)
.............................................................  915 / 2000 ( 45%)
.............................................................  976 / 2000 ( 48%)
............................................................. 1037 / 2000 ( 51%)
.........F................................................F.. 1098 / 2000 ( 54%)
.............................F............................... 1159 / 2000 ( 57%)
............................................................. 1220 / 2000 ( 61%)
............................................................. 1281 / 2000 ( 64%)
............................................................. 1342 / 2000 ( 67%)
....................F........................................ 1403 / 2000 ( 70%)
............................................................. 1464 / 2000 ( 73%)
.........................................F..........FF....... 1525 / 2000 ( 76%)
............................................................. 1586 / 2000 ( 79%)
............................................................. 1647 / 2000 ( 82%)
............................................................. 1708 / 2000 ( 85%)
............................................................. 1769 / 2000 ( 88%)
............................................................. 1830 / 2000 ( 91%)
............................................................. 1891 / 2000 ( 94%)
............................................................. 1952 / 2000 ( 97%)
..................................F.............

Time: 13.74 minutes, Memory: 4527.25Mb

There were 10 failures:

1) Symfony\Component\Process\Tests\SimpleProcessTest::testProcessPipes with data set #5 ('include \'/Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/Tests/PipeStdinInStdoutStdErrStreamSelect.php\';', 64)
Failed asserting that 24576 matches expected 65537.

/Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/Tests/AbstractProcessTest.php:116

2) Symfony\Component\Process\Tests\SimpleProcessTest::testProcessPipes with data set #2 ('fwrite(STDOUT, $in = file_get_contents(\'php://stdin\')); fwrite(STDERR, $in);', 16)
Failed asserting that 16384 matches expected 16385.

/Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/Tests/AbstractProcessTest.php:116

3) Symfony\Component\Process\Tests\SimpleProcessTest::testProcessPipes with data set #6 ('fwrite(STDOUT, $in = file_get_contents(\'php://stdin\')); fwrite(STDERR, $in);', 1024)
Failed asserting that 1032192 matches expected 1048577.

/Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/Tests/AbstractProcessTest.php:117

4) Symfony\Component\Process\Tests\SimpleProcessTest::testProcessPipes with data set #5 ('include \'/Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/Tests/PipeStdinInStdoutStdErrStreamSelect.php\';', 64)
Failed asserting that 16384 matches expected 65537.

/Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/Tests/AbstractProcessTest.php:116

5) Symfony\Component\Process\Tests\SimpleProcessTest::testProcessPipes with data set #7 ('include \'/Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/Tests/PipeStdinInStdoutStdErrStreamSelect.php\';', 1024)
Failed asserting that 999424 matches expected 1048577.

/Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/Tests/AbstractProcessTest.php:116

6) Symfony\Component\Process\Tests\SimpleProcessTest::testProcessPipes with data set #2 ('fwrite(STDOUT, $in = file_get_contents(\'php://stdin\')); fwrite(STDERR, $in);', 16)
Failed asserting that 8192 matches expected 16385.

/Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/Tests/AbstractProcessTest.php:116

7) Symfony\Component\Process\Tests\SimpleProcessTest::testProcessPipes with data set #5 ('include \'/Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/Tests/PipeStdinInStdoutStdErrStreamSelect.php\';', 64)
Failed asserting that 65536 matches expected 65537.

/Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/Tests/AbstractProcessTest.php:116

8) Symfony\Component\Process\Tests\SimpleProcessTest::testProcessPipes with data set #6 ('fwrite(STDOUT, $in = file_get_contents(\'php://stdin\')); fwrite(STDERR, $in);', 1024)
Failed asserting that 1032192 matches expected 1048577.

/Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/Tests/AbstractProcessTest.php:117

9) Symfony\Component\Process\Tests\SimpleProcessTest::testProcessPipes with data set #7 ('include \'/Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/Tests/PipeStdinInStdoutStdErrStreamSelect.php\';', 1024)
Failed asserting that 1015808 matches expected 1048577.

/Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/Tests/AbstractProcessTest.php:117

10) Symfony\Component\Process\Tests\SimpleProcessTest::testProcessPipes with data set #6 ('fwrite(STDOUT, $in = file_get_contents(\'php://stdin\')); fwrite(STDERR, $in);', 1024)
Failed asserting that 1048576 matches expected 1048577.

/Users/romain/Documents/workspace/symfony/src/Symfony/Component/Process/Tests/AbstractProcessTest.php:117

FAILURES!                                   
Tests: 2000, Assertions: 3994, Failures: 10.

Yes, I got it. The issue occurs when the cpu is under heavy load. The bug can be reproduce by running the test suite with the mentioned arguments --filter=testProcessPipes --repeat=200 Tests/SimpleProcessTest.php and running a composer update on a huge composer.json file and dev stability :)

I'm continuing to investigate

cmerot · 2013-10-09T14:11:51Z

I have the same random bug :(
On my computer this code almost always fails (it's ok after 10 retries). If that can help...

<?php
require 'vendor/autoload.php';
use Symfony\Component\Process\Process;

$cmd = "git show 'master':composer.json";
$cwd = "/home/co/apps/my-repo-with-a-big-composer.json/";
$process = new Process($cmd, $cwd);
$process->run();
print $process->getOutput();

romainneutron · 2013-10-09T15:26:13Z

I got a pending fix, linux only. Currently working on the windows compatibility

…mainneutron) This PR was merged into the 2.2 branch. Discussion ---------- [Process][2.2] Fix 9182 : random failure on pipes tests | Q | A | ------------- | --- | Bug fix? | yes | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | #9182 | License | MIT I'm not a big fan of this fix, but - at least - it works. With this code, finally, Process does not behave the same at all on Windows and Linux. This patch does not smells very good but it solves the random failing test issue (that produced at runtime too). Actually, calling `proc_get_status` within the waiting loop introduced the bug. So this PR reverts to the previous behavior (consider a process running as long as pipes give data). On Windows, this is not the same behavior as we're not using streams but file handles. Whereas the feof of a stream is detected when the other side closes, the feof of a file handle can be reached at any time. So, on Windows, `proc_get_status` is called (checking the feof of the file handle might be positive until the executable outputs something), and we consider a process running as long as the information returned says it's running. We could think of decouple windows and linux logic in two separated objects, using the interfaces I introduced in #8753. This could bring much more readability and make the code more easy to understand. Commits ------- 64a0b40 [Process] Fix #9182 : random failure on pipes tests

* 2.2: [Process] Fix #9182 : random failure on pipes tests Fixed propel guessed relations [FramworkBundle][HttpKernel] Check event listener services are not abstract fixed CS Check for lock existence before unlinking [FrameworkBundle] fixed path replacement on Windows Conflicts: src/Symfony/Component/Process/Process.php

* 2.3: [Process] Fix #9182 : random failure on pipes tests Fixed propel guessed relations [FramworkBundle][HttpKernel] Check event listener services are not abstract fixed CS Check for lock existence before unlinking remove MinCount and MaxCount contraints. It has been replaced by Count constraints. [FrameworkBundle] fixed path replacement on Windows Conflicts: src/Symfony/Bridge/Propel1/Tests/Form/PropelTypeGuesserTest.php

fabpot · 2013-10-09T21:08:16Z

apparently, I messed up the merge to the 2.3 branch.

romainneutron · 2013-10-09T21:09:21Z

If you have few min (maybe an hour) I'll take care of it

This PR was merged into the 2.3 branch. Discussion ---------- [Process] Fix latest merge from 2.2 in 2.3 | Q | A | ------------- | --- | Bug fix? | yes | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | yes (Process tests are okay) | License | MIT Fix latest merge from 2.2 in 2.3, see #9182 Process tests are okay, an error in HttpKernel seems to occur Commits ------- b5e3576 [Process] Fix latest merge from 2.2 in 2.3

Update the symfony process library which causes missing branches in satis due to this issue - symfony/symfony#9182

halaei · 2020-05-09T15:58:02Z

Hi there. I think the related commit to this fix, 64a0b40, has something to do with this issue:

Process AbstractPipes.php infinite fwrite() loop on error #21580

Maybe $this->isRunning() should be called for Linux inside wait() function as well?

romainneutron added a commit to romainneutron/symfony that referenced this issue Oct 9, 2013

[Process] Fix symfony#9182 : random failure on pipes tests

64a0b40

romainneutron mentioned this issue Oct 9, 2013

[Process][2.2] Fix 9182 : random failure on pipes tests #9257

Merged

fabpot closed this as completed Oct 9, 2013

romainneutron mentioned this issue Oct 9, 2013

[Process] Fix latest merge from 2.2 in 2.3 #9259

Merged

This was referenced Oct 10, 2013

Update symfony process library composer/satis#93

Merged

Update symfony process library composer/composer#2315

Closed

GCth mentioned this issue Oct 10, 2013

Exception on sp:bower:install after updating symfony/symfony from 2.3.4 to 2.3.5 Spea/SpBowerBundle#46

Closed

romainneutron mentioned this issue Oct 19, 2013

[Process] Output fetched on windows platform might be corrupted #9343

Closed

parsingphase pushed a commit to parsingphase/satis that referenced this issue Nov 1, 2013

Update symfony process library

cf688dd

Update the symfony process library which causes missing branches in satis due to this issue - symfony/symfony#9182

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Random failure of the Process tests #9182

Random failure of the Process tests #9182

fabpot commented Oct 1, 2013

romainneutron commented Oct 4, 2013

romainneutron commented Oct 9, 2013

cmerot commented Oct 9, 2013

romainneutron commented Oct 9, 2013

fabpot commented Oct 9, 2013

romainneutron commented Oct 9, 2013

halaei commented May 9, 2020

Random failure of the Process tests #9182

Random failure of the Process tests #9182

Comments

fabpot commented Oct 1, 2013

romainneutron commented Oct 4, 2013

romainneutron commented Oct 9, 2013

cmerot commented Oct 9, 2013

romainneutron commented Oct 9, 2013

fabpot commented Oct 9, 2013

romainneutron commented Oct 9, 2013

halaei commented May 9, 2020