Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Symfony\Component\Stopwatch\Tests\StopwatchEventTest::testEnsureStopped is volatile #14444

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
stof opened this issue Apr 22, 2015 · 8 comments
Closed

Comments

@stof
Copy link
Member

stof commented Apr 22, 2015

See https://travis-ci.org/symfony/symfony/jobs/59546578

@dosten
Copy link
Contributor

dosten commented Apr 22, 2015

I cannot see any error in that job, here yes https://travis-ci.org/symfony/symfony/jobs/59554714.

PHPUnit 4.5.0 by Sebastian Bergmann and contributors.

Configuration read from /home/travis/build/symfony/symfony/phpunit.xml.dist

................................................................. 65 / 66 ( 98%)
.

Time: 9.93 seconds, Memory: 17.40Mb

OK (66 tests, 71 assertions)

Running src/Symfony/Component/Stopwatch tests
PHPUnit 4.5.0 by Sebastian Bergmann and contributors.

Configuration read from /home/travis/build/symfony/symfony/phpunit.xml.dist

.........FF......F.....

Time: 11.86 seconds, Memory: 17.34Mb

There were 3 failures:

1) Symfony\Component\Stopwatch\Tests\StopwatchEventTest::testEnsureStopped
Failed asserting that 422 matches expected 300.

/home/travis/build/symfony/symfony/src/Symfony/Component/Stopwatch/Tests/StopwatchEventTest.php:134

2) Symfony\Component\Stopwatch\Tests\StopwatchEventTest::testStartTime
Failed asserting that false is true.

/home/travis/build/symfony/symfony/src/Symfony/Component/Stopwatch/Tests/StopwatchEventTest.php:145

3) Symfony\Component\Stopwatch\Tests\StopwatchTest::testStop
Failed asserting that 228 matches expected 200.

/home/travis/build/symfony/symfony/src/Symfony/Component/Stopwatch/Tests/StopwatchTest.php:79

FAILURES!
Tests: 23, Assertions: 37, Failures: 3.

we can increase the delta in StopwatchTest to 35, but increasing the delta in StopwatchEventTest to 150 is too much, right?

@stof
Copy link
Member Author

stof commented Apr 23, 2015

@dosten a volatile test means it fails randomly (depending on the load of the Travis servers for instance)

@dosten
Copy link
Contributor

dosten commented Apr 23, 2015

@stof yes, i know. I'm saying that the failing assertions have a 4th parameter named $delta. Ex:

public function testEnsureStopped()
{
    // this also test overlap between two periods
    $event = new StopwatchEvent(microtime(true) * 1000);
    $event->start();
    usleep(100000);
    $event->start();
    usleep(100000);
    $event->ensureStopped();
    $this->assertEquals(300, $event->getDuration(), null, self::DELTA); // delta = 37
}

In the example, phpunit assert as equals values between [263, 337].
The error is: "Failed asserting that 422 matches expected 300.", so the Travis server has 122 miliseconds of delay (in this case).
What i say in my previous comment is that we can increase self:DELTA to 150, so phpunit will assert as equals values between [150, 450].
It's a possible solution?
Sorry for my english, i'm noob

@stof
Copy link
Member Author

stof commented Apr 23, 2015

@dosten if we increase the delta too much, the test will be totally useless. So updating blindly the delta until we never see failures anymore is not the solution either. In the end, we would not detect real bugs in the logic anymore if we allow too much

@dosten
Copy link
Contributor

dosten commented Apr 29, 2015

Same problem here Symfony\Component\Process\Tests\SigchildEnabledProcessTest::testIdleTimeoutNotExceededWhenOutputIsSent i think

@marcojanssen
Copy link

@stof considering both fixes have been merged i assume this ticket can be closed?

@wouterj
Copy link
Member

wouterj commented Jul 26, 2015

@marcojanssen the first ticket is not relevant to this one (it just mentions that the build in that PR is failing because of the volatile test in this issue). The second ticket was rejected, as it was not testing the method.

Tl;dr: Nothing has been fixed around this afaics.

@fabpot
Copy link
Member

fabpot commented Oct 5, 2015

Closing as there is no point in keeping those tickets around. We either fix the problem or we don't care about them, we don't need reminders.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

6 participants