Also somewhere in the docs we should update the "contract" about application state:

Docblock of a Codeception\Module\Yii2 class is a good place for it.

I want to ask you few things:

• update documentation for helper
• list some breaking/minor changes so I could include them into changelog
• merge with 2.4 to fix travis.yml conflict
• provide some API for helpers so inside a helper we could access current Yii application (or maybe some other Yii services)

https://codeception.com/for/yii should be checked i.e. if everything there is still valid with these changes.

list some breaking/minor changes so I could include them into changelog

The changes are mainly in the configuration:

• If you connect to the same DSN with different settings you might receive an error. You must explicitly confirm that you are aware that these connections (even though they connect to the same database) will not share the same world view.
  While this change is a breaking one, I think it won't apply to a lot of users.

provide some API for helpers so inside a helper we could access current Yii application (or maybe some other Yii services)

In Yii2 the application is globally available via Yii::$app. While we could make it available via an alternative route as well this doesn't really offer any advantage. Because we want to support AR Yii::$app must be set anyway.

https://codeception.com/for/yii should be checked i.e. if everything there is still valid with these changes.

I still have to look into the different parts of a module (like orm, email), not sure how that works yet.

@SamMousa could you put the changes into Changelog.md to make sure no one faces some unpredicted breaks

Will do tomorrow.

@DavertMik I have added docs!

Ok, got it. Thanks!

@SamMousa please note that your changes break the app lifecycle. Currently I'm bootstraping the following code in my api modules:

Yii::$app->response->on(Response::EVENT_BEFORE_SEND, function ($event) {
            if (Yii::$app->request->url == '/rest' || Yii::$app->request->url == '/rest/') {
                return;
            }

            /** @var Response $response */
            $response = $event->sender;
            $response->format = Response::FORMAT_JSON;
            if ($response->statusCode < 400) {
                $response->data = [
                    'success' => true,
                    'status' => $response->statusCode,
                    'data' => $response->data,
                ];
            } else {
                ArrayHelper::remove($response->data, 'status');
                ArrayHelper::remove($response->data, 'type');
                $response->data = [
                    'success' => false,
                    'status' => $response->statusCode,
                    'data' => $response->data,
                ];
            }
        });

The following tests does not work anymore:

        $I->seeResponseCodeIs(200);
        $I->seeResponseIsJson();
        $I->seeResponseContainsJson(['success' => true]);

Issue is here: #4910

The response object gets recreated but your bootstrap code is not run again; thus the event is not attached to the second response object...

The problem here is what to do for any request that's not the first in a test.

In your example not recreating the response object would make all requests in the test case return json.

I can fix it for the first request, but not sure about the n-th request; any idea?

It's a common case to add a custom event handlers to the request/response components via configuration or via module bootstrapping. You need to persist them. As for me - keeping the app state between different tests is a bad idea. Different modules could attach different behaviors, event listeners, configurations to the app components. Keeping them persistent could break everything in a large apps.

I agree, but I'm not talking about different tests but different requests inside test cases:

public function testA($I)
..
$I->doRequest('a');
//here you get the response object that results from the request. This must be recreated when the next line is called.
$I->doRequest('b');

The cause here is that there are multiple response objects and the second one won't have your bootstrapping code.

How is your bootstrap code bootstrapped? It would work if you bound the event to the class or to the application.

What would happen if any code creates a new response object? Whether bootstrapped or not, the assumption you make is that there's only one response object.

We could of course store event handlers and reattach them when these 2 objects get recreated *-)

We bootstrap the code is usual: in the bootstrap section of the config.
Like

'bootstrap' => [ 'api_module1', 'api_module2' ]

Checked the code once again. We attach the event listeners Yii::$app->response->on(Response::EVENT_BEFORE_SEND, function ($event) {} in the init() of the modules. Not in the bootstrap($app).

Okay, both should work though. Of course the problem is that module only gets initialized once.

Workaround would be to use application after request event. I'll post a fix later today

@SamMousa how to configure mailer in single test now?

old helper

    public function configureMailer()
    {
        /** @var \Codeception\Module\Yii2 $module */
        $module = $this->getModule('Yii2');
        /** @var \yii\base\Application $app */
        $app = $module->app;
        /** @var \Codeception\Lib\Connector\Yii2 $connector */
        $connector = $module->client;
        $connector::$mailer = $app->get('mailer');
    }

old test case
https://github.com/bscheshirwork/yii2-app-advanced-rbac/blob/3ccfdae139bd508a9c5a06bb326326df07da172e/frontend/tests/functional/ContactFailCest.php#L13-L14

        \Yii::$app->set('mailer', 'frontend\stub\MockMailer');
        $I->configureMailer();

mock mailer: return fail on sending mail
php-code/frontend/tests/_stub/MockMailer.php
https://github.com/bscheshirwork/yii2-app-advanced-rbac/blob/3ccfdae139bd508a9c5a06bb326326df07da172e/frontend/tests/_stub/MockMailer.php#L6-L12

class MockMailer extends \Codeception\Lib\Connector\Yii2\TestMailer
{
    protected function sendMessage($message)
    {
        return false;
    }
}

@bscheshirwork i'll get back to you

@samdark @DavertMik please stop releasing BC breaks as bugfix. This PR completely changes the API of the involved classes and should have been tagged in 3.0.0.

Semver is not a joke, I had to drop each and every multiple range version constraint described in composer doc because the projects you maintain just ignore completely https://semver.org/

@Slamdunk the issue here is that the public API was never clearly defined.
Therefore it is impossible to decide whether a change breaks BC, this would mean according to semver, every release should increase the major version number.
This would of course not help you at all other than that your default constraint might work better; you would still be stuck on an old version since there is no development going on in different major versions.

If you would like to contribute something useful instead of complaints feel free to create tests at https://github.com/codeception/yii2-tests

This way we can test changes to identify if they break BC before actually releasing something.

@Slamdunk the issue here is that the public API was never clearly defined.
Therefore it is impossible to decide whether a change breaks BC

API is always clearly defined: everything public and not signed as @internal is an API.
The fact that an API makes sense or not is another story, but Semver has no correlation to the value of an API.
If the major number doesn't change, every public methods/variable/constant shouldn't change.

this would mean according to semver, every release should increase the major version number.

Yeah, and the issue for that is?

This would of course not help you at all other than that your default constraint might work better; you would still be stuck on an old version since there is no development going on in different major versions.

That's the exact behaviour I expect from Semver.
If I want to stick to a specific API, it's my choice.

If you have not the resources to maintain multiple versions (as I don't have in the FOSS I maintain) it would be perfectly ok to bump frequently the major number.

If you would like to contribute something useful instead of complaints feel free to create tests at https://github.com/codeception/yii2-tests

This way we can test changes to identify if they break BC before actually releasing something.

Tests has no correlation to API (although are a fundamental step to decide where an API should be driven to); if you want to identify BC breaks there are simple tool like https://github.com/Roave/BackwardCompatibilityCheck to help you

Does codeception even use semver at the moment?

The tag names suggest so.
If Codeception doesn't want to follow Semver that's OK, but if so it should immediately drop X.Y.Z tag template to avoid misleading informations.

Lol, I think the X.Y.Z format has existed long before semver did.
As far as I know it is not a patented format.

Of course is not, but Codeception uses composer to spread itself, and it is statistically near 100 the percentage of the users that, in a composer ecosystem, would consider X.Y.Z to be a Semver-following package.

Yet luckily there's very few that are so entitled as you are :)

Usually we try to avoid introducing breaking changes in patch releases.

@SamMousa I just want to pay your attention on changed behavior which I faced few days ago.
If it's what you wanted to implement, that there are no problem (perhaps). (All right, it's disputable case, but still.)

So:
If we use haveFixtures() method directly, i.e. in _before() method, then for now database connections will not be closed after every tests.
There was issue: #3456
And this issue was solved: https://github.com/Codeception/Codeception/pull/3586/files
But for now you changed code, so only fixtures in _fixtures() method are carefully handled: 08e487b#diff-938f756d763fc21aace78bb43ea26870R292
And previous code was removed: 08e487b#diff-938f756d763fc21aace78bb43ea26870L242

Thanks for attention.

Is this being worked on or is there a way for us to force close database connections?

@czzplnm there is a quick workaround I've described in #4926