Add an HTTP client dedicated to functional API testing #2608

dunglas · 2019-03-12T19:11:11Z

Q	A
Bug fix?	no
New feature?	yes
BC breaks?	no
Deprecations?	no
Tests pass?	yes
Fixed tickets	n/a
License	MIT
Doc PR	api-platform/docs#875

Currently, there are no very satisfactory solutions to write functional tests for web APIs built in PHP (API Platform and Symfony included):

Mink/Behat/Behatch are BDD-oriented (which is completely fine, but doesn't fit with all projects/teams), are a bit complex to setup, have a high barrier to entry and are still not fully compatible with Symfony 4 (you need to rely on a dev version of Behat). They also lack of utilities, for database testing for instance.
WebTestCase/BrowserKit are dedicated to webpage testing through web scraping (DOM Crawler, CSS selectors, simulation of browsers actions such as clicking or reloading a page...). Their API don't fit well with API testing. However, they benefit form the large ecosystem of PHPUnit, and give access to the numerous functional testing helpers provided by Symfony.
External solutions that don't manipulate the Symfony Kernel (but are true HTTP clients) such as Postman or BlackFire Player require to setup a web server for the testing env and don't give access to the service container (for instance, to test if the database has been updated, or if a mail has been sent).

It's time to say hi to ApiTestCase and Test\Client! This new set of API testing utilities is built on top of the Symfony's Client test class, and implements the exact same interface than the brand new Symfony HttpClient component.

As you'll see, it also plays very well with Alice (the fixture generator with an official Symfony recipe), that gets new power in the operation!

Let's see how it looks:

namespace App\Tests;

use ApiPlatform\Core\Bridge\Symfony\Bundle\Test\ApiTestCase;
use Dunglas\UserBundle\Entity\ForgotPassword\Jti;
use Hautelook\AliceBundle\PhpUnit\RefreshDatabaseTrait;
use Lcobucci\JWT\Parser;

class UsersTest extends ApiTestCase
{
    use RefreshDatabaseTrait; // This new trait I added to HautelookAliceBundle will take care of refreshing the database content to put it in a known state between every tests

     // A simple test
    public function testCreateUser()
    {
        $client = static::createClient(); // $client implements the new Symfony's HttpClientInterface
        $response = $client->request('POST', '/api/users', ['json' => [
            'email' => '[email protected]',
            'plainPassword' => 'foo',
        ]]);
        $this->assertSame(201, $response->getStatusCode());

        $responseContent = $response->toArray();
        $this->assertStringStartsWith('/api/users/', $responseContent['@id']);

        $this->assertArraySubset([
            '@type' => 'http://schema.org/Person',
            'email' => '[email protected]',
        ], $responseContent);
        $this->assertArrayNotHasKey('password', $responseContent);
        $this->assertArrayNotHasKey('plainPassword', $responseContent);
    }

    // something more advanced, accessing the mails and the DB
    public function testChangePassword()
    {
        $client = static::createClient();
        $client->enableProfiler();
        $client->disableReboot();

        // Send the request
        $response = $client->request('POST', '/api/users/token_request', ['json' => ['username' => '[email protected]']]);
        $this->assertSame(202, $response->getStatusCode());

        $mailCollector = $client->getProfile()->getCollector('swiftmailer');

        // extract the JWT contained in the mail
        preg_match('#[A-Za-z0-9-_=]+\.[A-Za-z0-9-_=]+\.?[A-Za-z0-9-_.+/=]*#', $mailCollector->getMessages()[0]->getBody(), $matches);
        $jwt = $matches[0];
        $decodedToken = (new Parser())->parse($jwt);

        // Change the password using the provided token
        $response = $client->request('PUT', $decodedToken->getClaim('sub'), [
            'json' => ['plainPassword' => 'newPassword'],
            'auth_bearer' => $jwt,
        ]);
        $this->assertSame(200, $response->getStatusCode());

        // Checks that the user have been changed
        $user = self::$container->get('test.security.helper')->getUser();
        $encoder = self::$container->get('test.security.encoder_factory')->getEncoder($user);
        $this->assertTrue($encoder->isPasswordValid($user->getPassword(), 'newPassword', $user->getSalt()));

        // Checks that the JTI has been saved
        $this->assertNotNull(
            self::$container->get('test.doctrine')->getRepository(Jti::class)->find($decodedToken->getClaim('jti'))
        );
    }
}

The following fixtures are loaded automatically. The database is reseted to a known test between every test (regardless of it fails or not) using transactions for maximum performance.

App\Entity\User:
  user_kevin:
    email: [email protected]
    password: \$argon2i\$v=19\$m=1024,t=2,p=2\$TW9YajBjdGpUVUp4Uks1aA\$vwFQakIF9aGGoC5KjT+S3kuOYDtwtghzMuEZ/xP6FnM # maman

TODO:

Import tests
Write docs
Add support for Basic auth
Wait for Symfony 4.3

composer.json

baudev · 2019-03-12T19:45:23Z

It could be great adding the possibility of testing objects with the Symfony Validator.
I recently created a gist (link) concerning that.

It would allow testing an object with the following assertion:

$this->assertSymfonyValidate($objectToTest);

What do you think about implementing it in ApiTestCase?

src/Bridge/Symfony/Bundle/Test/Response.php

dunglas · 2019-03-12T21:00:35Z

@baudev I plan to add extra assertions in other PRs (like a response against a JSON Schema), but the one you're talking about should be added directly to Symfony IMHO (it can benefit to non-API Platform projects as well).

src/Bridge/Symfony/Bundle/DependencyInjection/ApiPlatformExtension.php

teohhanhui · 2019-03-13T15:31:56Z

Actually, is there any reason why this goes into API Platform instead of in Symfony? I can see that the implementation is completely decoupled from API Platform?

dunglas · 2019-03-13T15:49:06Z

@teohhanhui there are some API Platform specifics (JSON-LD by default), and @bendavies and I plan to add more (specific helpers to navigate in links, JSON Schema assertions...). But the current implementation could go in Symfony FrameworkBundle yes, technically speaking. That being said, I prefer to keep API-related stuff in this project, and this client is definitely API-related.

dunglas · 2019-03-13T16:54:28Z

I just added a nice API to deal with API errors (validation errors for instance), and Basic authentication:

namespace App\Tests;

use ApiPlatform\Core\Bridge\Symfony\Bundle\Test\ApiTestCase;
use Hautelook\AliceBundle\PhpUnit\RefreshDatabaseTrait;
use Symfony\Contracts\HttpClient\Exception\ClientExceptionInterface;

class ProjectsTest extends ApiTestCase
{
    use RefreshDatabaseTrait;

    public function testNotAbleToCreateProjectWithoutOwner()
    {
        $this->expectException(ClientExceptionInterface::class);
        $this->expectExceptionCode(400); // HTTP status code
        $this->expectExceptionMessage(<<<ERROR
users: This collection should contain 1 element or more.
users: The current logged in user must be part of the users owning this resource.
ERROR
);

        $client = static::createClient();
        $response = $client->request('POST', '/api/projects', [
            'basic_auth' => ['[email protected]', 'maman'],
            'json' => [
                'name' => 'My project',
            ],
        ]);
        $response->getContent();
    }
}

To get these nice error messages pre-populated, and to be able to assert on them your API just have to respect the Hydra convention for errors, or the RFC 7807 one).

src/Bridge/Symfony/Bundle/Test/Response.php

bendavies · 2019-03-13T17:25:06Z

i would prefer optional throwing (configurable?). in my tests, is manually check response codes and response content on error.

dunglas · 2019-03-13T17:40:14Z

@bendavies you can do it too, it's natively supported by HttpClient, just pass false to $response->getContent() and it will not throw, then you'll be able to do your custom assertions.

bendavies · 2019-03-13T17:41:06Z

yeah thanks i should have remembered that... (you mean false)

This PR was squashed before being merged into the 4.3-dev branch (closes #30549). Discussion ---------- [HttpClient] Make exceptions public | Q | A | ------------- | --- | Branch? | master | Bug fix? | no | New feature? | no | BC breaks? | no  | Deprecations? | no  | Tests pass? | yes  | Fixed tickets | #...  | License | MIT | Doc PR | n/a Makes it easier to implement the interface. See api-platform/core#2608 Commits ------- 928d774e4a [HttpClient] Make exceptions public

This PR was squashed before being merged into the 4.3-dev branch (closes #30549). Discussion ---------- [HttpClient] Make exceptions public | Q | A | ------------- | --- | Branch? | master | Bug fix? | no | New feature? | no | BC breaks? | no  | Deprecations? | no  | Tests pass? | yes  | Fixed tickets | #...  | License | MIT | Doc PR | n/a Makes it easier to implement the interface. See api-platform/core#2608 Commits ------- 928d774 [HttpClient] Make exceptions public

src/Bridge/Symfony/Bundle/Test/ApiTestCase.php

… exception messages (dunglas) This PR was squashed before being merged into the 4.3-dev branch (closes #30559). Discussion ---------- [HttpClient] Parse common API error formats for better exception messages | Q | A | ------------- | --- | Branch? | master  | Bug fix? | no | New feature? | yes  | BC breaks? | no  | Deprecations? | no  | Tests pass? | yes  | Fixed tickets | n/a  | License | MIT | Doc PR | todo? Use extra details provided by popular error formats following to improve HTTP exception messages. The following formats are supported: * Hydra (default in API Platform) * RFC 7807 (followed by Symfony's [ConstraintViolationListNormalizer](https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Serializer/Normalizer/ConstraintViolationListNormalizer.php) and supported by API Platform and Apigility) * JSON:API (because it respects the semantic of the RFC 7807) It allows to write code like the following (here in a test context): ```php public function testBadRequest() { $this->expectException(ClientExceptionInterface::class); $this->expectExceptionCode(400); // HTTP status code $this->expectExceptionMessage(<<<ERROR Validation Failed users: This collection should contain 1 element or more. users: The current logged in user must be part of the users owning this resource. ERROR ); $response = (HttpClient::create())->request('POST', 'http://example.com/api/projects', [ 'json' => [ 'name' => 'My project', ], ]); $response->getContent(); } ``` Port of api-platform/core#2608 (comment). Commits ------- 96df4464a1 [HttpClient] Parse common API error formats for better exception messages

… exception messages (dunglas) This PR was squashed before being merged into the 4.3-dev branch (closes #30559). Discussion ---------- [HttpClient] Parse common API error formats for better exception messages | Q | A | ------------- | --- | Branch? | master  | Bug fix? | no | New feature? | yes  | BC breaks? | no  | Deprecations? | no  | Tests pass? | yes  | Fixed tickets | n/a  | License | MIT | Doc PR | todo? Use extra details provided by popular error formats following to improve HTTP exception messages. The following formats are supported: * Hydra (default in API Platform) * RFC 7807 (followed by Symfony's [ConstraintViolationListNormalizer](https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Serializer/Normalizer/ConstraintViolationListNormalizer.php) and supported by API Platform and Apigility) * JSON:API (because it respects the semantic of the RFC 7807) It allows to write code like the following (here in a test context): ```php public function testBadRequest() { $this->expectException(ClientExceptionInterface::class); $this->expectExceptionCode(400); // HTTP status code $this->expectExceptionMessage(<<<ERROR Validation Failed users: This collection should contain 1 element or more. users: The current logged in user must be part of the users owning this resource. ERROR ); $response = (HttpClient::create())->request('POST', 'http://example.com/api/projects', [ 'json' => [ 'name' => 'My project', ], ]); $response->getContent(); } ``` Port of api-platform/core#2608 (comment). Commits ------- 96df446 [HttpClient] Parse common API error formats for better exception messages

stale

soyuka

this stuff rocks

src/Bridge/Symfony/Bundle/Test/Response.php

tests/Bridge/Symfony/Bundle/Test/ClientTest.php

…ns with Panther and API Platform's test client (dunglas) This PR was submitted for the 4.3 branch but it was merged into the 4.4 branch instead (closes #32207). Discussion ---------- [FrameworkBundle] Allow to use the BrowserKit assertions with Panther and API Platform's test client | Q | A | ------------- | --- | Branch? | 4.3 | Bug fix? | yes | New feature? | no  | BC breaks? | no  | Deprecations? | no  | Tests pass? | yes  | Fixed tickets | n/a | License | MIT | Doc PR | n/a I'm adding a testing client for API Platform that implements the interfaces of HttpClient: api-platform/core#2608 Most PHPUnit assertions provided by Symfony are useful and can be reused, but the ones using the crawler are not relevant and pollute auto-complete suggestions (because a web API usually returns JSON, not HTML). This PR splits the existing trait to allow reusing the HTTP related assertions only. Commits ------- cd0341e [FrameworkBundle] Allow to use the BrowserKit assertions with Panther and API Platform's test client

This PR was squashed before being merged into the 4.3-dev branch (closes #30549). Discussion ---------- [HttpClient] Make exceptions public | Q | A | ------------- | --- | Branch? | master | Bug fix? | no | New feature? | no | BC breaks? | no  | Deprecations? | no  | Tests pass? | yes  | Fixed tickets | #...  | License | MIT | Doc PR | n/a Makes it easier to implement the interface. See api-platform/core#2608 Commits ------- 928d774e4a [HttpClient] Make exceptions public

… exception messages (dunglas) This PR was squashed before being merged into the 4.3-dev branch (closes #30559). Discussion ---------- [HttpClient] Parse common API error formats for better exception messages | Q | A | ------------- | --- | Branch? | master  | Bug fix? | no | New feature? | yes  | BC breaks? | no  | Deprecations? | no  | Tests pass? | yes  | Fixed tickets | n/a  | License | MIT | Doc PR | todo? Use extra details provided by popular error formats following to improve HTTP exception messages. The following formats are supported: * Hydra (default in API Platform) * RFC 7807 (followed by Symfony's [ConstraintViolationListNormalizer](https://github.com/symfony/symfony/blob/master/src/Symfony/Component/Serializer/Normalizer/ConstraintViolationListNormalizer.php) and supported by API Platform and Apigility) * JSON:API (because it respects the semantic of the RFC 7807) It allows to write code like the following (here in a test context): ```php public function testBadRequest() { $this->expectException(ClientExceptionInterface::class); $this->expectExceptionCode(400); // HTTP status code $this->expectExceptionMessage(<<<ERROR Validation Failed users: This collection should contain 1 element or more. users: The current logged in user must be part of the users owning this resource. ERROR ); $response = (HttpClient::create())->request('POST', 'http://example.com/api/projects', [ 'json' => [ 'name' => 'My project', ], ]); $response->getContent(); } ``` Port of api-platform/core#2608 (comment). Commits ------- 96df4464a1 [HttpClient] Parse common API error formats for better exception messages

teohhanhui previously requested changes Mar 12, 2019

View reviewed changes

composer.json Outdated Show resolved Hide resolved

GregoireHebert approved these changes Mar 12, 2019

View reviewed changes

GregoireHebert reviewed Mar 12, 2019

View reviewed changes

src/Bridge/Symfony/Bundle/Test/Response.php Show resolved Hide resolved

dunglas commented Mar 12, 2019

View reviewed changes

src/Bridge/Symfony/Bundle/DependencyInjection/ApiPlatformExtension.php Outdated Show resolved Hide resolved

dunglas mentioned this pull request Mar 13, 2019

[HttpClient] Make exceptions public symfony/symfony#30549

Merged

bendavies reviewed Mar 13, 2019

View reviewed changes

src/Bridge/Symfony/Bundle/Test/Response.php Outdated Show resolved Hide resolved

teohhanhui reviewed Mar 13, 2019

View reviewed changes

src/Bridge/Symfony/Bundle/Test/Response.php Outdated Show resolved Hide resolved

dunglas mentioned this pull request Mar 13, 2019

[HttpClient] Parse common API error formats for better exception messages symfony/symfony#30559

Merged

dunglas force-pushed the test-client branch from d1131a8 to 5777335 Compare March 14, 2019 10:43

cyve reviewed Mar 18, 2019

View reviewed changes

src/Bridge/Symfony/Bundle/Test/ApiTestCase.php Outdated Show resolved Hide resolved

soyuka mentioned this pull request Apr 9, 2019

Resource resolver for inheritance #2714

Closed

soyuka force-pushed the test-client branch from e5225b3 to 82a1649 Compare April 9, 2019 14:34

dunglas mentioned this pull request Apr 15, 2019

Add a test suite dunglas/ShopApiPlugin#7

Open

dunglas force-pushed the test-client branch 2 times, most recently from 478116b to 7d7d038 Compare June 24, 2019 20:58

Add an HTTP client dedicated to functional API testing

8a43ba2

dunglas force-pushed the test-client branch from 835c6e4 to 8a43ba2 Compare June 25, 2019 16:11

soyuka reviewed Jun 25, 2019

View reviewed changes

src/Bridge/Symfony/Bundle/Test/Response.php Show resolved Hide resolved

tests/Bridge/Symfony/Bundle/Test/ClientTest.php Outdated Show resolved Hide resolved

dunglas added 3 commits June 26, 2019 07:47

Remove duplicate docblock

99f84e1

Review

99e2f5a

Add more tests

b3e1c71

dunglas merged commit 44c5080 into api-platform:master Jun 26, 2019

dunglas deleted the test-client branch June 26, 2019 17:00

This was referenced Jun 27, 2019

[FrameworkBundle] Allow to use the BrowserKit assertions with Panther and API Platform's test client symfony/symfony#32207

Merged

Add convenient test assertions for web APIs #2887

Merged

dunglas mentioned this pull request Sep 23, 2019

ApiTestCase and assertions docs api-platform/docs#875

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add an HTTP client dedicated to functional API testing #2608

Add an HTTP client dedicated to functional API testing #2608

dunglas commented Mar 12, 2019 •

edited

Loading

baudev commented Mar 12, 2019

dunglas commented Mar 12, 2019 •

edited

Loading

teohhanhui commented Mar 13, 2019

dunglas commented Mar 13, 2019 •

edited

Loading

dunglas commented Mar 13, 2019 •

edited

Loading

bendavies commented Mar 13, 2019 •

edited

Loading

dunglas commented Mar 13, 2019 •

edited

Loading

bendavies commented Mar 13, 2019 •

edited

Loading

soyuka left a comment

Add an HTTP client dedicated to functional API testing #2608

Add an HTTP client dedicated to functional API testing #2608

Conversation

dunglas commented Mar 12, 2019 • edited Loading

baudev commented Mar 12, 2019

dunglas commented Mar 12, 2019 • edited Loading

teohhanhui commented Mar 13, 2019

dunglas commented Mar 13, 2019 • edited Loading

dunglas commented Mar 13, 2019 • edited Loading

bendavies commented Mar 13, 2019 • edited Loading

dunglas commented Mar 13, 2019 • edited Loading

bendavies commented Mar 13, 2019 • edited Loading

soyuka left a comment

Choose a reason for hiding this comment

dunglas commented Mar 12, 2019 •

edited

Loading

dunglas commented Mar 12, 2019 •

edited

Loading

dunglas commented Mar 13, 2019 •

edited

Loading

dunglas commented Mar 13, 2019 •

edited

Loading

bendavies commented Mar 13, 2019 •

edited

Loading

dunglas commented Mar 13, 2019 •

edited

Loading

bendavies commented Mar 13, 2019 •

edited

Loading