-
-
Notifications
You must be signed in to change notification settings - Fork 2.2k
[Unit] Add PHPUnit-Like Array Difference Visualization #16863
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
❌ Preview Environment deleted from BunnyshellAvailable commands:
|
06c1f9e to
e86f772
Compare
e86f772 to
0c4d261
Compare
0c4d261 to
c39ae0d
Compare
WalkthroughTest assertion helpers were refactored in tests/Api/JsonApiTestCase.php, changing method names and signatures and adding new response/content comparison utilities. Test files across Admin and Shop suites were updated to use the new helpers, generally removing explicit response arguments. Some ShippingMethods tests adjusted expected violation property paths. One Orders test updated a cart helper call. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant T as Test
participant C as Client
participant H as JsonApiTestCase Helper
participant R as Response
T->>C: Perform API request
C-->>T: Last HTTP response stored internally
T->>H: assertResponseViolations(expectedViolations, [assertCount?])
H->>C: Fetch last response
C-->>H: Return Response
alt assertCount = true
H->>H: assertResponseExactViolations(expected)
else assertCount = false
H->>H: assertJsonResponseContainsViolations(response, expected)
end
H-->>T: Assertion result
note over H: New flow removes explicit Response arg\nand supports exact vs contains checks
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Poem
Tip 🔌 Remote MCP (Model Context Protocol) integration is now available!Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats. ✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Actionable comments posted: 0
🧹 Nitpick comments (12)
tests/Api/JsonApiTestCase.php (6)
292-312: Good API simplification; consider clarifying the flag semantics and documenting behaviorThe refactor to fetch the response internally simplifies tests. The boolean
$assertViolationsCounttoggles between exact-match and contains-match, which is slightly opaque. Consider renaming it to$exactMatchor improving the docblock to explicitly state “true = exact, false = contains”. Also, since this method asserts 422 and JSON headers already, downstream tests don’t need to assert the status code again.
319-360: Harden JSON handling and reduce double-decoding in exact-violationsYou decode the response JSON twice and assume both
violationsandhydra:descriptionexist. Decode once, validate the structure, and produce clearer failure messages if keys are missing.Apply:
- $response = $this->client->getResponse(); - - $responseContent = $response->getContent() ?: ''; - $this->assertNotEmpty($responseContent); - - $actualViolations = json_decode($responseContent, true)['violations']; - $actualDescription = json_decode($responseContent, true)['hydra:description']; + $response = $this->client->getResponse(); + + $responseContent = $response->getContent() ?: ''; + $this->assertNotEmpty($responseContent); + + $decoded = json_decode($responseContent, true); + $this->assertIsArray($decoded, 'Response content is not valid JSON or not an object.'); + $this->assertArrayHasKey('violations', $decoded, 'Response JSON is missing "violations" key.'); + $this->assertArrayHasKey('hydra:description', $decoded, 'Response JSON is missing "hydra:description" key.'); + + $actualViolations = $decoded['violations']; + $actualDescription = $decoded['hydra:description'];
367-385: Containment check: guard for missing propertyPath and improve diagnosticsCurrent map-building assumes
propertyPathexists on each violation. For robustness, treat missing/empty paths explicitly and surface a focused message when expected paths are absent.You can minimally harden it with:
- foreach ($violations as $violation) { - $violationMap[$violation['propertyPath']][] = $violation['message']; - } + foreach ($violations as $violation) { + $path = $violation['propertyPath'] ?? ''; + $violationMap[$path][] = $violation['message']; + }And when asserting:
- $this->assertArrayHasKey($propertyPath, $violationMap, $responseContent); + $this->assertArrayHasKey( + $propertyPath, + $violationMap, + sprintf('Expected violation path "%s" not found. Full response: %s', $propertyPath, $responseContent), + );
424-437: Helper looks good; consider reusing existing response-code assertion for consistencyThe helper is a nice consolidation. For consistency with the rest of the suite, consider using
$this->assertResponseCode($response, $statusCode)instead ofself::assertEquals(...), which will keep error formatting uniform.- self::assertEquals( - $statusCode, - $response->getStatusCode(), - json_encode(json_decode($response->getContent(), true), JSON_PRETTY_PRINT) - ); + $this->assertResponseCode($response, $statusCode);
439-463: Surface “expected file missing/invalid” clearly and handle null ComparisonFailure safelyIf the expected file is missing, the current message suggests “not valid JSON”. Also,
getComparisonFailure()can be null—fallback to the exception message to avoid a fatal.protected function assertJsonResponseContent(Response $response, string $filename): void { $expectedFilePath = $this->expectedResponsesPath . '/' . $filename . '.json'; - $expected = file_get_contents($expectedFilePath); + if (!is_file($expectedFilePath)) { + $this->fail(sprintf('Expected response file does not exist: "%s".', $expectedFilePath)); + } + $expected = file_get_contents($expectedFilePath); $actual = $response->getContent() ?: ''; $expectedArray = json_decode($expected, true); if (null === $expectedArray) { - $this->fail(sprintf('Expected response content is not a valid JSON, check the file: "%s".json', $filename)); + $this->fail(sprintf('Expected response file contains invalid JSON: "%s".', $expectedFilePath)); } $actualArray = json_decode($actual, true); if (null === $actualArray) { $this->fail('Actual response content is not a valid JSON'); } $actualArray = $this->replaceDynamicValues($expectedArray, $actualArray); try { $this->assertJsonStringEqualsJsonString(json_encode($expectedArray), json_encode($actualArray)); } catch (ExpectationFailedException $e) { - $expectedFileReferenceMessage = "Check the expected response file: \n" . $expectedFilePath; - $this->fail($e->getComparisonFailure()->getDiff() . "\n" . $expectedFileReferenceMessage); + $diff = $e->getComparisonFailure() ? $e->getComparisonFailure()->getDiff() : $e->getMessage(); + $this->fail($diff . "\nCheck the expected response file:\n" . $expectedFilePath); } }
465-523: Dynamic placeholders: add existence/type guards to avoid notices and edge-case mis-replacementsGreat addition. A few guards will reduce brittle behavior when keys are missing or types differ (e.g., using substr on non-strings).
protected function replaceDynamicValues(array $expectedArray, array $actualArray): array { foreach ($expectedArray as $key => $value) { - if (is_array($value) && isset($actualArray[$key]) && is_array($actualArray[$key])) { + if (is_array($value) && isset($actualArray[$key]) && is_array($actualArray[$key])) { $actualArray[$key] = $this->replaceDynamicValues($value, $actualArray[$key]); continue; } if (is_string($value)) { if (str_contains($value, '@integer@')) { - if (!isset($actualArray[$key])) { + if (!isset($actualArray[$key])) { continue; } $position = strpos($value, '@integer@'); if (strcmp(substr($value, 0, $position), substr((string) $actualArray[$key], 0, $position)) !== 0) { continue; } $actualArray[$key] = substr_replace((string)$actualArray[$key], '@integer@', $position); } - elseif (str_contains($value, '@array@')) { + elseif (str_contains($value, '@array@')) { + if (!isset($actualArray[$key])) { + continue; + } $position = strpos($value, '@array@'); $actualArray[$key] = substr_replace(json_encode($actualArray[$key]), '@array@', $position); } - elseif (str_contains($value, '@boolean@')) { + elseif (str_contains($value, '@boolean@')) { + if (!isset($actualArray[$key])) { + continue; + } $position = strpos($value, '@boolean@'); $actualArray[$key] = substr_replace((string)$actualArray[$key], '@boolean@', $position); } elseif (str_contains($value, '@string@')) { if (!isset($actualArray[$key])) { continue; } $position = strpos($value, '@string@'); if (strcmp(substr($value, 0, $position), substr((string) $actualArray[$key], 0, $position)) !== 0) { continue; } if (!str_contains($actualArray[$key], substr($value, 8))) { continue; } $position = strpos($value, '@string@'); $limit = null; if (strlen($value) > strlen('@string@')) { $expectedRightSubstring = substr($value, $position + strlen('@string@')); $actualRightSubstringPosition = strpos($actualArray[$key], $expectedRightSubstring); if (false !== $actualRightSubstringPosition) { $limit = $actualRightSubstringPosition - $position; } } $actualArray[$key] = substr_replace((string)$actualArray[$key], '@string@', $position, $limit); - } elseif (str_contains($value, '@date@')) { + } elseif (str_contains($value, '@date@')) { + if (!isset($actualArray[$key])) { + continue; + } $position = strpos($value, '@date@'); $actualArray[$key] = substr_replace((string)$actualArray[$key], '@date@', $position); } } } return $actualArray; }If you want, I can follow up with a more type-aware placeholder strategy (e.g., strict boolean/array checks) without changing current semantics.
tests/Api/Admin/ChannelsTest.php (1)
128-130: Avoid redundant 422 assertions; assertResponseViolations already checks status + headers
assertResponseViolations([...], false)asserts HTTP_UNPROCESSABLE_ENTITY and JSON headers internally. The precedingassertResponseStatusCodeSame(...)is redundant and can be removed to reduce noise.- $this->assertResponseStatusCodeSame(Response::HTTP_UNPROCESSABLE_ENTITY); $this->assertResponseViolations([$validation], false);tests/Api/Shop/AddressesTest.php (1)
255-270: Remove redundant 422 assertion; helper already covers it
assertResponseViolations([...])asserts HTTP 422 and JSON headers internally; the precedingassertResponseStatusCodeSame(...)duplicates that check.- $this->assertResponseStatusCodeSame(Response::HTTP_UNPROCESSABLE_ENTITY); $this->assertResponseViolations([ [ 'propertyPath' => 'countryCode', 'message' => 'This value is not a valid country.', ], [ 'propertyPath' => 'phoneNumber', 'message' => 'This value is too long. It should have 255 characters or less.', ], [ 'propertyPath' => 'company', 'message' => 'This value is too long. It should have 255 characters or less.', ], ]);tests/Api/Admin/AddressesTest.php (1)
101-119: Avoid repeated status check; relies on helper’s internal 422 assertionSame as other tests:
assertResponseViolations([...], false)asserts 422 already, so the explicitassertResponseStatusCodeSame(...)can be dropped.- $this->assertResponseStatusCodeSame(Response::HTTP_UNPROCESSABLE_ENTITY); $this->assertResponseViolations( [ [ 'propertyPath' => 'countryCode', 'message' => 'This value is not a valid country.', ], [ 'propertyPath' => 'phoneNumber', 'message' => 'This value is too long. It should have 255 characters or less.', ], [ 'propertyPath' => 'company', 'message' => 'This value is too long. It should have 255 characters or less.', ], ], false, );tests/Api/Shop/OrdersTest.php (1)
463-467: Consider "contains" vs "exact" violation assertionThe new assertResponseViolations defaults to exact-count checks. If the API ever returns additional violations alongside the message you assert, this test may become brittle. If your intent is “contains,” add false as the second param.
Proposed change:
- $this->assertResponseViolations( + $this->assertResponseViolations( [ ['propertyPath' => '', 'message' => 'You cannot change the payment method for a cancelled order.'], ], - ); + false, + );tests/Api/Admin/ShippingMethodsTest.php (2)
270-297: WEB/MOBILE propertyPath swap — double-check backend mappingYou’ve switched expected paths to rules[2].configuration[WEB][amount] then [MOBILE], and likewise for rules[3]. If backend assembles violations per-channel without guaranteed order, relying on exact paths may be order-sensitive. If flakiness occurs, prefer contains mode (second param false) or ensure the comparison is order-insensitive.
Option if you prefer “contains” semantics here:
- $this->assertResponseViolations( + $this->assertResponseViolations( [ [ 'propertyPath' => 'rules[0].configuration[weight]', 'message' => 'This value should be of type numeric.', ], [ 'propertyPath' => 'rules[1].configuration[weight]', 'message' => 'This value should be of type numeric.', ], [ 'propertyPath' => 'rules[2].configuration[WEB][amount]', 'message' => 'This value should be of type numeric.', ], [ 'propertyPath' => 'rules[2].configuration[MOBILE][amount]', 'message' => 'This value should be of type numeric.', ], [ 'propertyPath' => 'rules[3].configuration[WEB][amount]', 'message' => 'This value should be of type numeric.', ], [ 'propertyPath' => 'rules[3].configuration[MOBILE][amount]', 'message' => 'This value should be of type numeric.', ], - ], - ); + ], + false, + );
536-560: Keep channel-key expectations stable across updatesSame WEB/MOBILE path ordering adjustment here. If validator internals change key iteration, exact matching might become brittle. Consider contains mode or confirm your JsonApiTestCase asserts ignore ordering when comparing arrays of violations.
If you decide to switch to contains:
- $this->assertResponseViolations( + $this->assertResponseViolations( [ [ 'propertyPath' => 'rules[0].configuration[weight]', 'message' => 'This value should be of type numeric.', ], [ 'propertyPath' => 'rules[1].configuration[weight]', 'message' => 'This value should be of type numeric.', ], [ 'propertyPath' => 'rules[2].configuration[WEB][amount]', 'message' => 'This value should be of type numeric.', ], [ 'propertyPath' => 'rules[2].configuration[MOBILE][amount]', 'message' => 'This value should be of type numeric.', ], [ 'propertyPath' => 'rules[3].configuration[WEB][amount]', 'message' => 'This value should be of type numeric.', ], [ 'propertyPath' => 'rules[3].configuration[MOBILE][amount]', 'message' => 'This value should be of type numeric.', ], - ], - ); + ], + false, + );
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (16)
tests/Api/Admin/AddressesTest.php(1 hunks)tests/Api/Admin/CatalogPromotionsTest.php(3 hunks)tests/Api/Admin/ChannelsTest.php(1 hunks)tests/Api/Admin/ProductOptionsTest.php(1 hunks)tests/Api/Admin/ProductReviewsTest.php(0 hunks)tests/Api/Admin/ProductVariantsTest.php(0 hunks)tests/Api/Admin/ShippingMethodsTest.php(2 hunks)tests/Api/Admin/StatisticsTest.php(2 hunks)tests/Api/JsonApiTestCase.php(4 hunks)tests/Api/Shop/AddressesTest.php(1 hunks)tests/Api/Shop/Checkout/CartTest.php(0 hunks)tests/Api/Shop/Checkout/CompletionTest.php(0 hunks)tests/Api/Shop/Checkout/PaymentMethodTest.php(0 hunks)tests/Api/Shop/Checkout/ShippingMethodTest.php(0 hunks)tests/Api/Shop/OrdersTest.php(1 hunks)tests/Api/Shop/ProductReviewsTest.php(0 hunks)
💤 Files with no reviewable changes (7)
- tests/Api/Shop/Checkout/ShippingMethodTest.php
- tests/Api/Shop/Checkout/PaymentMethodTest.php
- tests/Api/Admin/ProductReviewsTest.php
- tests/Api/Shop/ProductReviewsTest.php
- tests/Api/Admin/ProductVariantsTest.php
- tests/Api/Shop/Checkout/CompletionTest.php
- tests/Api/Shop/Checkout/CartTest.php
🧰 Additional context used
🧬 Code graph analysis (8)
tests/Api/Admin/ProductOptionsTest.php (2)
src/Sylius/Behat/Client/ResponseChecker.php (2)
hasViolationWithMessage(264-282)isViolationWithMessageInResponse(284-303)src/Sylius/Behat/Client/ResponseCheckerInterface.php (1)
isViolationWithMessageInResponse(20-20)
tests/Api/Admin/ChannelsTest.php (1)
tests/Api/JsonApiTestCase.php (1)
assertResponseViolations(297-312)
tests/Api/Admin/StatisticsTest.php (1)
tests/Api/JsonApiTestCase.php (1)
assertResponseViolations(297-312)
tests/Api/Shop/AddressesTest.php (1)
tests/Api/JsonApiTestCase.php (1)
assertResponseViolations(297-312)
tests/Api/Admin/AddressesTest.php (1)
tests/Api/JsonApiTestCase.php (1)
assertResponseViolations(297-312)
tests/Api/Admin/CatalogPromotionsTest.php (1)
tests/Api/JsonApiTestCase.php (1)
assertResponseViolations(297-312)
tests/Api/Admin/ShippingMethodsTest.php (2)
src/Sylius/Bundle/CoreBundle/Fixture/Factory/ShippingMethodExampleFactory.php (1)
ShippingMethodExampleFactory(34-143)src/Sylius/Behat/Client/ResponseChecker.php (1)
ResponseChecker(21-379)
tests/Api/JsonApiTestCase.php (2)
src/Sylius/Behat/Client/ApiPlatformClient.php (2)
getContent(278-281)ApiPlatformClient(22-392)src/Sylius/Behat/Client/ResponseChecker.php (1)
ResponseChecker(21-379)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (17)
- GitHub Check: Frontend / NodeJS 24.x
- GitHub Check: End-to-end tests (MySQL) / JS with Chromedriver, PHP 8.3, Symfony ^7.2 (test_cached), MySQL 8.4, Twig ^3.3
- GitHub Check: End-to-end tests (MySQL) / JS with Chromedriver, PHP 8.4, Symfony ^7.2 (test_cached), MySQL 8.4, Twig ^3.3
- GitHub Check: End-to-end tests (MySQL) / JS with Panther, PHP 8.3, Symfony ^7.2 (test_cached), MySQL 8.4, Twig ^3.3
- GitHub Check: End-to-end tests (MySQL) / JS with Chromedriver, PHP 8.2, Symfony ^6.4 (test_cached), MySQL 8.0, Twig ^3.3
- GitHub Check: End-to-end tests (MySQL) / JS with Panther, PHP 8.4, Symfony ^7.2 (test_cached), MySQL 8.4, Twig ^3.3
- GitHub Check: End-to-end tests (MySQL) / JS with Panther, PHP 8.2, Symfony ^6.4 (test_cached), MySQL 8.0, Twig ^3.3
- GitHub Check: End-to-end tests (MySQL) / Non-JS, PHP 8.2, Symfony ^6.4 (test_cached), MySQL 8.0, Twig ^3.3
- GitHub Check: End-to-end tests (MySQL) / Non-JS, PHP 8.4, Symfony ^7.2 (test_cached), MySQL 8.4, Twig ^3.3
- GitHub Check: End-to-end tests (MySQL) / Non-JS, PHP 8.3, Symfony ^7.2 (test_cached), MySQL 8.4, Twig ^3.3
- GitHub Check: End-to-end tests (PostgreSQL) / Non-JS, PHP 8.2, Symfony ^6.4, PostgreSQL 15.13
- GitHub Check: End-to-end tests (PostgreSQL) / Non-JS, PHP 8.4, Symfony ^7.2, PostgreSQL 17.5
- GitHub Check: Packages / PHP 8.4, Symfony ^7.2
- GitHub Check: Packages / PHP 8.3, Symfony ^7.2, ORM ^3.3
- GitHub Check: Packages / PHP 8.2, Symfony ^6.4
- GitHub Check: End-to-end tests (MariaDB) / Non-JS, PHP 8.4, Symfony ^7.2, MariaDB 11.4.7, State Machine Adapter symfony_workflow
- GitHub Check: End-to-end tests (MariaDB) / Non-JS, PHP 8.2, Symfony ^6.4, MariaDB 10.11.13, State Machine Adapter winzou_state_machine
🔇 Additional comments (10)
tests/Api/Admin/ChannelsTest.php (1)
179-184: Nice use of the new content helperSwitching to
assertResponse(...)with a fixture file keeps payload checks consistent and gives clearer diffs on failures.tests/Api/Admin/ProductOptionsTest.php (2)
121-141: Good migration to violations helper with “contains” semanticsUsing
assertResponseViolations([...], false)is appropriate here since you care about specific violations, not the full set/count.
95-101: Response fixture assertion is consistent and maintainableUsing
assertResponseCreated('admin/product_option/post_product_option_response')aligns with the new shared helpers and will benefit from the improved diff presentation.tests/Api/Shop/AddressesTest.php (1)
50-54: Adoption of assertResponse(...) improves readability and diff qualityGood use of the new helper for success responses; this centralizes formatting and failure reporting.
tests/Api/Admin/AddressesTest.php (1)
37-42: Consistent success-response assertionThe switch to
assertResponse(...)is clear and keeps test output consistent.tests/Api/Admin/StatisticsTest.php (2)
173-181: Switch to "contains" mode for violations is appropriate herePassing false explicitly opts into contains semantics, which is safer if backend adds extra violations while preserving the one you assert.
202-203: Consistent helper migrationUsing assertResponseViolations($expectedViolations, false) aligns with the new signature and avoids plumbing Response through tests. Good call.
tests/Api/Shop/OrdersTest.php (1)
54-55: Coupon-code helper delegation verifiedThe
updateCartWithAddressalias simply callsupdateCartWithAddressAndCouponCode($tokenValue, $email)with anullcoupon code, so replacing the direct call does not alter any order totals or state.
- In tests/Api/Utils/OrderPlacerTrait.php (lines 219–224),
updateCartWithAddressdelegates toupdateCartWithAddressAndCouponCodewithout providing a coupon code.- The full
updateCartWithAddressAndCouponCodeimplementation (lines 226–236) still accepts and handles an optional$couponCode.No further changes are required—the coupon-enabled helper remains intact, and this adjustment will not impact the
get_orderssnapshot.tests/Api/Admin/CatalogPromotionsTest.php (2)
280-348: Good move to "contains" mode for a large set of scope violationsFor broad validation suites where backend rules can evolve, asserting containment reduces churn while still checking key constraints and paths. Change looks correct and intentional.
459-515: Consistent migration to new violation helper with non-exact semanticsSame rationale as above; this keeps tests resilient to additional actionable messages without weakening intent.
|
@Rafikooo do you plan to move or backport this feature to the https://github.com/lchrusciel/ApiTestCase? It'd be beneficial for more people if this improvement will be shared. |
4db161b to
d95f292
Compare
d95f292 to
c4e372c
Compare
Here's an example of a failing test:
Before:



After: (it also provides a direct way to jump right into the expected response file):

Summary by CodeRabbit