Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Serializer] twice as slow as the JMS serializer #16179

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
Tobion opened this issue Oct 8, 2015 · 14 comments
Closed

[Serializer] twice as slow as the JMS serializer #16179

Tobion opened this issue Oct 8, 2015 · 14 comments

Comments

@Tobion
Copy link
Contributor

Tobion commented Oct 8, 2015

In our project, alot of time is spent serializing our object graph to JSON using the JMS serializer. So we were looking at alternatives to speed things up. Naturally we now tried the symfony serializer with the recent new features that we would need like groups, property naming strategies etc.

So we did some performance benchmarks to compare JMS serializer and symfony serializer with real-world data. Serializing our rather big object graph (which results in ~130 kB of json) 100 times takes

  • JMS: 6.75 seconds
  • Symfony: 14.11 seconds

So the symfony serializer is roughly TWICE as SLOW. No caching was involved. But that shouldn't have influence as the test was run in a single PHP process. That the symfony serializer is even slower is not what we expected since it's code base looks slimmer.

@dunglas any idea what is causing this huge difference?

@dunglas
Copy link
Member

dunglas commented Oct 9, 2015

Can you share the code you used for the benchmark to profile it?

@Tobion
Copy link
Contributor Author

Tobion commented Oct 9, 2015

The cannot share the models for the object graph currently. But the test is pretty straight forward:

<?php

use Doctrine\Common\Annotations\AnnotationReader;
use Doctrine\Common\Annotations\AnnotationRegistry;
use JMS\Serializer\SerializationContext;
use JMS\Serializer\SerializerBuilder;
use JMS\Serializer\SerializerInterface as JmsSerializerInterface;
use Symfony\Component\Serializer\Encoder\JsonEncoder;
use Symfony\Component\Serializer\Mapping\Factory\ClassMetadataFactory;
use Symfony\Component\Serializer\Mapping\Loader\AnnotationLoader;
use Symfony\Component\Serializer\NameConverter\CamelCaseToSnakeCaseNameConverter;
use Symfony\Component\Serializer\Normalizer\ObjectNormalizer;
use Symfony\Component\Serializer\Serializer;
use Symfony\Component\Serializer\SerializerInterface as SymfonySerializerInterface;

class SerializerComparisonTest extends \PHPUnit_Framework_TestCase
{
    /**
     * @var SymfonySerializerInterface
     */
    private $symfonySerializer;

    /**
     * @var JmsSerializerInterface
     */
    private $jmsSerializer;

    protected function setUp()
    {
        $classMetadataFactory = new ClassMetadataFactory(new AnnotationLoader(new AnnotationReader()));

        $callback = function ($dateTime) {
            return $dateTime instanceof \DateTime
                ? $dateTime->format(\DateTime::ISO8601)
                : '';
        };

        $encoders = array(new JsonEncoder());
        $normalizer = new ObjectNormalizer($classMetadataFactory, new CamelCaseToSnakeCaseNameConverter());
        $normalizer->setIgnoredAttributes(array('regions'));
        $normalizer->setCallbacks(
            array('startDate' => $callback, 'endDate' => $callback, 'validFrom' => $callback, 'validTo' => $callback, 'publicationDate' => $callback, 'updatedAt' => $callback)
        );

        $this->symfonySerializer = new Serializer([$normalizer], $encoders);

        AnnotationRegistry::registerLoader('class_exists');

        $this->jmsSerializer = SerializerBuilder::create()
            ->build();
    }

    public function testSymfonySerializer()
    {
        $productsExample = file_get_contents(__DIR__.'/products-example.json');
        $productCollection = $this->jmsSerializer->deserialize($productsExample, ProductCollection::class, 'json');

        for ($i = 0; $i < 100; $i++) {
            $this->symfonySerializer->serialize($productCollection, 'json', ['groups' => ['api']]);
        }

        file_put_contents(__DIR__ . '/products-example-symfony-serialized.json', $this->symfonySerializer->serialize($productCollection, 'json', ['groups' => ['api']]));
    }

    public function testJmsSerializer()
    {
        $productsExample = file_get_contents(__DIR__.'/products-example.json');
        $productCollection = $this->jmsSerializer->deserialize($productsExample, ProductCollection::class, 'json');

        for ($i = 0; $i < 100; $i++) {
            $this->jmsSerializer->serialize($productCollection, 'json', SerializationContext::create()->setGroups(['api']));
        }

        file_put_contents(__DIR__ . '/products-example-jms-serialized.json', $this->jmsSerializer->serialize($productCollection, 'json', SerializationContext::create()->setGroups(['api'])));
    }
}

Maybe you find an obvious error in the above test that explains the performance gap.
So if you have some models and data, you can easily do the test as well.

phpunit --filter testJmsSerializer
phpunit --filter testSymfonySerializer

@dunglas
Copy link
Member

dunglas commented Oct 10, 2015

Ok I've not used a profiler right now but there is obviously two things that slow down the Symfony Serializer here:

  • You use Serialization groups without cache. The metadata extraction system is known to be very slow because it uses reflection. Using the provided Doctrine Cache adapter will instantly improve performance.
  • You use the ObjectNormalizer. As it relies on the PropertyAccess component, it is very convenient but slow (in comparison, JMSSerializer use by default something similar to the PropertyNormalizer). @tucksaun @lyrixx and I discussed in the London Symfony Live of different ways to improve performance of the PropertyAccess component using a cache strategy but this is just some ideas for now. In the meantime when performance matter you can: use the GetSetMethodNormalizer or the PropertyNormalizer or inject in the ObjectNormalizer a custom implementation of the PropertyAccessInterface optimized according to the property access strategy of your project (e.g. don't rely on reflection and directly use a mutator method).

@linaori
Copy link
Contributor

linaori commented Oct 10, 2015

Your testSymfonySerializer calls $this->jmsSerializer->deserialize(, not sure if it's a small error or intentional.

@yosymfony
Copy link

@iltar Seems intentional for testing serialize method

@Tobion
Copy link
Contributor Author

Tobion commented Oct 11, 2015

@dunglas the metadata extraction caches it in memory anyway from what I've seen. So a cache wouldn't help in a single process test.

The ObjectNormalizer is really the slow part. I've changed it to the PropertyNormalizer and now the same test just takes 3.9 seconds.

But in general I've really hard time to replicate the behavior of the JMS serializer. So currently the serialization is broken and missing properties. Some findings:

  • One needs to configure a callback for each DateTime property which is kinda strange. By default the serializer serializes the internal properties of the DateTime instance like lastErrors which is kinda silly.
  • If you want to exclude properties, you can use $normalizer->setIgnoredAttributes(array('property'));. But what if the property you want to ignore is deep inside the object graph? So what if I want to exclude $object->subobject->property but do not want to exclude $object->property?
  • Serializing null cannot easily be omitted. The symfony serializer does not have a built-in solution. One can use callbacks but only per attribute and not for everything.
  • There seems to be a problem with inherited properties which are not serialized.

@dunglas
Copy link
Member

dunglas commented Oct 12, 2015

  • I personally never use callbacks, I prefer to register a dedicated normalizer such as this one for \DateTime normalization in API Platform. I will open a PR to add this normalizer directly in Symfony.
  • Use serialization groups to exclude properties anywhere in the graph (I never use the setIgnoredAttributes system too).
  • You're right. You can use a custom normalizer having this behavior but if this is something common maybe can we add a builtin support for this in Symfony. IMO null are real values with a different meaning than no value at all. For instance API Platform ignores absent properties but not those with null values during the denormalization process. But in some case it probably can make sense to ignore such values.
  • It should work (at least with the ObjectNormalizer), if it doesn't it's a bug. Do you have a failing test to reproduce this problem?

fabpot added a commit that referenced this issue Oct 30, 2015
This PR was squashed before being merged into the 2.3 branch (closes #16294).

Discussion
----------

[PropertyAccess] Major performance improvement

| Q             | A
| ------------- | ---
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #16179
| License       | MIT
| Doc PR        | n/a

This PR improves performance of the PropertyAccess component of ~70%.

The two main changes are:

* caching the `PropertyPath` initialization
* caching the guessed access strategy

This is especially important for the `ObjectNormalizer` (Symfony Serializer) and the JSON-LD normalizer ([API Platform](https://api-platform.com)) because they use the `PropertyAccessor` class in large loops (ex: normalization of a list of entities).

Here is the Blackfire comparison: https://blackfire.io/profiles/compare/c42fd275-2b0c-4ce5-8bf3-84762054d31e/graph

The code of the benchmark I've used (with Symfony 2.3 as dependency):

```php
<?php

require 'vendor/autoload.php';

class Foo
{
    private $baz;
    public $bar;

    public function getBaz()
    {
        return $this->baz;
    }

    public function setBaz($baz)
    {
        $this->baz = $baz;
    }
}

use Symfony\Component\PropertyAccess\PropertyAccess;

$accessor = PropertyAccess::createPropertyAccessor();

$start = microtime(true);

for ($i = 0; $i < 10000; ++$i) {
    $foo = new Foo();
    $accessor->setValue($foo, 'bar', 'Lorem');
    $accessor->setValue($foo, 'baz', 'Ipsum');
    $accessor->getValue($foo, 'bar');
    $accessor->getValue($foo, 'baz');
}

echo 'Time: '.(microtime(true) - $start).PHP_EOL;
```

This PR also adds an optional support for Doctrine cache to keep access information across requests and improve the overall application performance (even outside of loops).

Commits
-------

284dc75 [PropertyAccess] Major performance improvement
@fabpot fabpot closed this as completed Oct 30, 2015
fabpot added a commit that referenced this issue Nov 5, 2015
… 2.3 (dunglas)

This PR was squashed before being merged into the 2.7 branch (closes #16463).

Discussion
----------

[PropertyAccess] Port of the performance optimization from 2.3

| Q             | A
| ------------- | ---
| Bug fix?      | yes
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #16179
| License       | MIT
| Doc PR        | n/a

Portage of #16294 in the 2.7 branch.

Commits
-------

aa4cc90 [PropertyAccess] Port of the performance optimization from 2.3
@Tobion
Copy link
Contributor Author

Tobion commented Nov 9, 2015

@dunglas The optimization you added reduced the above test case with ObjectNormalizer to half. From 14 seconds to 7 seconds. On PHP 7, the PropertyNormalizer took the same time as the JMS serializer (2.1 seconds). So even the simplest of all normalizers is not faster than the JMS serializer which offers alot more built-in features.

@Tobion
Copy link
Contributor Author

Tobion commented Nov 9, 2015

We found the inheritence problem I was talking about above. Our model implements IteratorAggregate and ArrayAccess which causes all other properties on the model to be ignored in the serialization. This is kinda unexpected.

@stof
Copy link
Member

stof commented Nov 9, 2015

@Tobion providing a profile of your script would help finding what should be optimized next. We cannot profile it ourselves, as you haven't provided a full reproducing case above

fabpot added a commit that referenced this issue Nov 28, 2015
This PR was squashed before being merged into the 2.8 branch (closes #16547).

Discussion
----------

[Serializer] Improve ObjectNormalizer performance

| Q             | A
| ------------- | ---
| Bug fix?      | no
| New feature?  | no
| BC breaks?    | no
| Deprecations? | no
| Tests pass?   | yes
| Fixed tickets | #16179
| License       | MIT
| Doc PR        | n/a

Cache attributes detection in a similar way than in #16294 for PropertyAccess.

As for the PropertyAccess Component, I'll open another PR (in 2.8 or 3.1) allowing to cache these attributes between requests using Doctrine Cache.

@Tobion, can you try this PR with your benchmark to estimate the gain?

Commits
-------

683f0f7 [Serializer] Improve ObjectNormalizer performance
@egeloen
Copy link

egeloen commented Sep 25, 2016

IMO, this issue should be reopened. I have profiled Symfony, JMS & Ivory serializer and here you can find a full reproductive case that you can easily profile: https://github.com/egeloen/ivory-serializer-benchmark

And the last benchmark result using caching and dev-master for all libraries: https://travis-ci.org/egeloen/ivory-serializer-benchmark/jobs/162590026

It results that Symfony is the slowest even if the cache is used.

@dunglas
Copy link
Member

dunglas commented Sep 25, 2016

Running your benchmark with last versions of Symfony components (in your benchmark you use 3.0@dev dev master was used) I get different results:

./bin/benchmark --iteration 1000

Ivory\Tests\Serializer\Benchmark\IvoryBenchmark | 0.00075191283226013
Ivory\Tests\Serializer\Benchmark\SymfonyBenchmark | 0.00084061002731323
Ivory\Tests\Serializer\Benchmark\JmsBenchmark | 0.00086667966842651

The Symfony Serializer is faster than JMSSerializer and slower than yours, but nothing significative (IMO) in a real production app.

Btw the cache of the PropertyAccess component isn't enabled (not sure if it will change something for a case like this one) and to be fair, you should use the GetSetMethodNormalizer instead of the ObjectNormalizer: the convenience of the ObjectNormalizer has a cost in term of performance, but it can be drastically improved by implementing #19330.

@dunglas
Copy link
Member

dunglas commented Sep 25, 2016

Using GetSetMethodNormalizer, the Symfony Serializer is the fastest one:

./bin/benchmark --iteration 1000

Ivory\Tests\Serializer\Benchmark\IvoryBenchmark | 0.00076944851875305
Ivory\Tests\Serializer\Benchmark\SymfonyBenchmark | 0.00067409038543701
Ivory\Tests\Serializer\Benchmark\JmsBenchmark | 0.00083684277534485

@egeloen
Copy link

egeloen commented Sep 25, 2016

Thanks for the fast answer, I just update the benchmark according to your feedback

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

8 participants