[Serializer] Change visibility of getNormalizer and getDenormalizer #24206

welcoMattic · 2017-09-14T16:40:04Z

Q	A
Branch?	3.4
Bug fix?	no
New feature?	yes
BC breaks?	no
Deprecations?	no
Tests pass?	yes
Fixed tickets
License	MIT
Doc PR

I'm currently working on a projet that have many normalizers. Enough to get almost 20sec in getDenormalizer method:

I would like to write my own Serializer, and I figure out that getNormalizer and getDenormalizer methods are private.

Here what I would like to do:

<?php

class ChainSerializer extends Serializer
{
    public function __construct(array $normalizers = [], array $encoders = [])
    {
        $indexedNormalizers = [];

        foreach ($normalizers as $normalizer) {
            /* Little workaround to get clean indices */
            $normalizerClassName = get_class($normalizer);
            $modelClassName = str_replace('Normalizer', '', $normalizerClassName);
            $modelClassName = str_replace('\\\\', '\\Model\\', $modelClassName);
            $indexedNormalizers[$modelClassName] = $normalizer;
        }

        parent::__construct($indexedNormalizers, $encoders);
    }

    protected function getNormalizer($data, $format, array $context)
    {
        return $this->normalizers[get_class($data)];
    }

    protected function getDenormalizer($data, $class, $format, array $context)
    {
        return $this->normalizers[$class];
    }
}

I figure out too these $normalizers are protected which enforce me to submit this PR.

Ping @dunglas @lyrixx

lyrixx · 2017-09-14T16:43:23Z

I found this use case legit and right now I don't see an easy way to achieve this perf optim (quite huge in this project)

ogizanagi

It looks sensible to me.

dunglas · 2017-09-15T17:07:22Z

It used to have a cache for this method, it has been removed in #17140. But the removal of this cache introduced a huge performance penalty (the one you suffered).
Maybe can it be a good idea to reintroduce this cache but make it disabled by default?

nicolas-grekas · 2017-09-15T17:27:43Z

The reason why we remove the cache would be still valid, so that this option would be a foot gun I guess.
Instead, what about adding an interface that would provide a method that would return a cache key? Then if it is implemented, we use the cache.

dunglas · 2017-09-15T17:48:01Z

@welcoMattic, can you share your profile publicly?

dunglas · 2017-09-15T17:51:07Z

@nicolas-grekas I think it won't help, the previous cache allowed to skip the whole loop. Here we still need to loop and to call supports*. supports* is usually lightweight. Calling a method to get a cache key instead will have no benefit.

Another proposal: introduce a NonCacheableSupportsReturnInterface interface and use the cache if no normalizer implements it.

nicolas-grekas · 2017-09-15T18:58:51Z

we talked with @dunglas and might have found an idea:
we could add an new interface with a getProvidedTypes
this method would return an array keyed by PHP-type/class, valued by a bitfield of "vary" consts: DATA | CONTEXT.
this would tell the outside what the "supports*" methods vary on, so that good enough cache/optimizations strategies could be implemented with that info.

ro0NL · 2017-09-15T20:12:48Z

is the vary strategy worth it? It's nice.. but feels a bit over-engineered =/ Eventually the Serializer is simple chain implementation, which is fine to me.

Exposing here is the easiest no? Or introduce a new abstraction layer?

While at it; Serializer::denormalizeObject() can be inlined i guess.

dunglas · 2017-09-15T20:18:48Z

@ro0NL exposing only fix partially the problem, from my experience, this performance problem occurs often when you use the serializer intensively (yes I've API Platform in mind).

Unlike exposing, fixing this problem would benefit to every users without requiring custom developments.

ro0NL · 2017-09-15T20:31:48Z

What about a new AbstractKnownObjectNormalizer 😱 or so. It would have final supports method and exposes it supported object types.

So kinda following the same path proposed by @nicolas-grekas, but perhaps easier to implement.

nicolas-grekas · 2017-09-15T21:44:21Z

I'd be interested in knowing if a simple approach like #24228 could be enough?
Can any of you measure any significant improvement with it?

ro0NL · 2017-09-15T22:32:10Z

Also extends Serializer might not be the right trade-off. If $normalizerClassName = get_class($normalizer); is your convention, perhaps implement the required interfaces directly? Most would be simple method forwarding anyway.

Yet i understand you prefer this extension point, in favor to copy/pasting Serializer::__construct or so. A new abstraction layer effectively is the same as this PR, probably not worth it.

welcoMattic · 2017-09-18T07:46:06Z

@dunglas hmm, sorry it's a private application. But I can make a new simple Sf app with as many normalizers.

Thanks all for discussing about this performance issue. I'll try @nicolas-grekas PR soon.

lyrixx · 2017-09-18T09:43:07Z

Hi

I'd be interested in knowing if a simple approach like #24228 could be enough?

In our case it's not enough. It's logical because our normalizers don't have this cache layer. The only way to make it faster in our case is to allow us to override the getDenormalizer / getNormalizer method.

nicolas-grekas

Just blocking merge until #24206 (comment) has been addressed as OK or KO as a more generic fix for the perf issue.

…kas) This PR was merged into the 3.4 branch. Discussion ---------- [Serializer] Add local cache to normalizers | Q | A | ------------- | --- | Branch? | 3.4 | Bug fix? | no | New feature? | no | BC breaks? | no | Deprecations? | no | Tests pass? | yes | Fixed tickets | #24206 | License | MIT | Doc PR | - Should help making the Serializer a bit faster. Commits ------- b0c5cf0 [Serializer] Add local cache to normalizers

nicolas-grekas · 2017-09-26T10:25:58Z

@welcoMattic @lyrixx @dunglas any OK/KO about #24206 (comment)?

Status: waiting feedback

lyrixx · 2017-09-26T12:57:07Z

we talked with @dunglas and might have found an idea:
we could add an new interface with a getProvidedTypes
this method would return an array keyed by PHP-type/class, valued by a bitfield of "vary" consts: DATA | CONTEXT.
this would tell the outside what the "supports*" methods vary on, so that good enough cache/optimizations strategies could be implemented with that info.

I'm not sure to follow you. Because how would you generated a cache-id when the normalizer vary on the the data? If it's an object it's quite simple (spl_object_hash) but if it's an array ...

nicolas-grekas · 2017-09-26T12:59:21Z

how would you generated a cache-id when the normalizer vary on the the data

We would not. That'd then be the responsibility of the normalizer class. But in you example, you don't vary by "data", but only by "type(data)", which would be covered by the interface so caching would happen.

lyrixx · 2017-09-26T13:05:08Z

Ok, so your cache layer will only cache something based on the context ?

nicolas-grekas · 2017-09-26T13:17:42Z

Almost like http vary yes, with 3 vary-by possible: type(data) and/or context yes, state inside data cannot be considered from this layer

nicolas-grekas · 2017-09-26T13:18:53Z

Oh, but there would still be a benefit when the data is considered: if you say "for this type, it varies by data", then the cache layer will still be able to skip the normalizer for any other non specified type.

lyrixx · 2017-09-26T13:35:54Z

It looks good to me. We will try to update this PR as soon as possible.

welcoMattic · 2017-09-26T13:41:40Z

Thanks for all your ideas, I'll see with @lyrixx to update the PR before the feature freeze

nicolas-grekas · 2017-10-05T10:49:25Z

I think it's too late for 3.4, adding this vary-by cache should be done for 4.1 now.
I'm closing this PR as we figured out a way to not open these methods, which is always the solution we prefer for maintenance purpose, and replacing it by an issue with a description of what we discussed.

nicolas-grekas · 2017-10-05T10:55:10Z

See #24436

Change visibility of 2 getters of Serializer

e9f2ea4

carsonbot added Status: Needs Review Serializer labels Sep 14, 2017

nicolas-grekas added this to the 3.4 milestone Sep 15, 2017

nicolas-grekas added the Feature label Sep 15, 2017

ogizanagi approved these changes Sep 15, 2017

View reviewed changes

ogizanagi requested a review from dunglas September 15, 2017 16:57

nicolas-grekas mentioned this pull request Sep 15, 2017

[Serializer] Add local cache to normalizers #24228

Merged

nicolas-grekas requested changes Sep 18, 2017

View reviewed changes

nicolas-grekas added Status: Waiting feedback and removed Status: Needs Review labels Sep 26, 2017

nicolas-grekas closed this Oct 5, 2017

nicolas-grekas mentioned this pull request Oct 5, 2017

[Serializer] make normalizers expose their vary-by characteristics, and use it for accurate caching #24436

Closed

welcoMattic deleted the serializer-getters-visibility branch February 4, 2020 15:10

VincentLanglet mentioned this pull request Jul 21, 2020

Make handle xml http request response protected sonata-project/SonataAdminBundle#6214

Merged

bradjones1 mentioned this pull request Oct 16, 2023

[Serializer] Change visibility of Serializer::getNormalizer and ::getDenormalizer from private to protected #52084

Closed

Uh oh!

[Serializer] Change visibility of getNormalizer and getDenormalizer #24206

[Serializer] Change visibility of getNormalizer and getDenormalizer #24206

Uh oh!

Conversation

welcoMattic commented Sep 14, 2017 • edited by nicolas-grekas Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

lyrixx commented Sep 14, 2017

Uh oh!

ogizanagi left a comment

Choose a reason for hiding this comment

Uh oh!

dunglas commented Sep 15, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nicolas-grekas commented Sep 15, 2017

Uh oh!

dunglas commented Sep 15, 2017

Uh oh!

dunglas commented Sep 15, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nicolas-grekas commented Sep 15, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ro0NL commented Sep 15, 2017

Uh oh!

dunglas commented Sep 15, 2017

Uh oh!

ro0NL commented Sep 15, 2017

Uh oh!

nicolas-grekas commented Sep 15, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

ro0NL commented Sep 15, 2017

Uh oh!

welcoMattic commented Sep 18, 2017

Uh oh!

lyrixx commented Sep 18, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nicolas-grekas left a comment

Choose a reason for hiding this comment

Uh oh!

nicolas-grekas commented Sep 26, 2017

Uh oh!

lyrixx commented Sep 26, 2017

Uh oh!

nicolas-grekas commented Sep 26, 2017

Uh oh!

lyrixx commented Sep 26, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nicolas-grekas commented Sep 26, 2017

Uh oh!

nicolas-grekas commented Sep 26, 2017

Uh oh!

lyrixx commented Sep 26, 2017

Uh oh!

welcoMattic commented Sep 26, 2017 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

nicolas-grekas commented Oct 5, 2017

Uh oh!

nicolas-grekas commented Oct 5, 2017

Uh oh!

Uh oh!

welcoMattic commented Sep 14, 2017 •

edited by nicolas-grekas

Loading

dunglas commented Sep 15, 2017 •

edited

Loading

dunglas commented Sep 15, 2017 •

edited

Loading

nicolas-grekas commented Sep 15, 2017 •

edited

Loading

nicolas-grekas commented Sep 15, 2017 •

edited

Loading

lyrixx commented Sep 18, 2017 •

edited

Loading

lyrixx commented Sep 26, 2017 •

edited

Loading

welcoMattic commented Sep 26, 2017 •

edited

Loading