[Serializer] Putting the serializer component on steroids #7

mtarld · 2023-07-10T08:09:54Z

Q	A
Branch?	7.0
Bug fix?	no
New feature?	yes
Deprecations?	no
Tickets
License	MIT
Doc PR	TODO

This PR introduces a whole redesign of the Serializer component.

Why?

The Serializer component is critically important as it lies on the hot path for a good number of Symfony-powered apps out there. Yet there are quite a lot of known issues in the current implementation, notably:

Data shapes get computed every time (de)serialization happens, which is very expensive as it implies resource-intensive calls such as reflection.
Each time the serializer is called, normalizers and encoders are tried until one supporting the given data is found. While this pattern works well when looping through a relatively little amount of services (e.g. security authenticators), it rapidly becomes costly as the number of normalizers/encoders grows, even though the situation has been slightly improved in 6.3 with the addition of getSupportedTypes().
Plus, this design makes debugging hard, especially using nested normalizers.
Core normalizers make use of inheritance which leads to maintenance headaches.
This issue has been spotted a long time ago ([RFC][Serializer] Serializer redesign symfony/symfony#19330) and the ongoing refactoring is struggling ([Serializer][PropertyInfo][PropertyAccess] Refactoring plan for AbstractObjectNormalizer and related code symfony/symfony#30818)
Some features should've rather been left to userland and community packages, and there are often ways to achieve the same thing, sometimes less efficiently and/or less future-proof (e.g. [Serializer] Custom Normalizer broken after upgrading to 6.1 symfony/maker-bundle#1252). These add unnecessary complexity to the codebase which increases the maintenance burden.
The whole normalized data is at one point stored in memory, this can cause memory issues when dealing with huge collections.

This refactoring changes some existing features with backward compatibility in mind, improving the code as well as its public API, performance, and memory usage.

Main ideas

Cache

The main trick is to rely on the cache.
During cache warm-up (or on the fly once and for all), the data structure is computed and used to generate a cache PHP file that we could name "template".
Then, the generated template is called with the actual data to deal with serialization/deserialization.

Template generation is the main costly thing. And because the template is computed and written once, then only the template execution will be done all the other times, which implies lighting speed!

Here is the workflow during runtime:

cache miss	cache hit
search for template	search for template
build data model	execute template file
→ scalar nodes
→ collection nodes
→ object nodes
→→ load properties metadata (reflection, attributes, ...)
build template PHP AST
optimize template PHP AST
compile template PHP AST
write template file
execute template file

Streaming

To improve memory usage, serialization, and deserialization are relying on resources.
In that way, the whole serialized data will never be at once in memory as it is a stream.

Here is for example a serialization template PHP file:

<?php

/**
 * @param Symfony\Component\Serializer\Tests\Fixtures\Dto\ClassicDummy $data
 * @param resource $resource
 */
return static function (mixed $data, mixed $resource, \Symfony\Component\Serializer\Serialize\Config\SerializeConfig $config, \Psr\Container\ContainerInterface $services): void {
    \fwrite($resource, "{\"@id\":");
    \fwrite($resource, \json_encode($data->id, $config->json()->flags()));
    \fwrite($resource, ",\"name\":");
    \fwrite($resource, \json_encode(strtoupper($data->name), $config->json()->flags()));
    \fwrite($resource, "}");
};

And here is for example a (eager) deserialization template PHP file:

<?php

/**
 * @param resource $resource
 * @return iterable<int, int>
 */
return static function (mixed $resource, \Symfony\Component\Serializer\Deserialize\Config\DeserializeConfig $config, \Symfony\Component\Serializer\Deserialize\Instantiator\InstantiatorInterface $instantiator, \Psr\Container\ContainerInterface $services): mixed {
    $providers["iterable<int, mixed>"] = static function (?iterable $data) use ($config, $instantiator, &$providers): iterable {
        $iterable = static function (iterable $data) use ($config, $instantiator, &$providers): iterable {
            foreach ($data as $k => $v) {
                yield $k => ($providers["mixed"])($v);
            }
        };
        return ($iterable)($data);
    };
    $providers["int"] = static function (mixed $data) use ($config, $instantiator, &$providers): mixed {
        try {
            return (int) ($data);
        } catch (\Throwable $e) {
            throw new \Symfony\Component\Serializer\Exception\UnexpectedValueException(sprintf("Cannot cast \"%s\" to \"int\"", get_debug_type($data)));
        }
    };
    return ($providers["iterable<int, int>"])(\Symfony\Component\Serializer\Deserialize\Decoder\JsonDecoder::decode($resource, 0, -1, $config));
};

Laziness

To go further with memory improvements, laziness can be configured for either the data reading or the object instantiation during deserialization.

Lazy reading

Reading the data lazily means returning data before having read the whole serialized data.
Indeed, for lists and dictionaries, only boundaries are read, and the actual data will be read if asked.

For example, let's say that you have a list of 10 elements, and you iterate over that list until you're at the fifth element.
By doing that, the deserializer will parse the boundaries of the 4 first elements, parse the content of only the fifth one, and purely ignores the rest.

As another example, let's say you have a dictionary containing two items, foo and bar.
foo's value is huge and complex and bar's value is tiny.
And let's say you want to read only bar's value.
With laziness, only the boundaries of foo will be parsed, whereas bar will be fully parsed.

This can be powerful when combined with lazy instantiation.

At the time, only JSON supports lazy reading.

Lazy instantiation

The new serializer implementation leverages the lazy ghosts of the var exporter component.
Indeed, because we can read the serialized data lazily, we can therefore create a lazy ghost with callables that will read the serialized data on demand.

By doing that, the serialized data won't be read unless explicitly accessed by the object property.

For example, let's say we have a Dummy with a tiny name and a huge description.
By leveraging lazy reading and lazy instantiation, a Dummy object can be instantiated without reading the serialized name data and the serialized description data.
And if we need to access the name, only the serialized name data will be read.

Here is an example of a lazy reading deserialization template PHP file:

<?php

/**
 * @param resource $resource
 * @return Symfony\Component\Serializer\Tests\Fixtures\Dto\Dummy
 */
return static function (mixed $resource, \Symfony\Component\Serializer\Deserialize\Config\DeserializeConfig $config, \Symfony\Component\Serializer\Deserialize\Instantiator\InstantiatorInterface $instantiator, \Psr\Container\ContainerInterface $services): mixed {
    $providers["Symfony\\Component\\Serializer\\Tests\\Fixtures\\Dto\\Dummy"] = static function (mixed $resource, int $offset, int $length) use ($config, $instantiator, &$providers): Symfony\Component\Serializer\Tests\Fixtures\Dto\Dummy {
        $boundaries = \Symfony\Component\Serializer\Deserialize\Splitter\JsonSplitter::splitDict($resource, $offset, $length);
        $properties = [];
        foreach ($boundaries as $k => $b) {
            if ("name" === $k) {
                $properties["name"] = static function () use ($resource, $b, $config, $instantiator, &$providers): mixed {
                    return ($providers["string"])($resource, $b[0], $b[1]);
                };
                continue;
            }
            if ("description" === $k) {
                $properties["description"] = static function () use ($resource, $b, $config, $instantiator, &$providers): mixed {
                    return ($providers["string"])($resource, $b[0], $b[1]);
                };
                continue;
            }
        }

        // $instantiator is a Symfony\Component\Serializer\Deserialize\Instantiator\LazyInstantiator instance
        return $instantiator->instantiate("Symfony\\Component\\Serializer\\Tests\\Fixtures\\Dto\\Dummy", $properties);
    };
    $providers["string"] = static function (mixed $resource, int $offset, int $length) use ($config, $instantiator, &$providers): mixed {
        $data = \Symfony\Component\Serializer\Deserialize\Decoder\JsonDecoder::decode($resource, $offset, $length, $config);
        try {
            return (string) ($data);
        } catch (\Throwable $e) {
            throw new \Symfony\Component\Serializer\Exception\UnexpectedValueException(sprintf("Cannot cast \"%s\" to \"string\"", get_debug_type($data)));
        }
    };
    return ($providers["Symfony\\Component\\Serializer\\Tests\\Fixtures\\Dto\\Dummy"])($resource, 0, -1);
};

While it can have huge benefits in terms of memory, it most of the time costs in terms of performance.
Of course, there is no silver bullet and the laziness must be set depending on the data we are dealing with.
Indeed, for small data, eagerness must most of the time be preferred whereas for big data, laziness must most of the time be chosen.

Configuration and context

Contrary to the actual serializer implementation, a difference has been made between "configuration" and "context".

The configuration is meant to be provided by the developer when calling the serializer/deserializer.
It is an immutable object that can be extended in the userland.
That object is strongly typed, validated, and documented, which implies trustability, autocompletion, and IDE understanding.
The context can be compared to runtime serialization/deserialization information.
It is internal and isn't meant to be manipulated by the developer.
It is a basic hashmap such as the previous context.

<?php

- use Symfony\Component\Serializer\Serialize\Config\SerializeConfig;
- use Symfony\Component\Serializer\Serialize\Config\CsvSerializeConfig;
+ use Symfony\Component\Serializer\Encoder\CsvEncoder;
+ use Symfony\Component\Serializer\Normalizer\AbstractNormalizer;

- $context = [
-     AbstractNormalizer::GROUPS => ['foo', 'bar'],
-     CsvEncoder::DELIMITER => '#',
- ];
+ $csvConfig = (new CsvSerializeConfig())->withDelimiter('#');
+ $config = (new SerializeConfig())
+     ->withGroups(['foo', 'bar'])
+     ->withCsvConfig($csvConfig);

- $serializer->serialize(new Dummy(), 'json', $context);
+ $serializer->serialize(new Dummy(), 'json', config: $config);

Generics support

To be able to generate templates that stick as much as possible to the data model, the support of generics has been introduced.
It relies on PHPStan, which is an optional dependency. If the library is missing, the support of generics won't be enabled.

But as soon as the PHPStan library is available, the serializer can read PHP tags like @var array<string, list<float>> to generate the proper data model.

Furthermore, it can read @template tags and therefore generate the proper data models for properties with tags like @var list<T>.

For example, let's say you have a Collection class that has generic types:

<?php

/**
 * @template T
 */
class Collection
{
    /** @var iterable<T> */
    public iterable $items;
}

Then, during serialization/deserialization of Collection, if the given type is Collection<int>, it'll generate a template that handles items as an iterable list of integers.

API

Contrary to the actual Symfony\Component\Serializer\SerializerInterface, which has two methods serialize and deserialize, the new design will instead introduce two interfaces.

These are composing the main part of the available API.

<?php

use Symfony\Component\Serializer\Serialize\Config\SerializeConfig;
use Symfony\Component\Serializer\Stream\StreamInterface;

interface SerializerInterface
{
    public function serialize(mixed $data, string $format, StreamInterface $output = null, SerializeConfig $config = null): string|null;
}

<?php

use Symfony\Component\Serializer\Deserialize\Config\DeserializeConfig;
use Symfony\Component\Serializer\Stream\StreamInterface;
use Symfony\Component\Serializer\Type\Type;

interface DeserializerInterface
{
    public function deserialize(StreamInterface|string $input, Type $type, string $format, DeserializeConfig $config = null): mixed;
}

BC & Upgrade Path

The redesign implementation will be directly in the actual component but mainly under two new namespaces: Symfony\Component\Serializer\Serialize and Symfony\Component\Serializer\Deserialize.

Indeed, most of the time when you inject the Symfony\Component\Serializer\SerializerInterface, it is to do either serialization or deserialization, but not both.

That's why the following interfaces can be added without conflicting with the actual one: Symfony\Component\Serializer\Serialize\SerializerInterface, Symfony\Component\Serializer\Deserialize\DeserializerInterface.

Moreover, for most of the basic use cases, almost nothing will have to be changed to have the new serializer working:

<?php

- use Symfony\Component\Serializer\Annotation\Groups;
+ use Symfony\Component\Serializer\Attribute\Groups;
- use Symfony\Component\Serializer\Annotation\MaxDepth;
+ use Symfony\Component\Serializer\Attribute\MaxDepth;
- use Symfony\Component\Serializer\Annotation\SerializedName;
+ use Symfony\Component\Serializer\Attribute\SerializedName;

class Dummy
{
    #[SerializedName('@id')]
    public int $id;

    #[Groups('details')]
    public string $description;

    #[MaxDepth(1)]
    public self $related;
}

<?php

- use Symfony\Component\Serializer\SerializerInterface;
+ use Symfony\Component\Serializer\Deserialize\DeserializerInterface;
+ use Symfony\Component\Serializer\Serialize\SerializerInterface;
+ use Symfony\Component\Serializer\Type\Type;

final class MyService
{
    public function __construct(
        private readonly SerializerInterface $serializer,
+       private readonly DeserializerInterface $deserializer,
    ) {
    }

    public function __invoke(): void
    {
        $this->serializer->serialize(new Dummy(), 'json');
-       $this->serializer->deserialize('...', Dummy::class, 'csv');
+       $this->deserializer->deserialize('...', Type::class(Dummy::class), 'csv');
    }
}

Plus, by leveraging the Symfony\Component\Serializer\Serialize\Template\NormalizerEncoderTemplateGenerator (see the latter section about extension points), it is possible to take advantage of every already existing encoder.

Performance showcase

With all these ideas, performance has been greatly improved.

When serializing 10k objects to JSON, it is about 5 times faster than the legacy.

And it consumes about 2.5 times less memory.

When deserializing a 10Mb JSON file to a list of objects, iterating one the 9999 firsts and reading the 10000th eagerly, it is more than 10 times faster than the legacy deserialization, and even slightly faster than json_decode!

Reading lazily is slower but still 3 times faster than the legacy implementation.

In terms of memory consumption, the new implementation is comparable to the legacy one when reading eagerly.

And when reading lazily, it consumes about 8 times less memory!

And it doesn't stop there, indeed @dunglas is working on a PHP extension compatible with that new version of the component leveraging simdjson to make JSON serialization/deserialization even faster.

These improvements will benefit several big projects such as Drupal, Sylius, and API Platform (some integration tests already have been made for this).
And it'll as well benefit many other tiny projects as many projects are dealing with serialization.

Usage example

Install the component with the generics support:

composer require symfony/serializer phpdocumentor/reflection-docblock phpstan/phpdoc-parser

Configure the component-related parameters:

# config/packages/serializer.yaml

framework:
    serializer:
        # defines where to find serializable classes
        # this is useful to warm up the template cache as efficiently as possible
        # but as well to retrieve runtime services
        serializable_paths:
            - src/{Dto,ValueObject}
            - legacy/Model

        # whether to use the lazy instantiator service or the eager one
        # can as well be overridden using decoration or service replacement
        lazy_instantiation: false

        # wheter to read the serialized data eagerly or lazily
        # can as well be overridden using the DeserializeConfig
        lazy_deserialization: false

        # defines which formats that will be warmed up
        formats: [json, csv]

Configure PHP attributes:

<?php

use Symfony\Component\DependencyInjection\Attribute\Autowire;
use Symfony\Component\Serializer\Attribute\DeserializeFormatter;
use Symfony\Component\Serializer\Attribute\Groups;
use Symfony\Component\Serializer\Attribute\MaxDepth;
use Symfony\Component\Serializer\Attribute\SerializedName;
use Symfony\Component\Serializer\Attribute\SerializeFormatter;

class Dummy
{
    #[SerializedName('@id')]
    public int $id;

    #[SerializeFormatter('strtoupper')]
    public string $name;

    #[Groups('details')]
    public string $description;

    #[SerializeFormatter([self::class, 'unscaleAndCastToString'])]
    #[DeserializeFormatter([self::class, 'scaleAndCastToInt'])]
    public int $price;

    #[MaxDepth(1, [self::class, 'toLink'])]
    public self $related;

    public static function unscaleAndCastToString(string $value, ScalerInterface $scaler, CustomSerializeConfig $config): int
    {
        return (int) $scaler->unscale($value, $config->scale());
    }

    public static function scaleAndCastToInt(int $value, ScalerInterface $scaler, CustomDeserializeConfig $config): string
    {
        return (string) $scaler->scale($value, $config->scale());
    }

    public static function toLink(self $value, #[Autowire('link_converter')] LinkConverterInterface $linkConverter): string
    {
        return $linkConverter->toLink($value);
    }
}

Then use the serializer/deserializer:

<?php

use Symfony\Component\Serializer\Deserialize\Config\DeserializeConfig;
use Symfony\Component\Serializer\Deserialize\DeserializerInterface;
use Symfony\Component\Serializer\Serialize\Config\SerializeConfig;
use Symfony\Component\Serializer\Serialize\SerializerInterface;
use Symfony\Component\Serializer\Stream\MemoryStream;
use Symfony\Component\Serializer\Type\Type;

final class MyService
{
    public function __construct(
        private readonly SerializerInterface $serializer,
        private readonly DeserializerInterface $deserializer,
    ) {
    }

    public function __invoke(): void
    {
        // serialize dummy to JSON
        $serialized = $this->serializer->serialize(new Dummy(), 'json');

        // serialize to a resource
        $this->serializer->serialize(new Dummy(), 'json', $output = new MemoryStream());
        stream_get_contents($output->resource());

        // serialize a stringable dummy as a string
        $config = (new SerializeConfig())->withType(Type::string());
        $serialized = $this->serializer->serialize(new StringableDummy(), 'json', config: $config);

        // serialize collection with generics
        $config = (new SerializeConfig())->withType(Type::class(Collection::class, genericParameterTypes: [Type::class(Dummy::class)]));
        $config = (new SerializeConfig())->withType(Type::fromString(Collection::class.'<'.Dummy::class.'>')); // same as above
        $serialized = $this->serializer->serialize(new Collection([new Dummy(), new Dummy()]), 'json', config: $config);

        // serialize dummy according to groups
        $config = (new SerializeConfig())->withGroups(['foo', 'bar']);
        $serialized = $this->serializer->serialize(new StringableDummy(), 'json', config: $config);

        // deserialize JSON to dummy
        $dummy = $this->deserializer->deserialize('...', Type::class(Dummy::class), 'json');

        // deserialize JSON resource to dummy
        $input = new MemoryStream('{...}');
        $dummy = $this->deserializer->deserialize($input, Type::class(Dummy::class), 'json');

        // deserialize JSON to a collection with generics
        $collection = $this->deserializer->deserialize('...', Type::fromString(Collection::class.'<'.Dummy::class.'>'), 'json');

        // deserialize JSON to dummy while reading JSON lazily
        $config = (new DeserializeConfig())->withLazy(true);
        $dummy = $this->deserializer->deserialize('...', Type::class(Dummy::class), 'json', $config);

        // deserialize CSV to dummy with a specific CSV delimiter
        $csvConfig = (new CsvDeserializeConfig())->withDelimiter('#');
        $config = (new DeserializeConfig())->withCsvConfig($csvConfig);
        $dummy = $this->deserializer->deserialize('...', Type::class(Dummy::class), 'csv', $config);
    }
}

Extension points

A list of available extension points

Serialization

Config classes

To add extra data that will be used during the serialization process, such as we could have done using the $context argument of the actual serializer, we can extend the Symfony\Component\Serializer\Serialize\Config\SerializeConfig class to add our data in a strongly typed way:

<?php

use Symfony\Component\Serializer\Serialize\Config\SerializeConfig;

final class CustomSerializeConfig extends SerializeConfig
{
    protected int $scale = 100;

    public function scale(): int
    {
        return $this->scale;
    }

    public function withScale(int $scale): static
    {
        if ($scale < 0) {
            throw new \InvalidArgumentException('The scale must be positive.');
        }

        $clone = clone $this;
        $clone->scale = $scale;

        return $clone;
    }
}

Then, we just need to provide that configuration when processing serialization:

<?php

$config = (new CustomSerializeConfig())->withScale(200);
$serializer->serialize(new Dummy(), 'json', config: $config);

The same goes for format-related configs such as Symfony\Component\Serializer\Serialize\Config\CsvSerializeConfig and Symfony\Component\Serializer\Serialize\Config\JsonSerializeConfig:

<?php

$jsonConfig = (new CustomJsonSerializeConfig())->withIndentation(8);
$config = (new SerializeConfig())->withJsonConfig($jsonConfig);

$serializer->serialize(new Dummy(), 'json', config: $config);

`NormalizerEncoderTemplateGenerator`

The easiest way to add a serialization format is to leverage the Symfony\Component\Serializer\Serialize\Template\NormalizerEncoderTemplateGenerator.
Indeed, this template generator generates a template that serializes data by normalizing it and then encoding it using an encoder.

By doing that, you'll leverage the whole system of normalization (which is the data model graph building in the new implementation).
So you'll just have to implement an encoder:

<?php

use Symfony\Component\Serializer\Serialize\Encoder\EncoderInterface;
use Symfony\Component\Serializer\Serialize\Config\SerializeConfig;

final class NativeEncoder implements EncoderInterface
{
    public static function encode(mixed $resource, mixed $normalized, SerializeConfig $config): void
    {
        fwrite($resource, serialize($normalized));
    }
}

And then add a new template generator service:

services:
    app.serialize.template_generator.native:
        class: Symfony\Component\Serializer\Serialize\Template\NormalizerEncoderTemplateGenerator
        arguments:
            $encoderClassName: App\Serializer\Serialize\NativeEncoder
        tags:
            - { name: serializer.serialize.template_generator, format: native }

And you'll be able to use that format:

<?php

$serializer->serialize(new Dummy(), 'native');

`TemplateGeneratorInterface`

Another way to support a format is to implement its own Symfony\Component\Serializer\Serialize\Template\TemplateGeneratorInterface.

The template generator has to generate the whole serialization PHP syntax tree based on a given data model.

To ease a template generator implementation, it is possible to extend the Symfony\Component\Serializer\Serialize\Template\TemplateGenerator abstract class.
For example:

<?php

use Symfony\Component\Serializer\Serialize\Config\SerializeConfig;
use Symfony\Component\Serializer\Serialize\Template\TemplateGenerator;
use Symfony\Component\Serializer\Serialize\DataModel\CollectionNode;
use Symfony\Component\Serializer\Serialize\DataModel\DataModelNodeInterface;
use Symfony\Component\Serializer\Serialize\DataModel\ObjectNode;
use Symfony\Component\Serializer\Php\ForEachNode;
use Symfony\Component\Serializer\Php\AssignNode;
use Symfony\Component\Serializer\Php\FunctionNode;
use Symfony\Component\Serializer\Php\TemplateStringNode;
use Symfony\Component\Serializer\Php\ExpressionNode;
use Symfony\Component\Serializer\Php\VariableNode;
use Symfony\Component\Serializer\Php\ScalarNode;
use Symfony\Component\Serializer\Type\TypeExtractorInterface;

final class FullCustomTemplateGenerator extends TemplateGenerator
{
    public function __construct(
        private readonly TypeExtractorInterface $typeExtractor,
    ) {
    }

    public function doGenerate(DataModelNodeInterface $node, SerializeConfig $config, array $context): array
    {
        if ($node instanceof CollectionNode) {
            $prefixName = $this->scopeVariableName('prefix', $context);
            $keyName = $this->scopeVariableName('key', $context);

            return [
                new ExpressionNode(new FunctionNode('\fwrite', [new VariableNode('resource'), new ScalarNode('COLLECTION(')])),
                new ExpressionNode(new AssignNode(new VariableNode($prefixName), new ScalarNode(''))),

                new ForEachNode($node->accessor, $keyName, $node->childrenNode->accessor, [
                    new ExpressionNode(new AssignNode(new VariableNode($keyName), new VariableNode($keyName))),
                    new ExpressionNode(new FunctionNode('\fwrite', [new VariableNode('resource'), new TemplateStringNode(
                        new VariableNode($prefixName),
                        'KEY(',
                        new VariableNode($keyName),
                        ') = ',
                    )])),
                    ...$this->generate($node->childrenNode, $config, $context),
                    new ExpressionNode(new AssignNode(new VariableNode($prefixName), new ScalarNode(' - '))),
                ]),

                new ExpressionNode(new FunctionNode('\fwrite', [new VariableNode('resource'), new ScalarNode(')')])),
            ];
        }

        if ($node instanceof ObjectNode) {
            // ...

            return $nodes;
        }

        return [
            new ExpressionNode(new FunctionNode('\fwrite', [new VariableNode('resource'), $node->accessor])),
        ];
    }
}

This extension isn't simple to deal with and is mostly adapted to a few tricky use cases.

`PropertyMetadataLoaderInterface`

The Symfony\Component\Serializer\Serialize\DataModel\DataModelBuilderInterface default implementation calls a Symfony\Component\Serializer\Serialize\Mapping\PropertyMetadataLoaderInterface to retrieve object's properties, with their name, their type, and their formatters.

Therefore, it is possible to decorate (or replace) the Symfony\Component\Serializer\Serialize\Mapping\PropertyMetadataLoaderInterface.
In that way, it'll be possible for example to read extra custom PHP attributes, ignore specific object's properties, and rename every properties, ...

As an example, in the component, there are:

The PropertyMetadataLoader which reads basic properties information.
The AttributePropertyMetadataLoader which reads properties attributes such as Groups, SerializedName, SerializedFormatter and MaxDepth to ignore, rename or add formatters on the already retrieved properties
The TypePropertyMetadataLoader which updates properties' types according to generics, and cast date-times to strings

For example, you can hide sensitive data of sensitive classes and a sensitive marker:

<?php

use Symfony\Component\Serializer\Serialize\Config\SerializeConfig;
use Symfony\Component\Serializer\Serialize\Mapping\PropertyMetadata;
use Symfony\Component\Serializer\Serialize\Mapping\PropertyMetadataLoaderInterface;
use Symfony\Component\Serializer\Type\Type;

final class CustomPropertyMetadataLoader implements PropertyMetadataLoaderInterface
{
    public function __construct(
        private readonly PropertyMetadataLoaderInterface $decorated,
    ) {
    }

    public function load(string $className, SerializeConfig $config, array $context): array
    {
        $result = $this->decorated->load($className, $config, $context);
        if (!is_a($className, SensitiveInterface::class, true)) {
            return $result;
        }

        foreach ($result as &$metadata) {
            if ('sensitive' === $metadata->name()) {
                $metadata = $metadata
                    ->withType(Type::string())
                    ->withFormatter(self::hideData(...));
            }
        }

        $result['is_sensitive'] = new PropertyMetadata(
            name: 'wontBeUsed',
            type: Type::bool(),
            formatters: [self::true()],
        );

        return $result;
    }

    public static function hideData(mixed $value): string
    {
        return hash('xxh128', json_encode($value));
    }

    public static function true(): bool
    {
        return true;
    }
}

This extension point is kind of easy to use and powerful.
It can be compared to the way normalizers are used nowadays in terms of "data content"

`DataModelBuilderInterface`

It is as well possible to decorate (or replace) the Symfony\Component\Serializer\Serialize\DataModel\DataModelBuilderInterface implementation.

Indeed, the role of that service is to create a data model graph of a given type.
And this data model will be used by the Symfony\Component\Serializer\Serialize\Template\TemplateGeneratorInterface to generate a PHP serialization template.

So, by modifying the way the data model is created, we can modify the shape of the type representation and therefore the final shape of the serialized data.

This extension isn't simple to deal with and is mostly adapted to a few tricky use cases.

`TemplateVariation`

Several data structures can be related to a single type depending on the serialization config.
If a custom configuration alters the data structure, we must handle using template variations.

To do that, we must create our own Symfony\Component\Serializer\Template\TemplateVariation:

<?php

use Symfony\Component\Serializer\Deserialize\Config\DeserializeConfig;
use Symfony\Component\Serializer\Serialize\Config\SerializeConfig;
use Symfony\Component\Serializer\Template\TemplateVariation;

readonly class CustomTemplateVariation extends TemplateVariation
{
    public function __construct(string $role)
    {
        parent::__construct('role', $role);
    }

    /**
     * @param CustomSerializeConfig|CustomDeserializeConfig $config
     */
    public function configure(SerializeConfig|DeserializeConfig $config): SerializeConfig|DeserializeConfig
    {
        if (!$config instanceof CustomSerializeConfig) {
            return $config;
        }

        return $config->withRole($config->role());
    }
}

Then, it is needed to decorate the Symfony\Component\Serializer\Template\TemplateVariationExtractorInterface to handle that new variation:

<?php

use Symfony\Component\Serializer\Type\Type;
use Symfony\Component\Serializer\Template\TemplateVariation;
use Symfony\Component\Serializer\Template\TemplateVariationExtractorInterface;

final class CustomTemplateVariationExtractor implements TemplateVariationExtractorInterface
{
    public function __construct(
        private readonly TemplateVariationExtractorInterface $decorated,
    ) {
    }

    public function extractVariationsFromType(Type $type): array
    {
        $decorated = $this->decorated->extractVariationsFromType($type);
        if (!$type->isObject() || !$type->hasClass()) {
            return $decorated;
        }

        $roles = [];

        foreach ((new \ReflectionClass($className))->getProperties() as $reflectionProperty) {
            $reflectionAttribute = $reflectionProperty->getAttributes(Role::class, \ReflectionAttribute::IS_INSTANCEOF)[0] ?? null;
            if (null === $reflectionAttribute) {
                continue;
            }

            $roles[] = $reflectionAttribute->newInstance()->role;
        }

        $rolesVariations = array_map(fn (string $r): TemplateVariation => new RoleTemplateVariation($r), array_values(array_unique($roles)));

        array_push($decorated, ...$rolesVariations);

        return $decorated;
    }

    /**
     * @param CustomSerializeConfig|CustomDeserializeConfig $config
     */
    public function extractVariationsFromConfig(SerializeConfig|DeserializeConfig $config): array
    {
        $decorated = $this->extractVariationsFromConfig($config);
        if (!$config instanceof CustomSerializeConfig) {
            return $decorated;
        }

        $decorated[] = new RoleTemplateVariation($config->role());

        return $decorated;
    }
}

Most of the time, it's a bad idea to alter the data structure depending on the
serialization configuration, it is rather recommended to use an adapted and dedicated DTO
during serialization.

Deserialization

The Config classes, TemplateGeneratorInterface, PropertyMetadataLoaderInterface, DataModelBuilderInterface, and the TemplateVariation extension points are quite similar to serialization one.

`EagerTemplateGenerator`

The easiest way to add a deserialization eager format is to leverage the Symfony\Component\Serializer\Deserialize\Template\EagerTemplateGenerator.

To do that, you'll just have to implement a decoder:

<?php

use Symfony\Component\Serializer\Deserialize\Decoder\DecoderInterface;
use Symfony\Component\Serializer\Deserialize\Config\DeserializeConfig;
use Symfony\Component\Serializer\Exception\InvalidResourceException;

final class NativeDecoder implements DecoderInterface
{
    public static function decode(mixed $resource, int $offset, int $length, DeserializeConfig $config): mixed
    {
        if (false === $content = @stream_get_contents($resource, $length, $offset)) {
            throw new InvalidResourceException($resource);
        }

        try {
            return unserialize($content);
        } catch (\Throwable) {
            throw new InvalidResourceException($resource);
        }
    }
}

And then add a new template generator service:

services:
    app.deserialize.template_generator.native.eager:
        class: Symfony\Component\Serializer\Deserialize\Template\EagerTemplateGenerator
        arguments:
            $decoderClassName: App\Serializer\Deserialize\NativeDecoder
        tags:
            - { name: serializer.deserialize.template_generator.eager, format: native }

And you'll be able to use that format eagerly:

<?php

$deserializer->deserialize(new Dummy(), Type::class(Dummy::class), 'native');

`LazyTemplateGenerator`

To add a new lazy deserialization format, you can leverage the Symfony\Component\Serializer\Deserialize\Template\LazyTemplateGenerator.

To do that, you'll just have to implement a decoder (like we've done in the previous section).

And to implement a splitter:

<?php

use Symfony\Component\Serializer\Deserialize\Splitter\SplitterInterface;
use Symfony\Component\Serializer\Deserialize\Config\DeserializeConfig;
use Symfony\Component\Serializer\Exception\InvalidResourceException;

final class NativeSplitter implements SplitterInterface
{
    public static function splitList(mixed $resource, int $offset = 0, int $length = -1): ?\Iterator
    {
        // have a look at Symfony\Component\Serializer\Deserialize\Splitter\JsonSplitter::splitList for an example
    }

    public static function splitDict(mixed $resource, int $offset = 0, int $length = -1): ?\Iterator
    {
        // have a look at Symfony\Component\Serializer\Deserialize\Splitter\JsonSplitter::splitDict for an example
    }
}

Splitting is quite complicated because it requires lexing. If it isn't done
properly, performances can quickly become bad.

And then add a new template generator service:

services:
    app.deserialize.template_generator.native.lazy:
        class: Symfony\Component\Serializer\Deserialize\Template\LazyTemplateGenerator
        arguments:
            $decoderClassName: App\Serializer\Deserialize\NativeDecoder
            $splitterClassName: App\Serializer\Deserialize\NativeSplitter
        tags:
            - { name: serializer.deserialize.template_generator.lazy, format: native }

And you'll be able to use that format lazily:

<?php

$deserializer->deserialize(new Dummy(), Type::class(Dummy::class), 'native', (new DeserializeConfig())->withLazy());

`InstantiatorInterface`

At some point, the template has at his disposal a class name and a dictionary of callables corresponding to properties.

Then it has to instantiate a new object with that. To do that, it'll call a Symfony\Component\Serializer\Deserialize\Instantiator\InstantiatorInterface, which is another extension point.

Indeed, it is possible to decorate (or replace) that instantiator to add/remove an object property (or even change object class!):

<?php

use Symfony\Component\Serializer\Deserialize\Instantiator\InstantiatorInterface;

final class CustomInstantiator implements InstantiatorInterface
{
    public function __construct(
        private readonly InstantiatorInterface $decorated,
        private readonly ClockInterface $clock,
    ) {
    }

    public function instantiate(string $className, array $properties): object
    {
        $properties['instantiatedAt'] = fn () => $this->clock->now();
        unset($properties['hidden']);

        return $this->instantiator->instantiate($className, $properties);
    }
}

Thoughts about Type and the PropertyInfo component

Under the Symfony\Component\Serializer\Type namespace, you can find a bunch of stuff related to types and ways to retrieve them.
It is quite similar to what the PropertyInfo component does but goes further in terms of types.

Indeed, the Serializer component can extract types from properties, such as the PropertyInfo component does but can also extract types from function return and parameters.

This makes me things about widening the PropertyInfo component, and therefore maybe renaming it (like "ClassMemberInfo" or whatever), to handle these use cases as well.

fixes symfony#50511

…he monorepo (fabpot, dunglas, KorvinSzanto, xabbuh, aimeos, ahundiak, Danielss89, rougin, csunolgomez, Jérôme Parmentier, mtibben, Nyholm, ajgarlag, uphlewis, samnela, grachevko, nicolas-grekas, tinyroy, danizord, Daniel Degasperi, rbaarsma, Ekman, 4rthem, derrabus, mleczakm, iluuu1994, Tobion, chalasr, lemon-juice, franmomu, cidosx, erikn69, AurelienPillevesse) This PR was merged into the 6.4 branch. Discussion ---------- [PsrHttpMessageBridge] Import the bridge into the monorepo | Q | A | ------------- | --- | Branch? | 6.4 | Bug fix? | no | New feature? | yes | Deprecations? | no | Tickets | N/A | License | MIT | Doc PR | TODO ⚠️ Don't squash! I propose to import the `symfony/psr-http-message-bridge` package into the Symfony monorepo for further maintenance. Commits ------- e40dd66 [PsrHttpMessageBridge] Patch return types and fix CS 266c09f [PsrHttpMessageBridge] Import the bridge into the monorepo 0c0323a Add 'src/Symfony/Bridge/PsrHttpMessage/' from commit '581ca6067eb62640de5ff08ee1ba6850a0ee472e' 581ca60 Prepare release 2.3.1 45d0349 Fix CS 6410dda bug symfony#122 Don't rely on Request::getPayload() to populate the parsed body (nicolas-grekas) ef03b6d Don't rely on Request::getPayload() to populate the parsed body 3c62b81 minor symfony#120 Prepare release 2.3.0 (derrabus) 96acbfd Prepare release 2.3.0 7eedd34 feature symfony#119 Implement ValueResolverInterface (derrabus) 0b54b85 Implement ValueResolverInterface 6b2f5df feature symfony#117 Leverage `Request::getPayload()` to populate the parsed body of PSR-7 requests (AurelienPillevesse) 3a8caad Leverage `Request::getPayload()` to populate the parsed body of PSR-7 requests 18c9e82 minor symfony#118 Add native types where possible (derrabus) 4fd4323 Add native types where possible 28a732c minor symfony#115 Prepare the 2.2.0 release (derrabus) 7944831 cs fix 99ddcaa Prepare the 2.2.0 release 8a5748d feature symfony#113 Bump psr/http-message version (erikn69) ec83c1c Bump psr/http-message version 694016e feature symfony#114 Drop support for Symfony 4 (derrabus) b360b35 Drop support for Symfony 4 998d8d2 minor symfony#111 Adjustments for PHP CS Fixer 3 (derrabus) 5fa5f62 Adjustments for PHP CS Fixer 3 a125b93 minor symfony#110 Add PHP 8.2 to CI (derrabus) 4592df2 Add PHP 8.2 to CI 4617ac3 bug symfony#109 perf: ensure timely flush stream buffers (cidosx) 8c8a75b perf: ensure timely flush stream buffers d444f85 Update changelog 155a7ae bug symfony#107 Ignore invalid HTTP headers when creating PSR7 objects (nicolas-grekas) 9a78a16 Ignore invalid HTTP headers when creating PSR7 objects bdb2871 minor symfony#104 Add missing .gitattributes (franmomu) 808561a Add missing .gitattributes 316f5cb bug symfony#103 Fix for wrong type passed to moveTo() (lemon-juice) 7f3b5c1 Fix for wrong type passed to moveTo() 22b37c8 minor symfony#101 Release v2.1.2 (chalasr) c382d76 Release v2.1.2 c81476c feature symfony#100 Allow Symfony 6 (chalasr) c7a0be3 Allow Symfony 6 df83a38 minor symfony#98 Add PHP 8.1 to CI (derrabus) b2bd334 Add PHP 8.1 to CI 824711c minor symfony#99 Add return types to fixtures (derrabus) f8f70fa Add return types to fixtures d558dcd minor symfony#97 Inline $tmpDir (derrabus) d152649 Inline $tmpDir f12a9e6 minor symfony#96 Run PHPUnit on GitHub Actions (derrabus) ab64c69 Run PHPUnit on GitHub Actions c901299 bug symfony#95 Allow `psr/log` 2 and 3 (derrabus) 8e13ae4 Allow psr/log 2 and 3 26068fa Minor cleanups 87fabb9 Fix copyright year 3d9241f minor symfony#92 remove link to sensio extra bundle which removed psr7 support (Tobion) 7078739 remove link to sensio extra bundle which removed psr7 support 81db2d4 feature symfony#89 PSR HTTP message converters for controllers (derrabus) aa26e61 PSR HTTP message converters for controllers e62b239 minor symfony#91 Fix CS (derrabus) 2bead22 Fix CS 488df9b minor symfony#90 Fix CI failures with Xdebug 3 and test on PHP 7.4/8.0 as well (derrabus) a6697fd Fix CI failures with Xdebug 3 and test on PHP 7.4/8.0 as well c62f7d0 Update branch-alias 51a21cb Update changelog a20fff9 bug symfony#87 Fix populating server params from URI in HttpFoundationFactory (nicolas-grekas) 4933e04 bug symfony#86 Create cookies as raw in HttpFoundationFactory (nicolas-grekas) 66095a5 Fix populating server params from URI in HttpFoundationFactory 42cca49 Create cookies as raw in HttpFoundationFactory cffb3a8 bug symfony#85 Fix BinaryFileResponse with range to psr response conversion (iluuu1994) 5d5932d Fix BinaryFileResponse with range to psr response conversion e44f249 bug symfony#81 Don't normalize query string in PsrHttpFactory (nicolas-grekas) bc25829 Don't normalize query string in PsrHttpFactory df735ec bug symfony#78 Fix populating default port and headers in HttpFoundationFactory (mleczakm) 4f30401 Fix populating default port and headers in HttpFoundationFactory 1309b64 bug symfony#77 fix conversion for https requests (4rthem) e86de3f minor symfony#79 Allow installation on php 8 (derrabus) 9243f93 Allow installation on php 8. d336c73 fix conversion for https requests 126903c Fix format of CHANGELOG.md ce709cd feature symfony#75 Remove deprecated code (fabpot) dfc5238 Remove deprecated code 9d3e80d bug symfony#72 Use adapter for UploadedFile objects (nicolas-grekas) a4f9f6d Use adapter for UploadedFile objects ec7892b Fix CHANGELOG, bump branch-alias 7ab4fe4 minor symfony#70 Updated CHANGELOG (rbaarsma) 9ad4bcc Updated CHANGELOG c4c904a minor symfony#71 Cleanup after bump to Symfony v4.4 (nicolas-grekas) e9a9557 Cleanup after bump to Symfony v4.4 3d10a6c feature symfony#66 Add support for streamed Symfony request (Ekman) df26630 Add support for streamed Symfony request 5aa8ca9 bug symfony#69 Allow Symfony 5.0 (rbaarsma) 1158149 Allow Symfony 5.0 81ae86d Merge branch '1.1' a33352a bug symfony#64 Fixed createResponse (ddegasperi) 7a4b449 minor symfony#65 Fix tests (ajgarlag) 19905b0 Fix tests 580de38 Fixed createResponse 9ab9d71 minor symfony#63 Added links to documentation (Nyholm) 59b9406 Added links to documentation c1cb51c feature symfony#50 Add support for streamed response (danizord) 4133c7a bug symfony#48 Convert Request/Response multiple times (Nyholm) 8564bf7 Convert Request/Response multiple times 7cc1605 Add support for streamed response aebc14b feature symfony#62 bump to PHP 7.1 (nicolas-grekas) 8e10923 bump to PHP 7.1 5e5e0c3 Revert "Undeprecate DiactorosFactory for 1.1" 921f866 Undeprecate DiactorosFactory for 1.1 8592ca3 bug symfony#61 removed 'Set-Cookie' from header when it is already converted to a Symfony header cookie (tinyroy) dd1111e removed 'Set-Cookie' from header when it is already converted to a Symfony header cookie ba672d8 bump branch-alias 5f9a032 typo f2c48c5 fix tests 3a52e44 bug symfony#59 Fix SameSite attribute conversion from PSR7 to HttpFoundation (ajgarlag) 5ee1f8f Fix SameSite attribute conversion from PSR7 to HttpFoundation f6d7d3a bug symfony#58 [Bugfix] Typo header set-sookie (grachevko) 16eb6e1 minor symfony#57 Excluded tests from classmap (samnela) 36a8065 Deprecate DiactorosFactory, use nyholm/psr7 for tests 5076934 bug symfony#54 Fix symfony#51 (compatability issue with zendframework/zend-diactoros ^2.0) (uphlewis) 757ea81 [Bugfix] Typo header set-sookie 25f9c3a Excluded tests from classmap 8ff61e5 Fix compatability issue with "zendframework/zend-diactoros": "^2.0." (symfony#51) 53c15a6 updated CHANGELOG c821241 bumped version to 1.1 f26d01f minor symfony#47 Updated changelog (Nyholm) c2282e3 Updated changelog eddd6c8 feature symfony#43 Create PSR-7 messages using PSR-17 factories (ajgarlag) dd81b4b Create PSR-7 messages using PSR-17 factories f11f173 feature symfony#45 Fixed broken build (Nyholm) 8780dd3 Fixed broken build c2b7579 bug symfony#30 Fix the request target in PSR7 Request (mtibben) 94fcfa5 Fix the request target in PSR7 Request 64640ee minor symfony#38 Run PHP 5.3 tests on Precise (Lctrs) 64c0cb0 Run PHP 5.3 tests on Precise b209840 minor symfony#32 Allow Symfony 4 (dunglas) 97635f1 Allow Symfony 4 147a238 minor symfony#31 test suite compatibility with PHPUnit 6 (xabbuh) f5c46f0 test suite compatibility with PHPUnit 6 66085f2 preparing 1.0 release 533d3e4 added a CHANGELOG for 1.0 14269f9 bug symfony#28 Fix REQUEST_METHOD on symfony request (csunol) 98ab85a Fix REQUEST_METHOD on symfony request 29be4f8 updated LICENCE year d2db47c removed obsolete CHANGELOG file 1c30b17 bug symfony#22 Fixes symfony#16 Symfony Request created without any URI (rougin) a59c572 Fixes symfony#16 Symfony Request created without any URI 7a5aa92 bug symfony#23 Fixes #9 Bridge error when no file is selected (ahundiak, Danielss89) a1a631a Update assert error message e5d62e6 Fixes based on code-review 101b608 Handles null file in createrequest bridge. d16c63c bug symfony#18 Allow multiple calls to Request::getContent() (aimeos) 9624b8b Allow multiple calls to Request::getContent() 9c747c4 Merge pull request symfony#19 from xabbuh/travis-config a388c43 update Travis CI configuration ac5cd86 minor symfony#14 Remove use of deprecated 'deep' parameter in tests (KorvinSzanto) 305c0fe Remove use of deprecated 'deep' parameter 3664dc0 minor #7 Test Diactoros Factory with PHP 5.4 (dunglas) bab1530 Test Diactoros Factory with PHP 5.4 d7660b8 Suggest psr/http-message-implementation dc7e308 removed the branch alias for now as we are pre 1.0 3f8977e feature #1 Initial support (dunglas) ca41146 Initial support 01b110b added the initial set of files

… UrlRewriteModule (derrabus) This PR was merged into the 5.4 branch. Discussion ---------- [HttpFoundation] Fix base URI detection on IIS with UrlRewriteModule | Q | A | ------------- | --- | Branch? | 5.4 | Bug fix? | yes | New feature? | no | Deprecations? | no | Tickets | Workaround for php/php-src#11981 | License | MIT | Doc PR | N/A See the linked PHP issue for details on the issue I'm working around. If I setup a Symfony application in a IIS virtual directory and use UrlRewriteModule to rewrite all requests to my front controller script, routing requests might fail if I spell the virtual directory with once with lowercase and afterwards with uppercase letters or vice versa. This PR detects if we're on IIS with active URL rewriting and switches to a more fuzzy base URL detection in that case. Commits ------- 26aec0f [HttpFoundation] Fix base URI detection on IIS with UrlRewriteModule

…attribute (kbond) This PR was squashed before being merged into the 6.4 branch. Discussion ---------- [DependencyInjection] add `#[AutowireLocator]` attribute | Q | A | ------------- | --- | Branch? | 6.4 | Bug fix? | no | New feature? | yes | Deprecations? | no | Tickets | n/a | License | MIT | Doc PR | todo The `AutowireLocator` attribute allows configuring service locators inline: ```php use Symfony\Component\DependencyInjection\Attribute\AutowireLocator; public function someAction( #[AutowireLocator(RouterInterface::class, SluggerInterface::class)] ContainerInterface $container, ): Response { $container->get(RouterInterface::class); $container->get(SluggerInterface::class); } ``` You can customize the key and have optional services: ```php use Symfony\Component\DependencyInjection\Attribute\AutowireLocator; public function someAction( #[AutowireLocator( router: RouterInterface::class, slugger: '?'.SluggerInterface::class, )] ContainerInterface $container, ): Response { $container->get('router'); if ($container->has('slugger')) { $container->get('slugger'); } } ``` Commits ------- 5fa830d [DependencyInjection] add `#[AutowireLocator]` attribute

* 5.4: [HttpFoundation] Fix base URI detection on IIS with UrlRewriteModule

* 6.3: [HttpFoundation] Fix base URI detection on IIS with UrlRewriteModule

* 6.4: [DependencyInjection] add `#[AutowireLocator]` attribute [HttpFoundation] Fix base URI detection on IIS with UrlRewriteModule

…tarld) This PR was merged into the 6.4 branch. Discussion ---------- [Serializer] Allow Context to target classes | Q | A | ------------- | --- | Branch? | 6.4 | Bug fix? | no | New feature? | yes | Deprecations? | no | Tickets | Fix symfony#49450 | License | MIT | Doc PR | TODO Allow to target class with the `Context` attribute. The related context can be overridden by each property. Commits ------- caaf0a6 [Serializer] Allow Context to target classes

…n the first one is not resolvable (digilist) This PR was merged into the 5.4 branch. Discussion ---------- [Cache] fix using multiple Redis Sentinel hosts when the first one is not resolvable | Q | A | ------------- | --- | Branch? | 5.4 | Bug fix? | yes | New feature? | no | Deprecations? | no | Tickets | Fix symfony#51570 | License | MIT | Doc PR | See ticket symfony#51570 for details on this bugfix. As mentioned in the ticket, I am not sure if it's wise to catch all exceptions or if it would be better to only check for specific ones. But since I cannot think about any reasons other than an unreachable host to raise any exception, I decided to catch all exceptions for now. Commits ------- 578a152 [Cache] fix using multiple Redis Sentinel hosts when the first one is not resolvable

…ET and POST

…::connect() for both GET and POST (wivaku) This PR was squashed before being merged into the 6.4 branch. Discussion ---------- [HttpClient] Enable using EventSourceHttpClient::connect() for both GET and POST | Q | A | ------------- | --- | Branch? | 6.3 | Bug fix? | yes | New feature? | no | Deprecations? | no | License | MIT Fix so connect() can be used for SSE connections that require POST, e.g. GraphQL SSE subscriptions. This so we don't have to manually create the request() and include all of the options that are used for connect(). Commits ------- 35edcf8 [HttpClient] Enable using EventSourceHttpClient::connect() for both GET and POST

This PR was merged into the 6.3 branch. Discussion ---------- [Scheduler] Speed up tests | Q | A | ------------- | --- | Branch? | 6.3 | Bug fix? | no | New feature? | no  | Deprecations? | no  | Tickets | n/a | License | MIT | Doc PR | n/a Commits ------- ff45ca1 [Scheduler] Speed up tests

…nUser()`

…n `KernelBrowser::loginUser()` (Valmonzo) This PR was merged into the 6.4 branch. Discussion ---------- [FrameworkBundle] [Test] add token attributes in `KernelBrowser::loginUser()` | Q | A | ------------- | --- | Branch? | 6.4 | Bug fix? | no | New feature? | yes | Deprecations? | yes | Tickets | ~ | License | MIT | Doc PR | TODO We have an advanced use case where we define custom attributes in tokens from success handlers. Then, we use them for access controls. Currently we cannot define those attributes when testing and using the `loginUser()` method, and so we needed to copy paste the whole logic to do it. What do you think about this feature? I did not add tests because the method does not have any yet, should we add some? It's my first contribution, thanks to my brother `@HeahDude` for his help 🙏🏽 Commits ------- 1d38977 [FrameworkBundle][Test]: add token attributes in `KernelBrowser::loginUser()`

This PR was merged into the 6.4 branch. Discussion ---------- [Scheduler] Fix stateful scheduler | Q | A | ------------- | --- | Branch? | 6.3 | Bug fix? | yes | New feature? | no | Deprecations? | no | Tickets | Fix symfony#51646, symfony#51384 | License | MIT Stateful scheduler seems rather broken at the moment, see symfony#51384 (comment). Let's fix it by storing the original first run start time, that way it's always possible to recalculate the state. Catching-up works now: ``` [23:14:11.709710] Worker started [23:14:11.759318] every 2 seconds [23:14:11.760291] every 2 seconds [23:14:11.761257] every 2 seconds [23:14:11.763244] every 2 seconds [23:14:12.637054] every 2 seconds [23:14:14.620595] every 2 seconds [23:14:16.632170] every 2 seconds ``` Whereas before it would only start on from the current item, possibly skipping previous items like stated in symfony#51646. _(is this a bc break?)_ I will be waiting for input from authors of the related issues. --- One test is failing because because `getNextRunDate` is called with `2020-02-20T01:59:00+02:00` and then next run date is expected at `2020-02-20T02:09:00+02:00` but we get `2020-02-20T02:00:00+02:00` because that's set as `from`. I don't quite get the logic, I would assume that it is expected to be run immediately on `from` :thinking: Commits ------- 2d5856b Fix stateful scheduler

* 6.3: [Scheduler] Speed up tests

* 6.4: [Scheduler] Fix changelog [FrameworkBundle][Test]: add token attributes in `KernelBrowser::loginUser()` Fix stateful scheduler [Scheduler] Speed up tests [HttpClient] Enable using EventSourceHttpClient::connect() for both GET and POST [Serializer] Allow Context to target classes [Validator] Add is_valid function to Expression constraint Fix Form profiler toggles [FrameworkBundle] Fix missing PhraseProviderFactory import [Security] Fixing deprecation message

…ezone (valtzu) This PR was merged into the 6.3 branch. Discussion ---------- [Scheduler] Match next run timezone with "from" timezone | Q | A | ------------- | --- | Branch? | 6.3 | Bug fix? | yes | New feature? | no | Deprecations? | no | License | MIT As discussed in symfony#51651 (comment), when a datetime object is created from unix timestamp, the timezone constructor argument is ignored as demonstrated in https://onlinephp.io/c/b07d1 and also mentioned in [PHP documentation](https://www.php.net/manual/en/datetime.construct.php#refsect1-datetime.construct-parameters): > The $timezone parameter and the current timezone are ignored when the $datetime parameter either is a UNIX timestamp (e.g. `@946684800`) or specifies a timezone (e.g. 2010-01-28T15:00:00+02:00). This change shouldn't break any existing logic, given the places where this time is used already include timezone in the date format string. ### Changes in effect ```diff ------------- ------------------------------------ --------------------------- Message Trigger Next Run ------------- ------------------------------------ --------------------------- - TestMessage PeriodicalTrigger: every 2 seconds 2023-09-16T15:54:46+00:00 + TestMessage PeriodicalTrigger: every 2 seconds 2023-09-16T18:54:46+03:00 ------------- ------------------------------------ --------------------------- ``` Commits ------- 9baf427 Match next run timezone with from timezone

* 6.3: Match next run timezone with from timezone

* 6.4: Match next run timezone with from timezone

* 5.4: [String] Update wcswidth data with Unicode 15.1 [FrameworkBundle] no serializer mapping cache in debug mode without enable_annotations [Cache] fix using multiple Redis Sentinel hosts when the first one is not resolvable

* 6.3: [String] Update wcswidth data with Unicode 15.1 [FrameworkBundle] no serializer mapping cache in debug mode without enable_annotations [Cache] fix using multiple Redis Sentinel hosts when the first one is not resolvable

…nd security-http 7.0 (alexander-schranz) This PR was merged into the 6.4 branch. Discussion ---------- Fix incompatibility between security-bundle 6.4 and security-http 7.0 | Q | A | ------------- | --- | Branch? | 6.4 | Bug fix? | yes | New feature? | no  | Deprecations? | no  | Tickets | Fix #...  | License | MIT | Doc PR | symfony/symfony-docs#...  The security-bundle 6.4 is not compatible with security-http 7.0: > PHP Fatal error: Declaration of > `Sulu\Bundle\SecurityBundle\Security\AuthenticationEntryPoint::start(` > `Symfony\Component\HttpFoundation\Request $request, ?Symfony\Component\Security\Core\Exception\AuthenticationException $authException = null)` > must be compatible with > `Symfony\Component\Security\Http\EntryPoint\AuthenticationEntryPointInterface::start(` > `Symfony\Component\HttpFoundation\Request $request, ?Symfony\Component\Security\Core\Exception\AuthenticationException $authException = null): Symfony\Component\HttpFoundation\Response` > in /home/runner/work/sulu/sulu/src/Sulu/Bundle/SecurityBundle/Security/AuthenticationEntryPoint.php on line 25 Commits ------- 2a8f8f8 Fix incompatibility between security-bundle 6.4 and security-http 7.0

* 6.4: Fix incompatibility between security-bundle 6.4 and security-http 7.0 [String] Update wcswidth data with Unicode 15.1 [FrameworkBundle] no serializer mapping cache in debug mode without enable_annotations [Cache] fix using multiple Redis Sentinel hosts when the first one is not resolvable

…le 6.4 and security-http 7.0 (alexander-schranz)" This reverts commit 1708789, reversing changes made to 36161f0.

* 6.4: Revert "minor symfony#51672 Fix incompatibility between security-bundle 6.4 and security-http 7.0 (alexander-schranz)"

…hen publishing a message. (jwage) This PR was squashed before being merged into the 6.4 branch. Discussion ---------- [Messenger] [Amqp] Handle AMQPConnectionException when publishing a message. | Q | A | ------------- | --- | Branch? | 6.4 | Bug fix? | yes | New feature? | no | Deprecations? | no | Issues | Fix symfony#36538 Fix symfony#48241 | License | MIT If you have a message handler that dispatches messages to another queue, you can encounter `AMQPConnectionException` with the message "Library error: a SSL error occurred" or "a socket error occurred" depending on if you are using tls or not or if you are running behind a load balancer or not. You can manually reproduce this issue by dispatching a message where the handler then dispatches another message to a different queue, then go to rabbitmq admin and close the connection manually, then dispatch another message and when the message handler goes to dispatch the other message, you will get this exception: ``` a socket error occurred #0 /vagrant/vendor/symfony/amqp-messenger/Transport/AmqpTransport.php(60): Symfony\Component\Messenger\Bridge\Amqp\Transport\AmqpSender->send() #1 /vagrant/vendor/symfony/messenger/Middleware/SendMessageMiddleware.php(62): Symfony\Component\Messenger\Bridge\Amqp\Transport\AmqpTransport->send() #2 /vagrant/vendor/symfony/messenger/Middleware/FailedMessageProcessingMiddleware.php(34): Symfony\Component\Messenger\Middleware\SendMessageMiddleware->handle() #3 /vagrant/vendor/symfony/messenger/Middleware/DispatchAfterCurrentBusMiddleware.php(61): Symfony\Component\Messenger\Middleware\FailedMessageProcessingMiddleware->handle() #4 /vagrant/vendor/symfony/messenger/Middleware/RejectRedeliveredMessageMiddleware.php(41): Symfony\Component\Messenger\Middleware\DispatchAfterCurrentBusMiddleware->handle() #5 /vagrant/vendor/symfony/messenger/Middleware/AddBusNameStampMiddleware.php(37): Symfony\Component\Messenger\Middleware\RejectRedeliveredMessageMiddleware->handle() #6 /vagrant/vendor/symfony/messenger/Middleware/TraceableMiddleware.php(40): Symfony\Component\Messenger\Middleware\AddBusNameStampMiddleware->handle() #7 /vagrant/vendor/symfony/messenger/MessageBus.php(70): Symfony\Component\Messenger\Middleware\TraceableMiddleware->handle() #8 /vagrant/vendor/symfony/messenger/TraceableMessageBus.php(38): Symfony\Component\Messenger\MessageBus->dispatch() #9 /vagrant/src/Messenger/MessageBus.php(37): Symfony\Component\Messenger\TraceableMessageBus->dispatch() #10 /vagrant/vendor/symfony/mailer/Mailer.php(66): App\Messenger\MessageBus->dispatch() symfony#11 /vagrant/src/Mailer/Mailer.php(83): Symfony\Component\Mailer\Mailer->send() symfony#12 /vagrant/src/Mailer/Mailer.php(96): App\Mailer\Mailer->send() symfony#13 /vagrant/src/MessageHandler/Trading/StrategySubscriptionMessageHandler.php(118): App\Mailer\Mailer->sendEmail() symfony#14 /vagrant/src/MessageHandler/Trading/StrategySubscriptionMessageHandler.php(72): App\MessageHandler\Trading\StrategySubscriptionMessageHandler->handle() symfony#15 /vagrant/vendor/symfony/messenger/Middleware/HandleMessageMiddleware.php(152): App\MessageHandler\Trading\StrategySubscriptionMessageHandler->__invoke() symfony#16 /vagrant/vendor/symfony/messenger/Middleware/HandleMessageMiddleware.php(91): Symfony\Component\Messenger\Middleware\HandleMessageMiddleware->callHandler() symfony#17 /vagrant/vendor/symfony/messenger/Middleware/SendMessageMiddleware.php(71): Symfony\Component\Messenger\Middleware\HandleMessageMiddleware->handle() symfony#18 /vagrant/vendor/symfony/messenger/Middleware/FailedMessageProcessingMiddleware.php(34): Symfony\Component\Messenger\Middleware\SendMessageMiddleware->handle() symfony#19 /vagrant/vendor/symfony/messenger/Middleware/DispatchAfterCurrentBusMiddleware.php(68): Symfony\Component\Messenger\Middleware\FailedMessageProcessingMiddleware->handle() symfony#20 /vagrant/vendor/symfony/messenger/Middleware/RejectRedeliveredMessageMiddleware.php(41): Symfony\Component\Messenger\Middleware\DispatchAfterCurrentBusMiddleware->handle() symfony#21 /vagrant/vendor/symfony/messenger/Middleware/AddBusNameStampMiddleware.php(37): Symfony\Component\Messenger\Middleware\RejectRedeliveredMessageMiddleware->handle() symfony#22 /vagrant/vendor/symfony/messenger/Middleware/TraceableMiddleware.php(40): Symfony\Component\Messenger\Middleware\AddBusNameStampMiddleware->handle() symfony#23 /vagrant/vendor/symfony/messenger/MessageBus.php(70): Symfony\Component\Messenger\Middleware\TraceableMiddleware->handle() symfony#24 /vagrant/vendor/symfony/messenger/TraceableMessageBus.php(38): Symfony\Component\Messenger\MessageBus->dispatch() symfony#25 /vagrant/vendor/symfony/messenger/RoutableMessageBus.php(54): Symfony\Component\Messenger\TraceableMessageBus->dispatch() symfony#26 /vagrant/vendor/symfony/messenger/Worker.php(162): Symfony\Component\Messenger\RoutableMessageBus->dispatch() symfony#27 /vagrant/vendor/symfony/messenger/Worker.php(109): Symfony\Component\Messenger\Worker->handleMessage() symfony#28 /vagrant/vendor/symfony/messenger/Command/ConsumeMessagesCommand.php(238): Symfony\Component\Messenger\Worker->run() symfony#29 /vagrant/vendor/symfony/console/Command/Command.php(326): Symfony\Component\Messenger\Command\ConsumeMessagesCommand->execute() symfony#30 /vagrant/vendor/symfony/console/Application.php(1096): Symfony\Component\Console\Command\Command->run() symfony#31 /vagrant/vendor/symfony/framework-bundle/Console/Application.php(126): Symfony\Component\Console\Application->doRunCommand() symfony#32 /vagrant/vendor/symfony/console/Application.php(324): Symfony\Bundle\FrameworkBundle\Console\Application->doRunCommand() symfony#33 /vagrant/vendor/symfony/framework-bundle/Console/Application.php(80): Symfony\Component\Console\Application->doRun() symfony#34 /vagrant/vendor/symfony/console/Application.php(175): Symfony\Bundle\FrameworkBundle\Console\Application->doRun() symfony#35 /vagrant/vendor/symfony/runtime/Runner/Symfony/ConsoleApplicationRunner.php(49): Symfony\Component\Console\Application->run() symfony#36 /vagrant/vendor/autoload_runtime.php(29): Symfony\Component\Runtime\Runner\Symfony\ConsoleApplicationRunner->run() symfony#37 /vagrant/bin/console(11): require_once('...') symfony#38 {main} ``` TODO: - [x] Add test for retry logic when publishing messages Commits ------- f123370 [Messenger] [Amqp] Handle AMQPConnectionException when publishing a message.

…rsimpsons) This PR was merged into the 5.4 branch. Discussion ---------- [Yaml] 🐛 throw ParseException on invalid date | Q | A | ------------- | --- | Branch? | 5.4  | Bug fix? | yes | New feature? | no  | Deprecations? | no  | Issues | None  | License | MIT (found in symfony-tools/docs-builder#179) When parsing the following yaml: ``` date: 6418-75-51 ``` `symfony/yaml` will throw an exception: ``` $ php main.php PHP Fatal error: Uncaught Exception: Failed to parse time string (6418-75-51) at position 6 (5): Unexpected character in /tmp/symfony-yaml/vendor/symfony/yaml/Inline.php:714 Stack trace: #0 /tmp/symfony-yaml/vendor/symfony/yaml/Inline.php(714): DateTimeImmutable->__construct() #1 /tmp/symfony-yaml/vendor/symfony/yaml/Inline.php(312): Symfony\Component\Yaml\Inline::evaluateScalar() #2 /tmp/symfony-yaml/vendor/symfony/yaml/Inline.php(80): Symfony\Component\Yaml\Inline::parseScalar() #3 /tmp/symfony-yaml/vendor/symfony/yaml/Parser.php(790): Symfony\Component\Yaml\Inline::parse() #4 /tmp/symfony-yaml/vendor/symfony/yaml/Parser.php(341): Symfony\Component\Yaml\Parser->parseValue() #5 /tmp/symfony-yaml/vendor/symfony/yaml/Parser.php(86): Symfony\Component\Yaml\Parser->doParse() #6 /tmp/symfony-yaml/vendor/symfony/yaml/Yaml.php(77): Symfony\Component\Yaml\Parser->parse() #7 /tmp/symfony-yaml/main.php(8): Symfony\Component\Yaml\Yaml::parse() #8 {main} thrown in /tmp/symfony-yaml/vendor/symfony/yaml/Inline.php on line 714 ``` This is because the "month" is invalid. Fixing the "month" will trigger about the same issue because the "day" would be invalid. With the current change it will throw a `ParseException`. Commits ------- 6d71a7e 🐛 throw ParseException on invalid date

mtarld force-pushed the redesign branch from 28b216b to 82d4a44 Compare July 11, 2023 09:47

mtarld force-pushed the redesign branch 4 times, most recently from 9d616d1 to 91fb21f Compare July 21, 2023 09:06

Fix breaking change in AccessTokenAuthenticator

21532cb

fixes symfony#50511

mtarld force-pushed the redesign branch 3 times, most recently from 6307f06 to 3fcad54 Compare August 14, 2023 10:00

mtarld force-pushed the redesign branch 5 times, most recently from a05386b to aa66756 Compare August 18, 2023 10:20

[Webhook][RemoteEvent] Add Sendgrid symfony#50704

bb57ad0

mtarld force-pushed the redesign branch 3 times, most recently from 1f8de3d to 3d3b6d9 Compare August 18, 2023 13:46

mtarld changed the title ~~[Serializer] Redesign component~~ [Serializer] Putting the serializer component on steroids Aug 18, 2023

mtarld force-pushed the redesign branch 2 times, most recently from a0cd694 to fdd36d1 Compare August 18, 2023 16:19

derrabus and others added 2 commits August 21, 2023 09:23

[HttpFoundation] Fix base URI detection on IIS with UrlRewriteModule

26aec0f

mtarld force-pushed the redesign branch from fdd36d1 to e675871 Compare August 21, 2023 11:56

kbond and others added 5 commits August 21, 2023 14:39

[DependencyInjection] add #[AutowireLocator] attribute

5fa830d

Merge branch '5.4' into 6.3

64f58cb

* 5.4: [HttpFoundation] Fix base URI detection on IIS with UrlRewriteModule

Merge branch '6.3' into 6.4

8fe4804

* 6.3: [HttpFoundation] Fix base URI detection on IIS with UrlRewriteModule

Merge branch '6.4' into 7.0

f1c8b07

* 6.4: [DependencyInjection] add `#[AutowireLocator]` attribute [HttpFoundation] Fix base URI detection on IIS with UrlRewriteModule

nicolas-grekas and others added 25 commits September 14, 2023 12:05

[HttpClient] Enable using EventSourceHttpClient::connect() for both G…

35edcf8

…ET and POST

[Scheduler] Speed up tests

ff45ca1

Fix stateful scheduler

2d5856b

[FrameworkBundle][Test]: add token attributes in `KernelBrowser::logi…

1d38977

…nUser()`

[Scheduler] Fix changelog

00e0c07

Merge branch '6.3' into 6.4

f128a58

* 6.3: [Scheduler] Speed up tests

Fix incompatibility between security-bundle 6.4 and security-http 7.0

2a8f8f8

Match next run timezone with from timezone

9baf427

Merge branch '6.3' into 6.4

517128f

* 6.3: Match next run timezone with from timezone

Merge branch '6.4' into 7.0

bc82d59

* 6.4: Match next run timezone with from timezone

Merge branch '5.4' into 6.3

4ba165a

* 5.4: [String] Update wcswidth data with Unicode 15.1 [FrameworkBundle] no serializer mapping cache in debug mode without enable_annotations [Cache] fix using multiple Redis Sentinel hosts when the first one is not resolvable

Merge branch '6.3' into 6.4

36161f0

* 6.3: [String] Update wcswidth data with Unicode 15.1 [FrameworkBundle] no serializer mapping cache in debug mode without enable_annotations [Cache] fix using multiple Redis Sentinel hosts when the first one is not resolvable

Revert "minor symfony#51672 Fix incompatibility between security-bund…

5bfce3f

…le 6.4 and security-http 7.0 (alexander-schranz)" This reverts commit 1708789, reversing changes made to 36161f0.

Merge branch '6.4' into 7.0

f3c9644

* 6.4: Revert "minor symfony#51672 Fix incompatibility between security-bundle 6.4 and security-http 7.0 (alexander-schranz)"

[Serializer] Redesign component

ee2e070

mtarld force-pushed the redesign branch from e675871 to ee2e070 Compare September 22, 2023 07:08

mtarld closed this Sep 22, 2023

mtarld added a commit that referenced this pull request Oct 9, 2023

chore: link to component (#7)

d2758bb

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Serializer] Putting the serializer component on steroids #7

[Serializer] Putting the serializer component on steroids #7

mtarld commented Jul 10, 2023 •

edited

Loading

[Serializer] Putting the serializer component on steroids #7

[Serializer] Putting the serializer component on steroids #7

Conversation

mtarld commented Jul 10, 2023 • edited Loading

Why?

Main ideas

Cache

Streaming

Laziness

Lazy reading

Lazy instantiation

Configuration and context

Generics support

API

BC & Upgrade Path

Performance showcase

Usage example

Extension points

Serialization

Config classes

NormalizerEncoderTemplateGenerator

TemplateGeneratorInterface

PropertyMetadataLoaderInterface

DataModelBuilderInterface

TemplateVariation

Deserialization

EagerTemplateGenerator

LazyTemplateGenerator

InstantiatorInterface

Thoughts about Type and the PropertyInfo component

mtarld commented Jul 10, 2023 •

edited

Loading

`NormalizerEncoderTemplateGenerator`

`TemplateGeneratorInterface`

`PropertyMetadataLoaderInterface`

`DataModelBuilderInterface`

`TemplateVariation`

`EagerTemplateGenerator`

`LazyTemplateGenerator`

`InstantiatorInterface`