@dunglas was against this kind of change in the past and in fact l do agree with him now.
This makes the serializer much harder to maintain for a very small benefit and the symfony serializer is not meant to be a jms-serializer-like library: it's aim (at least it was that when it was created?) is to be simple and to cover most needs but not the ones too specific.
So i'd say 👎 in the current state.

BTW i advise you to open an issue before a pr or at least to not work too much on a pr before it is accepted as you dont know how it will be received by the deciders team ☺

Edit: and it looks good to me to have a third party library doing that for the people who want to have more or less the jms api but with a better support. maybe we could mention your bundle in the docs ?

Thank you for the reviews. I did not see that PR before.
I guess one could argue about flexibility, usability and maintainability here. I understand that this is 2500 lines more to maintain which is a drawback of course.

BTW i advise you to open an issue before a pr or at least to not work too much on a pr before it is accepted as you dont know how it will be received by the deciders team ☺

Thank you. Yes, This PR ended up being too big for not having an issue before. Sorry about that. Though, The work I've put here will (if not merged) end up in a library for it own. I've gotten (to my surprise) lots of feedback on twitter for this PR. Lots of positive voices. =)

@Nyholm as much as I like the features it brings, as @Ener-Getick I'm against having so much changes in the core.

PS: take over JmsSerializer :P

EI prefer to have one strong, powerful, maintained and MIT licensed serializer library (JMS libs are under the Apache 2 license and cannot be used in GPL v2 projects like Drupal or phpBB) than 2 or 3 incomplete and poorly maintained competitive solutions. IMO @Nyholm is right to contribute here to improve the component.

That being said, features regarding exclusion strategies and those regarding access strategies should be handled in a different manner and reuse existing Symfony components.

Having annotations and configuration files to easily specify properties to include or not is a great improvement. I suggest to add a new metadata object comtaining the list of readable and writeable properties of a class. This metadata object will be populated using reflection (default), the existing groups system (refactoring) and of course the new options you implemented. This metadata system will allow to provide more features to the end user (the goal of this PR) and -at the same time - to drastically improve the performance of the component by generating automatically specialized classes implementing the PropertyAccessorInterface In the cache directory as suggested by @webmozart. This metadata system would probably fit well in the PropertyInfo component, the end user annotations in the Serializer component and (later) the static accessor class generator in the PropertyAccess component.

The other topic is to let the user configuring how to access properties. We worked hard to make the serializer able to leverage PropertyAccess and PropertyInfo components. My goal is to deprecate all objects normalizers except ObjectNormalizer when performances will be similar or better (an objective that can be achieved using the strategy exposed in my prevouis point btw).

All the stuff related to accessing properties (annotations specifying the method to call for instance) must be moved in the PropertyAccess component. By doing this, your work will also benefit to all other components depending of it like Form.

To summaries what I suggest:

• In the ProperyInfo component: add new getReadableProperties and getWritableProperties methods returning a VO as well as a default extractor using reflection.
• In the Serializer component: add annotations, config file and PropertyInfo extractors generating the VO for the system mentioned in the previous bullet to let the user customizing the list of accessible properties. Under the hood it relies on the new PropertyInfo metadata system.
• In Serializer: refactor the group system to also use this new foundation
• Move annotations and metadata related to the how access to the property's value in the PropertyAccess component (can be done in a second time)

WDYT?

P.S: I review and reply from a phone, and it's far from being convenient.

Speaking about the list of readable and writable properties, I've already implemented it in API Platform: https://github.com/api-platform/core/tree/master/src/Metadata

It fits well with the design of the PropertyInfo component (stores metadata in immutable Value Objects) because the current version of the PropertyInfo component has been extracted from API Platform. We can also move the property list in Symfony and use it in the Serializer component. Feel free to import the code you need and transfer the copyright to @fabpot if it can help.

I like @dunglas proposition about creating annotations in the PropertyInfo and PropertyAccess components.
But I'm still not certain about some annotations:

• @Type what does it brings against @var or @param or even @return ?
• @Expose, @Exclude, @ExclusionPolicy, @ReadOnly isn't enough to use groups ?
• @SerializedName I like this one but its name looks weird to me, maybe @ExportAs would be clearer ?

The namespace PropertyManager should be removed and the PropertyInfo and PropertyAccess components use instead.
Not sure I completely understood the aim of the MetadataAwareNormalizer but imo it should be merged with the ObjectNormalizer.

After double checking, the PropertyInfo component is already able to guess if a property is readable / writable but th Serializer doesn't use this feature yet.

@Ener-Getick:

• I agree that @type is useless, PHPDock blocks are already supported
• Groups allow to do everything but they are not always convenient (e.g. to exclude 1 property, you need to add a group to all others). It's a feature often missed by people switching from JMSserializer and I'm in favor of having more user friendly strategies fitting better for specific use cases
• SerilializedName, Alias, ExportAs, I let you find the best name but this is also a must have feature that has been requested many times

@dunglas ok then but i think we should keep the current defaults (e.g. include all properties unless specified by the user) and try to make it as simple as possible.
For @ReadOnly i'm not convinced this is really useful but if you really want it then we should also create @WriteOnly imo (e.g. could be useful for passwords).

The part about accessing properties can be removed in favor of #18016. If we really want to have annotations in the Serializer namespace then we will just have to extend the ones created in #18016.

Follow up of @dunglas comments in #19330 :)

I still struggle to use the Symfony serializer, and revert back to JMS (which sucks, since it's unmaintained). The biggest missing things for me are @SerializedName and a simpler @Groups (e.g. one where I could pass a string instead, @Groups("main")). The second is a super simple change. And if we could add @SerializedName, that would be huge.

@weaverryan the missing part to implement a feature like @SerializedName is that NameConverterInterface receives only the property name, but not the class name. And so, it cannot be used for it unless all properties names are unique in your project (quite unlikely)

Moving to 4.1. Rebase on master needed, where PHP 7.1 features can be used btw.

I’ve tried to figure out how to to use PropertyInfo and PropertyAccess components more. But I’m stuck, probably just because of my ignorance :)
Could anyone give me some more guidance?

I will add some more tests and fix one more bug. But later (my battery is dead)

Proposal: Redesign of Generic Object Normalizers

I discussed this PR with @Nyholm on Slack, and I think object normalizers deserve a major redesign: we should capitalize on other existing Symfony components (namely PropertyInfo and PropertyAccess), use them when possible, and make their interfaces hard dependencies of the Serializer (we may move them in symfony/contracts).
Doing so will allow to implement the new annotations proposed by this PR in an easier way, will make the code easier to maintain and will improve performance.

Here is my detailed proposal:

• @ExclusionPolicy, @Exclude, @Expose: injecting an instance of PropertyInfo\PropertyListExtractorInterface in the new enhanced normalizer should be mandatory. The normalizer will just loop over the properties returned by PropertyListExtractorInterface. We can then provide a PropertyListExtractorInterface implementation reading metadata related to these annots to return a specific list of properties.
• @ReadOnly: similarly, we should make injecting an instance of PropertyInfo\PropertyAccessExtractorInterface mandatory, and provide a decorator that makes isWritable returning false using the metadata related to this new annot. We should also introduce @WriteOnly.
  • @Type: this annot can be dropped, the Serializer already supports this feature (https://symfony.com/doc/current/components/serializer.html#recursive-denormalization-and-type-safety), it uses the PropertyInfo component and get the type data from PHPDoc, getters/setters and even constructor's type hints. We should made this dependency a mandatory one.
  • @SerializedName: the serializer already has a similar feature (https://symfony.com/doc/current/components/serializer.html#converting-property-names-when-serializing-and-deserializing). Thanks to [Serializer] Allow to access extra infos in name converters #27021, we can just create a new implementation of the NameConverterInterface that will use the new metadata related to this annot. This dependency will stay an optional dependency.
  • @Accessor: PropertyAccessorInterface must become a hard dependency. A decorator that will use this serializer metadata and call the defined accessor should be added. We should also introduce a @Mutator annotation.
  • @Groups: this annot already exists. Like for @Exclude/... we should create an implem of PropertyInfo\PropertyListExtractorInterface (or maybe use the same than for @Exclude) using this metadata instead of "hardcoding" this in the new normalizer as it's done today.

In addition to improving code reuse, doing so will have another major benefit: the huge AbstractNormalizer and AbstractObjectNormalizer classes will be split in smaller classes (SOLID code).
Implementations of PropertyListExtractorInterface, PropertyListExtractorInterface and PropertyAcessorInterface will become hard dependencies of the new improved normalizer.

The current ObjectNormalizer will be immediately deprecated. At some point, we may also deprecate GetSetMethodNormalizer and PropertyNormalizer and the abstract classes. They would be replaced by minimalist implementations of the previously mentioned interfaces.
All these classes are too big. Splitting them will ease the maintenance, remove some code duplication in SF and make it easier to introduce new features in the Serializer component because we'll have just one normalizer for all kind of generic objects.
The only remaining generic object normalizer will be the new one.

Another major improvement with this strategy is that both PropertyListExtractorInterface, PropertyListExtractorInterface and PropertyAcessorInterface already have cache decorators. Always using them will improve performance: for instance, currently the list of properties is always computed at runtime in ObjectNormalizer::extractAttributes while PropertyListExtractorInterface has a cached implementation.

We would even introduce cache warmers for classes in predefined directories (for instance App\Entity when using API Platform, or path configured through FrameworkBundle).

Note: API Platform already sucessfully uses a similar strategy for its own normalizers.

@symfony/deciders, would you mind challenging this redesign proposal?

@dunglas one of the main reasons for this proposed refactoring is to improve DX (Developer experience). Could you please share some simple before/after code snippets to see those improvements? Thanks a lot!

This will not answer the question, but it may give a little idea how the "new" use would be. Check the readme at the "proof of concept" bundle: https://github.com/Happyr/SerializerBundle

Just asking: instead of deprecating/breaking things in the current serializer, which will only cause frustration to users ... if the proposed change is so big, what if we create a new modern component instead? We could get inspiration from the nice "encoding" package from Go which makes all this in a great, modern and concise way.

That is a good question. Currently Im just adding a new way of configure the serializer. The PR is massive but there is a lot of tests. Kevin is proposing to keep the current feature set in this PR and reduce the added code. (Which is great.)

There is currently no deprecations or breaking behavior for the user. This will be opt-in.

OK. Good to know. Let's wait for those before/after examples because right now the proposal is too abstract for me. Thanks!

@javiereguiluz

if the proposed change is so big, what if we create a new modern component instead?

They aren't that big. Actually only ObjectNormalizer and the related abstract classes will be really impacted by this change. No interfaces will be even touched, and the main class (Serializer) will not be impacted either. The scope of the Serializer component is very broader than that, we have a a bunch of specialized normalizers (dates, files, RFC7807), many encoders (JSON but also YAML, CSV, XML) etc etc.

I think it doesn't wort it to create a new component (that will use all the interfaces of the current package anyway) for "just" this change.

Also, asking users to switch to a new, totally different component instead of "just" injecting new services instead of using an abstract class will be a painfuller upgrade process.

We could get inspiration from the nice "encoding" package from Go which makes all this in a great, modern and concise way.

encoding is great but low level. It is more comparable to json_encode than to the Symfony Serializer component. It's usually not a problem in Go because the language has capabilities that PHP has not (especially typed fields, it's what brings PropertyInfo).
However, it lacks circular reference support, is not really convenient when dealing with dynamic data, and above all has no high level features like ours (groups, specific fields encoding, global name conversion. We may take inspiration from it, but we cannot just "copy it" because it is not really "phpish".

What's the status of this PR? I can see a lot of interesting ideas but I fail to see a path to move forward. Is it possible to "finish" it before feature freeze?

@Nyholm friendly ping :)

I think this PR brings some great features. I will keep working to make sure they are implemented. But we should do it with a different approach.

See here #30818 for the general plan.

Thank you all for reviews and spending time on this.