Hey!

Thanks for your PR. You are targeting branch "6.4" but it seems your PR description refers to branch "6.4 for features".
Could you update the PR description or change target branch? This helps core maintainers a lot.

Cheers!

Carsonbot

Let's compromise and make these protected? Serializer is not final so you can extend and get what you need there?

Test failures look unrelated?

You may be interested in existing API Platform packages allowing to generate JSON Schema and adding support for extra normalization formats to the serializer such as JSON:API, JSON-LD and HAL: https://packagist.org/packages/api-platform/

Maybe can we collaborate on this topic?

(We managed to do that without changing the visibility).

HI @dunglas, thanks so much for your comment and offer to explore this in a broader context. I took a fresh look at API Platform and really like what the project does to solve some really common use cases.

I think the Drupal implementation is different enough, though, that I'm not able to follow the pattern you have in SchemaFactory::buildSchema(). The main difference being that api-platform builds schema for classes that are (AFAICT, not an expert) more-or-less directly exposed over the public API. While Drupal does represent its entities in classes, the fields are not necessarily properties of the class and are loaded through an internal API. The field definitions are also dynamic in that they can be essentially subclassed through "bundles" that can be, but generally aren't, subclassed. Reflection doesn't tell you anything about the fields exposed on the API.

I'm sure the api-platform implementation is more complex than just "give me schema for X class's public properties" but in Drupal it's more like "give me the schema for the various normalizers that will be called for its fields, that can't be statically analyzed purely from the class structure."

Or perhaps more succinctly, api-platform's schema generation is invoked in a similar way to what we need for Drupal, however when it gets to

foreach ($this->propertyNameCollectionFactory->create($inputOrOutputClass, $options) as $propertyName) {
  // Uses reflection
  $propertyMetadata = $this->propertyMetadataFactory->create($inputOrOutputClass, $propertyName, $options);
  ...
  $this->buildPropertySchema($schema, $definitionName, $normalizedPropertyName, $propertyMetadata, $serializerContext, $format);
  ...
}

we have to do something like (pseudocode)

foreach ($entityFieldManager->getFieldsForEntity($entity) as $field) {
  // Can't use reflection here.
  $schema = $serializer->normalizer->getNormalizer($field->dataClass)->getSchema();
}

It would be amazing to be able to re-use some existing code from elsewhere in the PHP ecosystem, however Drupal's ORM (field, database and entity APIs) are home grown so we can't for instance borrow from Doctrine.

So in the Drupal pseudocode above, the catch is being able to do ->getNormalizer() so we can introspect its schema.

In any case, this is too late to land this in 6.4 or 7.0 (the feature freeze was 2 weeks ago, so we are only merging PRs that were already in progress at that time and for which we decided to include their review in the initial part of the feature freeze period).
As the feature freeze for 7.1 is in 5.5 months, this gives the time to identify whether the API-Platform architecture could inspire a Drupal architecture that does not need to extend the public API.

Side note: wouldn't the suggested Drupal architecture require every normalizer to implement the Drupal interface, forcing to re-implement all core normalizers of Symfony as well ?

Thanks @stof for your help and input. Good to know about the schedule. This isn't likely to land in Drupal core for a few months at least - still lots to hash out - so while Drupal 10 is currently on Symfony 6 I don't think it's a huge lift if this component might be required at 7.1.

...time to identify whether the API-Platform architecture could inspire a Drupal architecture that does not need to extend the public API.

I'm curious about Kévin's take on my feedback above, however I am pretty confident that api-platform's approach won't work here. To be clear though, I've updated this from my initial thought that it needed to be public to protected instead - we can subclass the serializer and make it public ourselves, but it need not be so in Symfony.

Side note: wouldn't the suggested Drupal architecture require every normalizer to implement the Drupal interface, forcing to re-implement all core normalizers of Symfony as well?

Most of Drupal's normalizers are custom, honestly. We have a "Typed Data" API which provides value objects for field properties, and those are (IIRC) 100% covered by entirely custom normalizers. I don't even think we're using any of the core Symfony normalizers at all. But if we were/someone is, the calling code definitely has to check that the retrieved normalizer implements our new SchematicNormalizerInterface, use reflection, or just output a default. However because of our entity and field API, most user custom field types rely on these core typed data primitives, so they will likely not need to change anything. That's an implementation detail though, I don't think it really impacts whether these methods can be changed to protected. But good question and hopefully that answers it.

I'm not convinced yet that exposing the normalizer resolution is the best solution to your problem. Your normalizers could for instance be wrapped (see TraceableNormalizer) which would break all your instanceof checks.

Can we explore possible alternatives? For instance, your normalizers could react to the $format parameter and emit a JSON schema when being called with $format === 'json_schema'.

Either way, consider taking a look at the extensions API Platform provides for the serializer. This very much sounds like the kind of issue that they've solved already.

Thanks, @derrabus. It sounds like consensus around changing the visibility on these is not going to materialize, so I appreciate the suggestion of using an alternative $format for the normalizer to get a schema. I didn't know about TraceableNormalizer but that makes sense.

I am still struggling to find a real analog to the challenge we're facing in Drupal to API Platform's structure. API Platform is great but as I tried to outline above it seems like our data models are sufficiently different that I haven't yet found a pattern that would translate. You mention extensions to the serializer but I haven't yet found something that fits that description from my analysis of the code. Would you be so kind as to point me to an example that might apply here?

Hi @bradjones1 you wrote

foreach ($entityFieldManager->getFieldsForEntity($entity) as $field) {
  // Can't use reflection here.
  $schema = $serializer->normalizer->getNormalizer($field->dataClass)->getSchema();
}

As you saw in API Platform, we're using Metadata Factories to build our schemas. It's definitely possible to create metadata based on your field manager no? We're not using reflection in most of these. Then you would just rely on the SchemaFactory to create schemas. Could you provide more details about this (but probably through another channel like the Symfony slack, or by mail)? Thanks.

Hi @soyuka, yes, I'm already refactoring based on the feedback here and the overwhelming consensus that this change won't be accepted. I appreciate everyone's help. Likely will be pursuing an approach where the resolved normalizer will get passed a $format parameter of json_schema or something along those lines.