Thanks to visit codestin.com
Credit goes to github.com

Skip to content

PoEditor translation provider pulls wrong translation keys #41374

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
majermi4 opened this issue May 21, 2021 · 13 comments
Closed

PoEditor translation provider pulls wrong translation keys #41374

majermi4 opened this issue May 21, 2021 · 13 comments

Comments

@majermi4
Copy link

Symfony version(s) affected: 5.3.0-RC1

Description

The translation keys pushed to PoEditor translation provider with bin/console translation:push poeditor are not the same as the ones pulled back using bin/console translation:pull poeditor --format yml.

How to reproduce

Create a Symfony project and install symfony/translation with symfony/po-editor-translation-provider.

Configure a poeditor provider and create 2 translation files for locales en and cs in the messages domain.

Screenshot 2021-05-21 at 23 36 16

Run bin/console translation:push poeditor which pushes the translations to Po Editor as it should.

Screenshot 2021-05-21 at 23 37 05

Run bin/console translation:pull poeditor --format yml. The local translation file was updated:

Screenshot 2021-05-21 at 23 45 29

I would expect that nothing should have happened because nothing was changed in Po Editor. Instead, it seems that the pull command uses the english translation as key when it reads data exported from Po Editor.

Possible Solution

I was able to fix the problem when I replaced this line:
$source = isset($attributes['resname']) && $attributes['resname'] ? $attributes['resname'] : $translation->source;
with:
$source = $attributes['id'];

Additional context

In the Po Editor project settings, I have set English as the Default Reference Language.

Screenshot 2021-05-21 at 23 53 44

I'm not sure if it's related to the issue but before I did that, the pull command would respond with these errors:

Screenshot 2021-05-21 at 23 55 02

@OskarStark
Copy link
Contributor

cc @welcoMattic

@welcoMattic
Copy link
Member

welcoMattic commented May 23, 2021

It remembers me an edge case that I've meet during the development. I will check on it tomorrow, in order to fix it before the final release of 5.3.

AFAIK, I've fixed it by changing the reference language in PoEditor settings of the projet. I will double check it.

@welcoMattic
Copy link
Member

I've quickly checked the problem, it looks like we have to handle language creation from the Provider, like it's done in Lokalise provider. I'll try to do it by the end of the week, unless @majermi4 wants to open a PR 😉

@majermi4
Copy link
Author

unless @majermi4 wants to open a PR 😉

@welcoMattic: I can have a look on the weekend and see if I can figure something out :) but can't promise anything

it looks like we have to handle language creation from the Provider, like it's done in Lokalise provider.

I'm not sure I get what you mean and how this is related 🤔 The main problem is that the keys pulled from PoEditor provider are indexed by the english (or whatever the main language is) translation instead of the keys that were pushed to PoEditor.

@welcoMattic
Copy link
Member

I see 2 problems:

1: If you have no languages in PoEditor Project, you can not push anything. Event if you have 1 language, which if the default reference language, you can not push other languages.
2: PoEditor seems to misuse the XLIFF format by using id attribute instead of resname attribute.

Possible Solution
I was able to fix the problem when I replaced this line:
$source = isset($attributes['resname']) && $attributes['resname'] ? $attributes['resname'] : $translation->source;
with:
$source = $attributes['id'];

This code fixes the problem, but it breaks many other things.
The XLIFF specification says:

Identifier - The id attribute is used in many elements as a reference to the original corresponding code data or format for the given element. The value of the id element is determined by the tool creating the XLIFF document. It may or may not be a resource identifier. The identifier of a resource should, at least, be stored in the resname attribute.

I'll try to figure it out tomorrow and this weekend. @majermi4 if you are on the Symfony Devs Slack, we can talk about this issue there.

@welcoMattic
Copy link
Member

I sent an email to the PoEditor support, because I understood that the export files in XLIFF from PoEditor do not include the resname attribute on trans-unit tag. But, this attribute is mandatory in XLIFF to identify a key between systems, and PoEditor uses id attribute for this. Which is not recommended by OASIS.

I will try to make them add the resname attribute quickly in their XLIFF files, but if they do not want, or not can not do it quickly, I think we will have to move from XLIFF for exchange format between Symfony and PoEditor.

@welcoMattic
Copy link
Member

welcoMattic commented May 29, 2021

I contacted PoEditor and asked them to add the missing resname attribute. They are not able to add it quickly.
This is a problem for the Provider. During the development of the feature, I had managed to get the export to work with the right keys with the following solution: set the Default Reference Language to a value equal to a language that is NOT in our application, and the "Export Empty Translations from" parameter to the same language. But this doesn't seem to work anymore.

I guess we'll have to wait for PoEditor to fix the problem, and follow the XLIFF 1.2 specification correctly, by adding the resname attribute to the trans-unit tags.

I also tried to work around the problem by changing the exchange format between the Provider and the SaaS, I tried YML, CSV, INI, but none of them brings a satisfactory result.

Another solution would be to parse the XLIFF before loading it as a Catalog, create the resname attribute, copy the value of the id attribute into the resname attribute for each trans-unit tag. But this implies either modifying the XliffFileLoader for this specific need (which is not a good thing), or copying part of this file into PoEditorProvider, modifying the code to do the attribute copy as I said before.

@Nyholm @nicolas-grekas @OskarStark @fabpot should we have to remove the PoEditor Provider from the 5.3 as it is not fully functional due to the lack of support of the resname attribute?

@fabpot
Copy link
Member

fabpot commented May 29, 2021

I think we should remove support if it does not work.

@stof
Copy link
Member

stof commented May 29, 2021

I also vote for removing the PoEditor provider from 5.3. We can revisit it for 5.4 if poeditor fixes their XLIFF in the meantime (which gives them 6 months)

fabpot added a commit that referenced this issue May 31, 2021
This PR was merged into the 5.3 branch.

Discussion
----------

[Translation] Remove PoEditor Provider

| Q             | A
| ------------- | ---
| Branch?       | 5.3
| Bug fix?      | yes
| New feature?  | no <!-- please update src/**/CHANGELOG.md files -->
| Deprecations? | no <!-- please update UPGRADE-*.md and src/**/CHANGELOG.md files -->
| Tickets       | Fix #41374 <!-- prefix each issue number with "Fix #", no need to create an issue if none exist, explain below instead -->
| License       | MIT
| Doc PR        |

As it has been discussed in #41374, we have to remove the support of PoEditor as Translation Provider. The `translation:pull` command do not work as expected with PoEditor, translations keys are not retrieved as they do.

I contacted PoEditor support in order to fix the bug (it miss the `resname` attribute of `trans-unit` tag in their XLIFF 1.2 export files). I will follow up their change in the next coming weeks/months, and I hope we could re-add the PoEditor Provider in Symfony for 5.4/6.0.

Commits
-------

b2ea42e Remove PoEditor Provider
@fabpot fabpot closed this as completed May 31, 2021
OskarStark added a commit to symfony/symfony-docs that referenced this issue May 31, 2021
This PR was submitted for the 5.4 branch but it was merged into the 5.3 branch instead.

Discussion
----------

[Translation] Remove PoEditor Provider

Following up to symfony/symfony#41374, we have to remove the PoEditor support until their XLIFF export file are fixed.

Commits
-------

3a35658 Remove PoEditor Provider
@timwhite
Copy link

@welcoMattic Any luck with getting POEditor to fix the export? I tried a manual export today and it doesn't look any better. Just wondering what I should be asking POEditor support what to fix, i.e. if you have an existing case we can reference to encourage them to get it fixed?

@welcoMattic
Copy link
Member

welcoMattic commented Jul 30, 2022

@timwhite I've just checked my last email exchanges with POeditor in september 2021, and they told me that they were about to add the support of the missing attribute very soon. I didn't checked since. Can you open a new issue about adding POEditor Provider? Referencing this one, then we will discuss there. Thanks

@NovakHonza
Copy link

Hi, I tried to upload a Symfony generated xliff to POeditor for cs and tried to download POeditor's xliff of en and upload worked fine and downloaded file contains trans-unit in this format:

<trans-unit id="1UmtMTs" resname="last_price_of_day">
  <source>last_price_of_day</source>
  <target>Last Price of the Day</target>
</trans-unit>

Does it mean that POeditor fixed the problem? Or did I misunderstand the problem?

@stof
Copy link
Member

stof commented Mar 7, 2025

@NovakHonza please create a new issue instead of commenting on a PR merged 3 years ago that nobody looks at (I saw your comment with a 3 month delay, and totally by luck)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

9 participants