Bernhard did a test before deciding on a format: https://github.com/webmozart/json-res-benchmark

@mvrhov thanks for the reference! According to the benchmark results, JSON is the worst format for performance.

The discussion is here #11920

@mvrhov in the discussion, Bernhard demonstrated that JSON is not only the slowest format ... but also the biggest in size (because, unlike PHP, can't save UTF8 chars directly). And all this is before the massive improvements introduced by PHP 7. The difference between PHP and JSON would be even bigger today.

I'm not against it. And have no idea why @webmozart at the end decided to use json.

👍 to do it, static arrays in shared memory rulez.

👍 I'll have a look

would it be possible to reuse the benchmark of https://github.com/webmozart/json-res-benchmark to see what happens when using static arrays in shared memory ?

PHP 7.1.10

Memory & Peak Memory 2048kB in all cases.

Notes:

• used en_GI instead of en_GB as the later's not created anymore
• loaded an additional meta file in each benchmark as the root file doesn't contain Alpha3ToNumeric anymore

Previous benchmarks in brackets.

Results (opcache disabled)

Results (opcache enabled)

I think the reason why json was chosen might've been @bojanz'es comment:

I've been told multiple times that if the definitions are in PHP, the memory spent on the included data can never be reclaimed (unlike JSON), which is point against it.

I still think JSON is fast enough to justify being the only option.

and @rszrama's

I could be misunderstanding what you're benchmarking or how we expect these files to be used, but if it's similar to the conversation bojanz and I had last week, my primary point in favor of JSON was the simplicity of using the same data in both the server side and client side code (assuming the raw data was used on the front end for some client side formatting).

@jakzal what would be your recommendation? Switch to PHP? Keep things unchanged? Thanks!

@javiereguiluz research in progress ;)

@javiereguiluz do you remember which page were you profiling exactly?

I don't remember, so I profiled several pages again. The time spent on this json_encode() is pretty constant, so it has a high impact on fast pages:

Blog index: http://symfony-demo.test/en/blog/
Profile: https://blackfire.io/profiles/f50e1db0-1024-409c-b606-a0cd7461a7b5/graph
Total time: 191 ms
json_decode: 3.21 ms (1.6% of total)

Blog show: http://symfony-demo.test/en/blog/posts/lorem-ipsum-dolor-sit-amet-consectetur-adipiscing-elit
Profile: https://blackfire.io/profiles/fda8f579-6961-4b2c-bee6-ebe3af7b53a2/graph
Total time: 150 ms
json_decode: 3.03 ms (2% of total)

Backend index: http://symfony-demo.test/en/admin/post/
Profile: https://blackfire.io/profiles/88526f65-2c68-4a16-9873-376d9e51d197/graph
Total time: 95.7 ms
json_decode: 2.82 ms (3% of total)

I made some profiling too, but with 2-3ms overhead for json_decode it's hard to observe significant differences:

• https://blackfire.io/profiles/compare/6f6e0384-e8fe-4389-80f8-2006c712f28a/graph
• https://blackfire.io/profiles/compare/cf24f004-aea5-442e-9ff2-2ee7c3afaae3/graph

Still, my previous raw tests show that php format is over twice faster than json (even if it's just 1.12ms vs 2.84ms).

After being loaded, is the translation array ever modified? Like merged with others, or entries added before being used? If yes, that should be fixed first: this would kill all the potential benefit.

It's returned right away without modifications: https://github.com/symfony/intl/blob/master/Data/Bundle/Reader/PhpBundleReader.php#L54

Will this still get merged then sr_Latn_BA.json with sr_Latn.json?

It's surprising that the memory usage doesn't drop, that's why I'm asking. Should be double checked to ensure the return value itself is not triggering COW. (Sorry can't check now)

I rerun the benchmarks.

PHP 7.3-rc

PHP 7.2

PHP 7.1

PHP 7.0

PHP 5.6

PHP 5.5

After being loaded, is the translation array ever modified? Like merged with others, or entries added before being used? If yes, that should be fixed first: this would kill all the potential benefit.

Entries are being merged in some cases: https://github.com/symfony/intl/blob/master/Data/Bundle/Reader/BundleEntryReader.php#L101-L122

@javiereguiluz @nicolas-grekas what are your opinions on this? I'd like to either make the conversion soon, or close this :) Any more tests/benchmarks I could do?

If no one can foresee technical issues by making this change, I'd say that the conversion from JSON to PHP is a no brainer. You get a significant performance improvement "for free".

I agree with @javiereguiluz if it's a free performance gain and it does not change anything in the memory spent then this can be done ;). cc @jakzal

I made an app gain a +1000% performance boost in average by changing some config files from JSON to PHP.

Costless, painless, performance gain.

And by the way, it seems that it's only a few lines of changes in the update-data.php script and the ResourceBundle class to default to PHP instead of JSON, so this seems to be a true no-brainer and clearly an easy-pick

@Pierstoval thank you for sharing your results. Impressive!

Thank you for reminding me about this. It's been long on my list of things to do. Let me submit a PR for this to finally make this happen!

Nice thank you @jakzal

@jakzal I started the work on my machine (this is also why I noticed the change is not "that big"), if you want I can submit it :)

@Pierstoval it's fine. I also started it, just need to refresh and run performance tests again.

@jakzal @Pierstoval What is the state of your work on this issue? Is there a chance to have it in 4.4?

sorry for the delay. I was waiting for a new icu release (which has happened now). I’ll rerun my tests this weekend and send a pr.

It’s a very interesting thread, thank you all for the discussions, benchmarks and feedback.

I read sometimes that using named classes with public properties are the „fastest“ choice. Sorry I cannot get the source and to be honest, I am not that deep into those topics. But would you consider using a class over an assoc array?
If yes/no why?

Thanks for the info 👍🏻

Named classes can be faster because they're always passed as references, so it may improve perfs a bit because arrays are passed as copy. However, with copy-on-write for arrays and engine array optimizations from latest PHP versions, I'm not sure it's really useful to use a class rather than a plain PHP array.

Maybe PHP experts could correct me if I'm wrong

Arrays will always be faster thanks to copy-on-write. Classes in PHP 7.4 might be as fast, but not in earlier versions. Of course, this supposes we do NO operations on the loaded arrays. Anything that would trigger COW would break the perf benefit.

Thank you very much for the clarification 🙂

Thank you for this suggestion.
There has not been a lot of activity here for a while. Would you still like to see this feature?

Could I get an answer? If I do not hear anything I will assume this issue is resolved or abandoned. Please get back to me <3

Hey,

I didn't hear anything so I'm going to close it. Feel free to comment if this is still relevant, I can always reopen!