Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Commit 8ca8aef

Browse files
fabpotfabpot
committed
merged branch stof/gettext_loader (PR symfony#2412)
Commits ------- d974a4a Merge pull request #4 from stealth35/test_mo_loader cf05646 delete useless tests 19f9de9 [Translation] fix gettext tests 965f2bf Merge pull request #3 from stealth35/test_mo_loader 9c2a26d [Translation] add Mo loader tests 9af2342 [Translation] Added the gettext loaders Discussion ---------- [Translation] Added the gettext loaders This is the squashed version of the work done by @xaav in symfony#634. @stealth35 you said you will work on the dumpers. do you have some stuff on it ? --------------------------------------------------------------------------- by drak at 2011/10/24 19:28:43 -0700 Is there any more progress with this? --------------------------------------------------------------------------- by stealth35 at 2011/10/25 00:57:19 -0700 I work on the dumpers, but the Po loader is wrong, caus' the Po ressource can be multiline, msgid "" "Here is an example of how one might continue a very long string\n" "for the common case the string represents multi-line output.\n" http://www.gnu.org/software/gettext/manual/gettext.html#PO-Files Anyway the Po format is an intermediate format to Mo file, (like .txt to .res file for ICU), IMO we can just support the real gettext format : Mo --------------------------------------------------------------------------- by stealth35 at 2011/11/03 02:00:24 -0700 @stof The MO Dumper is ready (stealth35/symfony@f2d1d5b), should we keep the PO format ? --------------------------------------------------------------------------- by fabpot at 2011/11/07 08:50:59 -0800 @stealth35: The PO is what people will use for their translations. They will then dump it to MO. So, we need both PO and MO loaders and dumpers. --------------------------------------------------------------------------- by stealth35 at 2011/11/08 01:25:39 -0800 @fabpot, I'm ready for both dumpers, you can merge this, and I'll open a PR for the dumpers --------------------------------------------------------------------------- by fabpot at 2011/11/08 22:37:47 -0800 I've just had a look at this PR code again and I see that the unit tests are pretty slim. Is it possible to add some tests for the mo loader? --------------------------------------------------------------------------- by stealth35 at 2011/11/09 01:15:25 -0800 @fabpot test send to @stof ✌️ --------------------------------------------------------------------------- by stof at 2011/11/09 02:22:55 -0800 and merged in this branch --------------------------------------------------------------------------- by fabpot at 2011/11/09 02:39:09 -0800 The tests do not pass for me: There was 1 error: 1) Symfony\Tests\Component\Translation\Loader\MoFileLoaderTest::testLoadDoesNothingIfEmpty InvalidArgumentException: MO stream content has an invalid format. /Users/fabien/work/symfony/git/symfony/src/Symfony/Component/Translation/Loader/MoFileLoader.php:79 /Users/fabien/work/symfony/git/symfony/src/Symfony/Component/Translation/Loader/MoFileLoader.php:46 /Users/fabien/work/symfony/git/symfony/tests/Symfony/Tests/Component/Translation/Loader/MoFileLoaderTest.php:34 -- There was 1 failure: 1) Symfony\Tests\Component\Translation\Loader\PoFileLoaderTest::testLoad Failed asserting that two arrays are equal. --- Expected +++ Actual @@ @@ Array ( - 'foo' => 'bar' ) /Users/fabien/work/symfony/git/symfony/tests/Symfony/Tests/Component/Translation/Loader/PoFileLoaderTest.php:25
2 parents ad6ffea + d974a4a commit 8ca8aef

9 files changed

Lines changed: 365 additions & 0 deletions

File tree

‎src/Symfony/Bundle/FrameworkBundle/Resources/config/translation.xml‎

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,6 +11,8 @@
1111
<parameter key="translation.loader.php.class">Symfony\Component\Translation\Loader\PhpFileLoader</parameter>
1212
<parameter key="translation.loader.yml.class">Symfony\Component\Translation\Loader\YamlFileLoader</parameter>
1313
<parameter key="translation.loader.xliff.class">Symfony\Component\Translation\Loader\XliffFileLoader</parameter>
14+
<parameter key="translation.loader.po.class">Symfony\Component\Translation\Loader\PoFileLoader</parameter>
15+
<parameter key="translation.loader.mo.class">Symfony\Component\Translation\Loader\MoFileLoader</parameter>
1416
<parameter key="translation.loader.qt.class">Symfony\Component\Translation\Loader\QtTranslationsLoader</parameter>
1517
<parameter key="translation.loader.csv.class">Symfony\Component\Translation\Loader\CsvFileLoader</parameter>
1618
<parameter key="translation.loader.rb.class">Symfony\Component\Translation\Loader\ResourceBundleLoader</parameter>
@@ -56,6 +58,14 @@
5658
<tag name="translation.loader" alias="xliff" />
5759
</service>
5860

61+
<service id="translation.loader.po" class="%translation.loader.po.class%">
62+
<tag name="translation.loader" alias="po" />
63+
</service>
64+
65+
<service id="translation.loader.mo" class="%translation.loader.mo.class%">
66+
<tag name="translation.loader" alias="mo" />
67+
</service>
68+
5969
<service id="translation.loader.qt" class="%translation.loader.qt.class%">
6070
<tag name="translation.loader" alias="ts" />
6171
</service>

‎src/Symfony/Component/Translation/Loader/MoFileLoader.php‎

Lines changed: 169 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,169 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <[email protected]>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Translation\Loader;
13+
14+
use Symfony\Component\Config\Resource\FileResource;
15+
16+
/**
17+
* @copyright Copyright (c) 2010, Union of RAD http://union-of-rad.org (http://lithify.me/)
18+
*/
19+
class MoFileLoader extends ArrayLoader implements LoaderInterface
20+
{
21+
/**
22+
* Magic used for validating the format of a MO file as well as
23+
* detecting if the machine used to create that file was little endian.
24+
*
25+
* @var float
26+
*/
27+
const MO_LITTLE_ENDIAN_MAGIC = 0x950412de;
28+
29+
/**
30+
* Magic used for validating the format of a MO file as well as
31+
* detecting if the machine used to create that file was big endian.
32+
*
33+
* @var float
34+
*/
35+
const MO_BIG_ENDIAN_MAGIC = 0xde120495;
36+
37+
/**
38+
* The size of the header of a MO file in bytes.
39+
*
40+
* @var integer Number of bytes.
41+
*/
42+
const MO_HEADER_SIZE = 28;
43+
44+
public function load($resource, $locale, $domain = 'messages')
45+
{
46+
$messages = $this->parse($resource);
47+
48+
// empty file
49+
if (null === $messages) {
50+
$messages = array();
51+
}
52+
53+
// not an array
54+
if (!is_array($messages)) {
55+
throw new \InvalidArgumentException(sprintf('The file "%s" must contain a valid mo file.', $resource));
56+
}
57+
58+
$catalogue = parent::load($messages, $locale, $domain);
59+
$catalogue->addResource(new FileResource($resource));
60+
61+
return $catalogue;
62+
}
63+
64+
/**
65+
* Parses machine object (MO) format, independent of the machine's endian it
66+
* was created on. Both 32bit and 64bit systems are supported.
67+
*
68+
* @param resource $stream
69+
* @return array
70+
* @throws InvalidArgumentException If stream content has an invalid format.
71+
*/
72+
private function parse($resource)
73+
{
74+
$stream = fopen($resource, 'r');
75+
76+
$stat = fstat($stream);
77+
78+
if ($stat['size'] < self::MO_HEADER_SIZE) {
79+
throw new \InvalidArgumentException("MO stream content has an invalid format.");
80+
}
81+
$magic = unpack('V1', fread($stream, 4));
82+
$magic = hexdec(substr(dechex(current($magic)), -8));
83+
84+
if ($magic == self::MO_LITTLE_ENDIAN_MAGIC) {
85+
$isBigEndian = false;
86+
} elseif ($magic == self::MO_BIG_ENDIAN_MAGIC) {
87+
$isBigEndian = true;
88+
} else {
89+
throw new \InvalidArgumentException("MO stream content has an invalid format.");
90+
}
91+
92+
$header = array(
93+
'formatRevision' => null,
94+
'count' => null,
95+
'offsetId' => null,
96+
'offsetTranslated' => null,
97+
'sizeHashes' => null,
98+
'offsetHashes' => null,
99+
);
100+
foreach ($header as &$value) {
101+
$value = $this->readLong($stream, $isBigEndian);
102+
}
103+
extract($header);
104+
$messages = array();
105+
106+
for ($i = 0; $i < $count; $i++) {
107+
$singularId = $pluralId = null;
108+
$translated = null;
109+
110+
fseek($stream, $offsetId + $i * 8);
111+
112+
$length = $this->readLong($stream, $isBigEndian);
113+
$offset = $this->readLong($stream, $isBigEndian);
114+
115+
if ($length < 1) {
116+
continue;
117+
}
118+
119+
fseek($stream, $offset);
120+
$singularId = fread($stream, $length);
121+
122+
if (strpos($singularId, "\000") !== false) {
123+
list($singularId, $pluralId) = explode("\000", $singularId);
124+
}
125+
126+
fseek($stream, $offsetTranslated + $i * 8);
127+
$length = $this->readLong($stream, $isBigEndian);
128+
$offset = $this->readLong($stream, $isBigEndian);
129+
130+
fseek($stream, $offset);
131+
$translated = fread($stream, $length);
132+
133+
if (strpos($translated, "\000") !== false) {
134+
$translated = explode("\000", $translated);
135+
}
136+
137+
$ids = array('singular' => $singularId, 'plural' => $pluralId);
138+
$item = compact('ids', 'translated');
139+
140+
if (is_array($item['translated'])) {
141+
$messages[$item['ids']['singular']] = stripslashes($item['translated'][0]);
142+
if (isset($item['ids']['plural'])) {
143+
$messages[$item['ids']['plural']] = stripslashes(end($item['translated']));
144+
}
145+
} elseif($item['ids']['singular']) {
146+
$messages[$item['ids']['singular']] = stripslashes($item['translated']);
147+
}
148+
}
149+
150+
fclose($stream);
151+
152+
return array_filter($messages);
153+
}
154+
155+
/**
156+
* Reads an unsigned long from stream respecting endianess.
157+
*
158+
* @param resource $stream
159+
* @param boolean $isBigEndian
160+
* @return integer
161+
*/
162+
private function readLong($stream, $isBigEndian)
163+
{
164+
$result = unpack($isBigEndian ? 'N1' : 'V1', fread($stream, 4));
165+
$result = current($result);
166+
167+
return (integer) substr($result, -8);
168+
}
169+
}

‎src/Symfony/Component/Translation/Loader/PoFileLoader.php‎

Lines changed: 104 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,104 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <[email protected]>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Component\Translation\Loader;
13+
14+
use Symfony\Component\Config\Resource\FileResource;
15+
16+
/**
17+
* @copyright Copyright (c) 2010, Union of RAD http://union-of-rad.org (http://lithify.me/)
18+
*/
19+
class PoFileLoader extends ArrayLoader implements LoaderInterface
20+
{
21+
public function load($resource, $locale, $domain = 'messages')
22+
{
23+
$messages = $this->parse($resource);
24+
25+
// empty file
26+
if (null === $messages) {
27+
$messages = array();
28+
}
29+
30+
// not an array
31+
if (!is_array($messages)) {
32+
throw new \InvalidArgumentException(sprintf('The file "%s" must contain a valid po file.', $resource));
33+
}
34+
35+
$catalogue = parent::load($messages, $locale, $domain);
36+
$catalogue->addResource(new FileResource($resource));
37+
38+
return $catalogue;
39+
}
40+
41+
/**
42+
* Parses portable object (PO) format.
43+
*
44+
* This parser sacrifices some features of the reference implementation the
45+
* differences to that implementation are as follows.
46+
* - No support for comments spanning multiple lines.
47+
* - Translator and extracted comments are treated as being the same type.
48+
* - Message IDs are allowed to have other encodings as just US-ASCII.
49+
*
50+
* Items with an empty id are ignored.
51+
*
52+
* @param resource $stream
53+
* @return array
54+
*/
55+
private function parse($resource)
56+
{
57+
$stream = fopen($resource, 'r');
58+
59+
$defaults = array(
60+
'ids' => array(),
61+
'translated' => null,
62+
);
63+
64+
$messages = array();
65+
$item = $defaults;
66+
67+
while ($line = fgets($stream)) {
68+
$line = trim($line);
69+
70+
if ($line === '') {
71+
if (is_array($item['translated'])) {
72+
$messages[$item['ids']['singular']] = stripslashes($item['translated'][0]);
73+
if (isset($item['ids']['plural'])) {
74+
$messages[$item['ids']['plural']] = stripslashes(end($item['translated']));
75+
}
76+
} elseif($item['ids']['singular']) {
77+
$messages[$item['ids']['singular']] = stripslashes($item['translated']);
78+
}
79+
$item = $defaults;
80+
} elseif (substr($line, 0, 7) === 'msgid "') {
81+
$item['ids']['singular'] = substr($line, 7, -1);
82+
} elseif (substr($line, 0, 8) === 'msgstr "') {
83+
$item['translated'] = substr($line, 8, -1);
84+
} elseif ($line[0] === '"') {
85+
$continues = isset($item['translated']) ? 'translated' : 'ids';
86+
87+
if (is_array($item[$continues])) {
88+
end($item[$continues]);
89+
$item[$continues][key($item[$continues])] .= substr($line, 1, -1);
90+
} else {
91+
$item[$continues] .= substr($line, 1, -1);
92+
}
93+
} elseif (substr($line, 0, 14) === 'msgid_plural "') {
94+
$item['ids']['plural'] = substr($line, 14, -1);
95+
} elseif (substr($line, 0, 7) === 'msgstr[') {
96+
$item['translated'][(integer) substr($line, 7, 1)] = substr($line, 11, -1);
97+
}
98+
99+
}
100+
fclose($stream);
101+
102+
return array_filter($messages);
103+
}
104+
}

‎tests/Symfony/Tests/Component/Translation/Loader/MoFileLoaderTest.php‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <[email protected]>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Tests\Component\Translation\Loader;
13+
14+
use Symfony\Component\Translation\Loader\MoFileLoader;
15+
use Symfony\Component\Config\Resource\FileResource;
16+
17+
class MoFileLoaderTest extends \PHPUnit_Framework_TestCase
18+
{
19+
public function testLoad()
20+
{
21+
$loader = new MoFileLoader();
22+
$resource = __DIR__.'/../fixtures/resources.mo';
23+
$catalogue = $loader->load($resource, 'en', 'domain1');
24+
25+
$this->assertEquals(array('foo' => 'bar'), $catalogue->all('domain1'));
26+
$this->assertEquals('en', $catalogue->getLocale());
27+
$this->assertEquals(array(new FileResource($resource)), $catalogue->getResources());
28+
}
29+
30+
/**
31+
* @expectedException \InvalidArgumentException
32+
*/
33+
public function testLoadInvalidResource()
34+
{
35+
$loader = new MoFileLoader();
36+
$resource = __DIR__.'/../fixtures/empty.mo';
37+
$catalogue = $loader->load($resource, 'en', 'domain1');
38+
}
39+
}

‎tests/Symfony/Tests/Component/Translation/Loader/PoFileLoaderTest.php‎

Lines changed: 40 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,40 @@
1+
<?php
2+
3+
/*
4+
* This file is part of the Symfony package.
5+
*
6+
* (c) Fabien Potencier <[email protected]>
7+
*
8+
* For the full copyright and license information, please view the LICENSE
9+
* file that was distributed with this source code.
10+
*/
11+
12+
namespace Symfony\Tests\Component\Translation\Loader;
13+
14+
use Symfony\Component\Translation\Loader\PoFileLoader;
15+
use Symfony\Component\Config\Resource\FileResource;
16+
17+
class PoFileLoaderTest extends \PHPUnit_Framework_TestCase
18+
{
19+
public function testLoad()
20+
{
21+
$loader = new PoFileLoader();
22+
$resource = __DIR__.'/../fixtures/resources.po';
23+
$catalogue = $loader->load($resource, 'en', 'domain1');
24+
25+
$this->assertEquals(array('foo' => 'bar'), $catalogue->all('domain1'));
26+
$this->assertEquals('en', $catalogue->getLocale());
27+
$this->assertEquals(array(new FileResource($resource)), $catalogue->getResources());
28+
}
29+
30+
public function testLoadDoesNothingIfEmpty()
31+
{
32+
$loader = new PoFileLoader();
33+
$resource = __DIR__.'/../fixtures/empty.po';
34+
$catalogue = $loader->load($resource, 'en', 'domain1');
35+
36+
$this->assertEquals(array(), $catalogue->all('domain1'));
37+
$this->assertEquals('en', $catalogue->getLocale());
38+
$this->assertEquals(array(new FileResource($resource)), $catalogue->getResources());
39+
}
40+
}

‎tests/Symfony/Tests/Component/Translation/fixtures/empty.mo‎

Whitespace-only changes.

‎tests/Symfony/Tests/Component/Translation/fixtures/empty.po‎

Whitespace-only changes.

‎tests/Symfony/Tests/Component/Translation/fixtures/resources.mo‎

52 Bytes
Binary file not shown.

‎tests/Symfony/Tests/Component/Translation/fixtures/resources.po‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,3 @@
1+
msgid "foo"
2+
msgstr "bar"
3+

0 commit comments

Comments
 (0)