Thanks to visit codestin.com
Credit goes to github.com

Skip to content

[Yaml] Fix Yaml Parser with quote end in a new line #48022

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Feb 3, 2024
Merged

[Yaml] Fix Yaml Parser with quote end in a new line #48022

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
57 changes: 57 additions & 0 deletions src/Symfony/Component/Yaml/Parser.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -597,8 +597,12 @@ private function getNextEmbedBlock(?int $indentation = null, bool $inSequence =
}

$data = [];
$isInMultiLineQuote = false;

if ($this->getCurrentLineIndentation() >= $newIndent) {
if ($this->isCurrentLineMultiLineQuoteStart()) {
$isInMultiLineQuote = true;
}
$data[] = substr($this->currentLine, $newIndent ?? 0);
} elseif ($this->isCurrentLineEmpty() || $this->isCurrentLineComment()) {
$data[] = $this->currentLine;
Expand Down Expand Up @@ -635,6 +639,16 @@ private function getNextEmbedBlock(?int $indentation = null, bool $inSequence =
if ($this->isCurrentLineBlank()) {
$data[] = substr($this->currentLine, $newIndent);
continue;
} elseif (!$isInMultiLineQuote && $this->isCurrentLineMultiLineQuoteStart()) {
$isInMultiLineQuote = true;
$data[] = substr($this->currentLine, $newIndent);
continue;
} elseif ($isInMultiLineQuote) {
$data[] = $this->currentLine;
if ("'" === (rtrim($this->currentLine)[-1] ?? '')) {
$isInMultiLineQuote = false;
}
continue;
}

if ($indent >= $newIndent) {
Expand Down Expand Up @@ -965,6 +979,49 @@ private function isCurrentLineLastLineInDocument(): bool
return ($this->offset + $this->currentLineNb) >= ($this->totalNumberOfLines - 1);
}

/**
* Returns true if the current line is the beginning of a multiline quoted block.
*/
private function isCurrentLineMultiLineQuoteStart(): bool
{
$trimmedLine = trim($this->currentLine);
$trimmedLineLength = \strlen($trimmedLine);
$quoteCount = 0;
$value = '';
// check if the key is quoted
for ($i = 0; $i < $trimmedLineLength; ++$i) {
$char = $trimmedLine[$i];
if ("'" === $char) {
++$quoteCount;
} elseif (':' === $char && 0 === $quoteCount % 2 && ($i === $trimmedLineLength - 1 || ' ' === $trimmedLine[$i + 1])) {
// key and value are separated by the first colon after the (quoted) key followed by a space or linebreak
$value = trim(substr($trimmedLine, ++$i), ' ');
break;
}
}

if (0 !== strpos($value, "'")) {
return false;
}

$lineEndQuoteCount = 0;
for ($i = \strlen($value) - 1; $i > 0; --$i) {
$char = $value[$i];
if ("'" === $char) {
++$lineEndQuoteCount;
} else {
break;
}
}

return 0 === $lineEndQuoteCount % 2;
}

/**
* Cleanups a YAML string to be parsed.
*
* @param string $value The input YAML string
*/
private function cleanup(string $value): string
{
$value = str_replace(["\r\n", "\r"], "\n", $value);
Expand Down
51 changes: 51 additions & 0 deletions src/Symfony/Component/Yaml/Tests/YamlTest.php
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
namespace Symfony\Component\Yaml\Tests;

use PHPUnit\Framework\TestCase;
use Symfony\Component\Yaml\Exception\ParseException;
use Symfony\Component\Yaml\Yaml;

class YamlTest extends TestCase
Expand All @@ -24,6 +25,56 @@ public function testParseAndDump()
$this->assertEquals($data, $parsed);
}

public function testParseWithMultilineQuotes()
{
$yaml = <<<YAML
foo:
bar: 'baz
biz

'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not valid YAML AFAIU. The quote must be indented.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Then we should close this PR and the issue with "Won't fix", because this is exactly the case, what is requested to be fixed!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I am not mistaken, this is indeed valid YAML.

Copy link
Contributor Author

@maxbeckers maxbeckers Feb 10, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @xabbuh and @fabpot,

I have taken some time to read up on this YAML topic. In the internet are two kinds of YAML parser for this topic. Either the content must be indented even if it is in quotes or everything in quotes can be indented free, because the part in quotes is simply considered as a text block.

What I want to say is that the PR should not be merged as it is and we should decide here how to proceed.

Solution A: everything should be how it is and we only support indented quotes, seems to be how it's described in the spec https://yaml.org/spec/1.2.2/#8111-block-indentation-indicator.

foo:
  bar: 'baz

       '

Solution B: be a bit more generous and allow what requested in the issue and how a lot of online parsers/validators see the quoted part as a string and allow stuff like that:

foo:
  bar: 'baz1
baz2
                baz3

'

Based on that I'd prefer solution A and keep it how it is as @fabpot mentioned. What do you think?

baz: 'Lorem

ipsum'
error: Une erreur s'est produite.
trialMode: 'période d''essai'
double_line: 'Les utilisateurs sélectionnés
n''ont pas d''email.

'
a: 'b''
c'
empty: ''
foo:bar: 'foobar'
YAML;

$this->assertSame(['foo' => [
'bar' => "baz biz\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
'bar' => "baz biz\n",
'bar' => 'baz biz ',

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the behavior you expect @xabbuh? Currently with this input we get this result without your suggestion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, as far as I know that should be the outcome.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the YAML spec I didn't find sth. for this case so it might be a very special case. This is the ChatGPT answer for that:

An empty line is treated as a line break (\n)

This is also the way it works in different online interpreters which are supporting this (like this one: http://www.yaml-online-parser.appspot.com/.
This one also interpretes

baz: 'Lorem

   ipsum'

As {"baz": "Lorem\nipsum"}.
While other interpreters like https://codebeautify.org/yaml-parser-online interpret this as {"baz": "Lorem ipsum"}.

So it seems to be not really clear how to deal with this and we should decide how to deal with it. My suggestion would be "keep it simple for this very special case" and keep it how it is atm in this PR with the \n. What do you think?

'baz' => "Lorem\nipsum",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
'baz' => "Lorem\nipsum",
'baz' => 'Lorem ipsum',

'error' => "Une erreur s'est produite.",
'trialMode' => "période d'essai",
'double_line' => "Les utilisateurs sélectionnés n'ont pas d'email.\n",
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
'double_line' => "Les utilisateurs sélectionnés n'ont pas d'email.\n",
'double_line' => "Les utilisateurs sélectionnés n'ont pas d'email. ",

'a' => "b' c",
'empty' => '',
'foo:bar' => 'foobar',
]], Yaml::parse($yaml));
}

public function testParseWithMultilineQuotesExpectException()
{
$yaml = <<<YAML
foo:
bar: 'baz

'
'
YAML;

$this->expectException(ParseException::class);
$this->expectExceptionMessage('Unable to parse at line 5 (near "\'").');
Yaml::parse($yaml);
}

public function testZeroIndentationThrowsException()
{
$this->expectException(\InvalidArgumentException::class);
Expand Down