Rationalize name-char #1008

macchiati · 2025-02-16T01:00:05Z

#724 Rationalize name-char

#724

macchiati · 2025-02-16T01:27:58Z

Here is a first cut at changes for name-char. Feedback welcome

eemeli

A rather cursory initial pass.

In addition to line comments, this PR needs to be accompanied by some test cases.

eemeli · 2025-02-16T09:55:21Z

spec/message.abnf

+                  / %xA1-61B      ;          omit Cc %x7F-9F, Whitespace %xA0, Ascii 【`】 【{|}~】
+                  / %x61D-167F    ;          omit BidiControl %x61C
+                  / %x1681-1FFF   ;          omit Whitespace %x1680


In the other character range definitions, the "omit" comments are offset by one compared to how they're here, as in (showing these three lines only as an example):

Suggested change

/ %xA1-61B ; omit Cc %x7F-9F, Whitespace %xA0, Ascii 【`】【{|}~】

/ %x61D-167F ; omit BidiControl %x61C

/ %x1681-1FFF ; omit Whitespace %x1680

/ %xA1-61B ; omit BidiControl %x61C

/ %x61D-167F ; omit Whitespace %x1680

/ %x1681-1FFF ; omit Whitespace %x2000-200A

The same style should be used in all these comments.

I don't see a difference on my screen. Do you want an additional space before or after 'omit', or a space deleted before or after 'omit'.

I changed the EA brackets to guillemets, since they line up better for monospace.

I meant offset in a vertical direction, so a comment like "omit BidiControl %x61C" should follow the range %xA1-61B, rather than the range %x61D-167F.

Ok, will work on that.

eemeli · 2025-02-16T09:56:41Z

spec/message.abnf

+                  / %xA1-61B      ;          omit Cc %x7F-9F, Whitespace %xA0, Ascii 【`】 【{|}~】
+                  / %x61D-167F    ;          omit BidiControl %x61C
+                  / %x1681-1FFF   ;          omit Whitespace %x1680
+                  / %x200B-200D   ;          omit Whitespace %x2000-200A


This set is ZWSP, ZWNJ, and ZWJ. Should they really be included in name-start? That seems surprising to me, and with no positive utility.

We will need ZWNJ and ZWJ within names, though, so maybe it's fine for them to be here. But why ZWSP?

We still have name-char. Why not put the joiners in there?

I kind of also question ZWSP

There is no particular utility to having ZWSP start a name-char, nor any utility to having it end a name-char. (

It doesn't hurt to move that one (ZWSP) to name-char, but it doesn't really make a dent either — and we really wouldn't want to go too far down the very long and slippery slope. That's for linters and guidance.

That being said, if people want it out I can remove it.

The question here is why put these characters in name-start, where they have no utility? At least in name-char they would be enclosed or at the end?

What I'm afraid of is that if we move that to name-char, it will just open it up to people endlessly complaining that:

"ZWSP" is in name-chart instead of name-start: why is XXX in name-start when it should also be just be in name-chart???:

That is, the basic difference between name-char and name-start is that

name is used in identifiers and variables, and can't start with a digit, -, or .

name-char is used in literals, and can start with digit, -, .

The syntactic motivation is clear: to make sure that identifiers and variables are distinguishable from numbers. That is a clear syntactic need.

ZWSP certainly isn't needed at the start of an identifier or variable, but there is an large and complicated list of characters that are also not needed at start of identifiers and variables, and plucking just one of those characters out, without any syntactic need, doesn't actually provide much value.

Given that we do not allow space characters or control characters, I'd prefer not allowing zero-width spaces in names or unquoted literals.

A zero-width space is not a space; that is just a name used for familiarity. It is a Format character, like many others.

https://util.unicode.org/UnicodeJsps/list-unicodeset.jsp?a=%5Cp%7Bgc%3Dformat%7D&g=&i=

spec/message.abnf

spec/syntax.md

aphillips

A good start. Some small things.

spec/message.abnf

spec/syntax.md

Co-authored-by: Eemeli Aro <[email protected]>

Co-authored-by: Addison Phillips <[email protected]>

eemeli · 2025-02-17T12:17:51Z

This also needs to be fixed:

message-format-wg/spec/syntax.md

Lines 789 to 793 in 1fccd1e

    
           Valid content for _names_ is based on <cite>Namespaces in XML 1.0</cite>'s  
        
           [NCName](https://www.w3.org/TR/xml-names/#NT-NCName). 
        
           This is different from XML's [Name](https://www.w3.org/TR/xml/#NT-Name) 
        
           in that it MUST NOT contain a U+003A COLON `:`. 
        
           Otherwise, the set of characters allowed in a _name_ is large.

eemeli · 2025-02-17T15:36:35Z

spec/syntax.md

+They are similar to <cite>Namespaces in XML 1.0</cite>'s [NCName](https://www.w3.org/TR/xml-names/#NT-NCName),
+but have been updated to be more consistent.


Given how far we've departed from that spec, this note seems only relevant within the history of the spec, and not with where we're ending up with this PR. I'd prefer dropping it, and moving the preceding sentence (if keeping) above the preceding note.

spec/message.abnf

spec/syntax.md

Co-authored-by: Addison Phillips <[email protected]> Co-authored-by: Eemeli Aro <[email protected]>

gibson042 · 2025-02-17T17:25:54Z

spec/message.abnf

+                  / %x2B            ; «+»      omit Ascii: «,-./0123456789:;<=>?@» «[\]^»
+                  / %x5F            ; «_»      omit Cc: %x7F-9F, Whitespace: %xA0, Ascii: «`» «{|}~»


Why are there two sets of guillemet-quoted ASCII characters here?

Suggested change

/ %x2B ; «+» omit Ascii: «,-./0123456789:;<=>?@» «[\]^»

/ %x5F ; «_» omit Cc: %x7F-9F, Whitespace: %xA0, Ascii: «`» «{|}~»

/ %x2B ; «+» omit Ascii: «,-./0123456789:;<=>?@[]^» and REVERSE SOLIDUS "\"

/ %x5F ; «_» omit Ascii: «`{|}~», Cc: %x7F-9F, Whitespace: %xA0

…er of unquoted-literal (unicode-org/message-format-wg#990, unicode-org/message-format-wg#1008)

macchiati added 4 commits February 15, 2025 16:59

Rationalize name-char

3293cba

#724

Make corresponding changes in syntax.md

677878d

Fix long lines

5ad3a72

Fix long lines 2

f0d9d54

macchiati requested review from eemeli and aphillips February 16, 2025 01:27

macchiati marked this pull request as ready for review February 16, 2025 01:27

eemeli requested changes Feb 16, 2025

View reviewed changes

aphillips requested changes Feb 16, 2025

View reviewed changes

spec/message.abnf Outdated Show resolved Hide resolved

spec/syntax.md Outdated Show resolved Hide resolved

spec/syntax.md Outdated Show resolved Hide resolved

aphillips and others added 3 commits February 16, 2025 07:47

Update spec/syntax.md

39ece5a

Co-authored-by: Eemeli Aro <[email protected]>

Apply suggestions from code review

14eb173

Co-authored-by: Addison Phillips <[email protected]>

Review comment re XML

0eb3c22

aphillips added syntax Issues related with syntax or ABNF normative Issue affects normative text in the specification LDML47 LDML 47 Release (Stable) labels Feb 16, 2025

macchiati added 2 commits February 16, 2025 21:03

Drop Cs

70c3c9c

Remove Cs from exclusions

a668429

Fix reference to XML

479e41c

eemeli reviewed Feb 17, 2025

View reviewed changes

macchiati added 3 commits February 17, 2025 07:49

Put 'omit' on line before

1bf206a

Put omit on previous line

8ff73f7

Drop XML reference

074346a

aphillips reviewed Feb 17, 2025

View reviewed changes

spec/message.abnf Outdated Show resolved Hide resolved

aphillips approved these changes Feb 17, 2025

View reviewed changes

eemeli reviewed Feb 17, 2025

View reviewed changes

spec/message.abnf Outdated Show resolved Hide resolved

spec/syntax.md Outdated Show resolved Hide resolved

Apply suggestions from code review

bffc098

Co-authored-by: Addison Phillips <[email protected]> Co-authored-by: Eemeli Aro <[email protected]>

eemeli approved these changes Feb 17, 2025

View reviewed changes

gibson042 reviewed Feb 17, 2025

View reviewed changes

aphillips merged commit b343bfa into main Feb 17, 2025
1 check passed

aphillips deleted the macchiati-i724-rationalize-name-char branch February 17, 2025 18:04

eemeli mentioned this pull request Feb 18, 2025

Add tests for extended name character range #1020

Merged

eemeli added a commit to messageformat/messageformat that referenced this pull request Mar 2, 2025

feat(mf2): Rationalize name-char and allow name-char as first charact…

6aa468c

…er of unquoted-literal (unicode-org/message-format-wg#990, unicode-org/message-format-wg#1008)

eemeli mentioned this pull request Mar 2, 2025

MF2 spec updates for LDML 47 release messageformat/messageformat#440

Merged

		They are similar to <cite>Namespaces in XML 1.0</cite>'s [NCName](https://www.w3.org/TR/xml-names/#NT-NCName),
		but have been updated to be more consistent.

		/ %x2B ; «+» omit Ascii: «,-./0123456789:;<=>?@» «[\]^»
		/ %x5F ; «_» omit Cc: %x7F-9F, Whitespace: %xA0, Ascii: «`» «{\|}~»

Uh oh!

Rationalize name-char #1008

Rationalize name-char #1008

Uh oh!

Conversation

macchiati commented Feb 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

macchiati commented Feb 16, 2025

Uh oh!

eemeli left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

macchiati Feb 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

aphillips left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

eemeli commented Feb 17, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

macchiati commented Feb 16, 2025 •

edited

Loading

macchiati Feb 16, 2025 •

edited

Loading