Fix problem when model maps integer id to a GoType #2657

stoikheia · 2022-06-16T16:24:55Z

Two errors occurred when I use integer PK in a DB and associate GoType in a model.

// Fields of the IntSid.
func (IntSid) Fields() []ent.Field {
	return []ent.Field{
		field.Int64("id").
			GoType(sid.New()).
			Unique().
			Immutable(),
	}
}

sid has been decrared in /ent/entc/integration/customid/sid/sid.go.

Errors

1.

intsid_create.go : (*IntSidCreateBulk) Save()

ent/intsid_create.go:394:48: invalid operation: nodes[i].ID == 0 (mismatched types sid.ID and untyped int)

2.

intsid.go : (*IntSid) assignValues()

ent/intsid.go:92:12: conversion from int64 to ID (string) yields a string of one rune, not a string of digits (did you mean fmt.Sprint(x)?)
ent/intsid.go:98:26: conversion from int64 to ID (string) yields a string of one rune, not a string of digits (did you mean fmt.Sprint(x)?)

Fix

1.

I took a way is similar to sqlSave() in *_create.go below to fix this problem.

ent/entc/gen/template/dialect/sql/create.tmpl

Line 23 in 7017cbc

ent/entc/gen/template/dialect/sql/create.tmpl

Lines 33 to 35 in 7017cbc

    
           {{- if $.ID.Type.ValueScanner }} 
        
           	} else if err := _node.ID.Scan(_spec.ID.Value); err != nil { 
        
           		return nil, err

Because, Code generator will check GoType implements ValueScanner.

ent/schema/field/field.go

Lines 1177 to 1182 in 7017cbc

    
           switch pt := reflect.PtrTo(t); { 
        
           case pt.Implements(valueScannerType), t.Implements(valueScannerType), 
        
           	t.Kind() == expectType.Kind() && t.ConvertibleTo(expectType): 
        
           default: 
        
           	d.Err = fmt.Errorf("GoType must be a %q type or ValueScanner", expectType) 
        
           }

2.

I fixed it so that it is assigned as the value of the column type of model when it is GoType.

It has the same behavior as String, Bytes, UUID and Other.

And I aggregated check condition IsUUID() and IsOther() into HasGoType().

Because it indicates true when HasGoType() is called if the column is set field.Other and field.UUID.

giautm

This PR look good to merge.

@a8m

entc/gen/template/dialect/sql/create.tmpl

entc/integration/customid/ent/schema/intsid.go

a8m · 2022-06-28T21:47:41Z

entc/integration/customid/customid_test.go

+	if strings.HasPrefix(t.Name(), "TestPostgres/") {
+		require.Equal(t, sid.ID("2"), iSIDChildren[0].ID)
+		require.Equal(t, sid.ID("3"), iSIDChildren[1].ID)
+	} else {
+		require.Equal(t, sid.ID("101"), iSIDChildren[0].ID)
+		require.Equal(t, sid.ID("102"), iSIDChildren[1].ID)
+	}


Can you elaborate on why we have this TestPostgres check? Why it's different than the rest?

First, the purpose of these tests is to check return values from ent whether they are set returned values from DB.

The reasons why the code of each RDB is different as follows.

Mysql and sqlite assign value greater than maximum value of the column when auto increment.

https://dev.mysql.com/doc/refman/8.0/en/example-auto-increment.html
"When you insert any other value into an AUTO_INCREMENT column, the column is set to that value and the sequence is reset so that the next automatically generated value follows sequentially from the largest column value. "
https://www.sqlite.org/autoinc.html
"The ROWID chosen for the new row is at least one larger than the largest ROWID"

Auto-increment of Postgres is provided as a type named Serial. ( of course, you know. ) And it behaves as a function nextval('tablename_colname_seq') internally.

https://www.postgresql.org/docs/13/datatype-numeric.html#DATATYPE-SERIAL
https://www.postgresql.org/docs/13/functions-sequence.html

Ent creates a table by SQL below when test of IntSID run.

CREATE TABLE "int_si_ds" ( "id" bigint NOT NULL GENERATED BY DEFAULT AS IDENTITY, "int_sid_parent" bigint NULL, PRIMARY KEY ("id"), CONSTRAINT "int_si_ds_int_si_ds_parent" FOREIGN KEY ("int_sid_parent") REFERENCES "int_si_ds" ("id") ON DELETE SET NULL )

https://www.geeksforgeeks.org/postgresql-identity-column/#:~:text=The%20GENERATED%20ALWAYS%20instructs%20PostgreSQL,value%20for%20the%20identity%20column.
"The GENERATED AS IDENTITY constraint is the SQL standard-conforming variant of the PostgreSQL’s SERIAL column."

Unfortunately, Serial will not associate with column value user inserted.

Therefore, SaveX() will not return "101" or "102" as ID after insertion of sid.ID("100") on Postgres.

Even worse, Postgres will occer an error if insert ID "2" instead of "100". Because it will conflict between sequence and exsisting value.

panic: ent: constraint failed: insert nodes to table "int_si_ds": pq: duplicate key value violates unique constraint "int_si_ds_pkey"

Therefore, I cannot unify the test Postgres and the others.

↓↓↓ edited ↓↓↓

A way to unify these tests, I will move "children id test" to before "ID assignment test".

However, it will spoil bit of robustness in test. It means that it lacks some states of the test are after user assignment id.
Additionally, this code has attention calling for developers about difference of each RDBs.

I unified the tests by rearranging the tests.
Because, fundamental purpose is just checking children IDs. This if-statement is noisy.
94163af

a8m · 2022-06-28T21:52:25Z

entc/integration/customid/customid_test.go

+	iSID := client.IntSID.Create().SaveX(ctx)
+	require.Equal(t, sid.ID("1"), iSID.ID)
+	iSIDChildID := sid.ID("100")
+	iSIDChild := client.IntSID.Create().SetID(iSIDChildID).SetParent(iSID).SaveX(ctx)
+	require.Equal(t, iSIDChildID, iSIDChild.ID)
+	require.Equal(t, iSID.ID, iSIDChild.QueryParent().OnlyX(ctx).ID)
+	iSIDBulk := make([]*ent.IntSIDCreate, 2)
+	iSIDBulk[0] = client.IntSID.Create().SetParent(iSID)
+	iSIDBulk[1] = client.IntSID.Create().SetParent(iSID)
+	iSIDChildren := client.IntSID.CreateBulk(iSIDBulk...).SaveX(ctx)


Suggested change

iSID := client.IntSID.Create().SaveX(ctx)

require.Equal(t, sid.ID("1"), iSID.ID)

iSIDChildID := sid.ID("100")

iSIDChild := client.IntSID.Create().SetID(iSIDChildID).SetParent(iSID).SaveX(ctx)

require.Equal(t, iSIDChildID, iSIDChild.ID)

require.Equal(t, iSID.ID, iSIDChild.QueryParent().OnlyX(ctx).ID)

iSIDBulk := make([]*ent.IntSIDCreate, 2)

iSIDBulk[0] = client.IntSID.Create().SetParent(iSID)

iSIDBulk[1] = client.IntSID.Create().SetParent(iSID)

iSIDChildren := client.IntSID.CreateBulk(iSIDBulk...).SaveX(ctx)

root := client.IntSID.Create().SaveX(ctx)

require.EqualValues(t, "1", root.ID)

cid := sid.ID("100")

child := client.IntSID.Create().SetID(cid).SetParent(root).SaveX(ctx)

require.Equal(t, cid, child.ID)

require.Equal(t, rott.ID, child.QueryParent().OnlyX(ctx).ID)

children := client.IntSID.CreateBulk(

client.IntSID.Create().SetParent(root),

client.IntSID.Create().SetParent(root)

).SaveX(ctx)

Give these variables shorter names, please 🙏

ok, but some variable names will conflict.
I'll wrap it by sub test.

in this way.
~~stoikheia@a6f4dbd~~

I pushed a commit.
5f5b11a
thanks

a8m · 2022-06-28T21:52:58Z

Looks great! Last minor comments, please.

Co-authored-by: Ariel Mashraki <[email protected]>

…to 20220608_fix_templates2

Wrapped tests of IntSID for avoid conflicts.

Order of tests was changed. As a result, if-statement for difference in RDB is deleted. + force pushing same commit to invoke ci.

a8m

🚀

Amazing @stoikheia! Thanks for working on it and I'll love to see you on our Discord server.

stoikheia added 2 commits June 16, 2022 17:08

Fix problem when model maps integer id to a GoType

77601fb

Update generated files

3a7e622

giautm approved these changes Jun 23, 2022

View reviewed changes

a8m reviewed Jun 23, 2022

View reviewed changes

entc/gen/template/dialect/sql/create.tmpl Outdated Show resolved Hide resolved

entc/gen/template/dialect/sql/create.tmpl Outdated Show resolved Hide resolved

entc/integration/customid/ent/schema/intsid.go Outdated Show resolved Hide resolved

stoikheia added 4 commits June 24, 2022 06:24

apply suggestions from code review

cb7ee1b

Update generated files

0406176

Merge branch 'master' into 20220608_fix_templates2

bd23d28

Update generated files after merge from master

f438e2e

a8m reviewed Jun 28, 2022

View reviewed changes

stoikheia and others added 4 commits July 1, 2022 09:01

Cleanup test code in integration

c60a2f4

Co-authored-by: Ariel Mashraki <[email protected]>

Merge remote-tracking branch 'origin' into 20220608_fix_templates2

3721108

Merge branch '20220608_fix_templates2' of github.com:stoikheia/ent in…

7a9d0ca

…to 20220608_fix_templates2

Modify integration test IntSID.

5f5b11a

Wrapped tests of IntSID for avoid conflicts.

stoikheia force-pushed the 20220608_fix_templates2 branch 3 times, most recently from ebae19f to abfb905 Compare July 1, 2022 18:08

Modify tests in integrations

3085e25

Order of tests was changed. As a result, if-statement for difference in RDB is deleted. + force pushing same commit to invoke ci.

stoikheia force-pushed the 20220608_fix_templates2 branch from abfb905 to 3085e25 Compare July 1, 2022 18:10

a8m approved these changes Jul 4, 2022

View reviewed changes

a8m merged commit 91b6430 into ent:master Jul 4, 2022

timoha mentioned this pull request Oct 7, 2022

custom window function selector timoha/ent#1

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix problem when model maps integer id to a GoType #2657

Fix problem when model maps integer id to a GoType #2657

Uh oh!

stoikheia commented Jun 16, 2022 •

edited

Loading

Uh oh!

giautm left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

a8m Jun 28, 2022

Uh oh!

stoikheia Jun 29, 2022 •

edited

Loading

Uh oh!

stoikheia Jul 1, 2022 •

edited

Loading

Uh oh!

a8m Jun 28, 2022

Uh oh!

stoikheia Jun 29, 2022 •

edited

Loading

Uh oh!

stoikheia Jul 1, 2022 •

edited

Loading

Uh oh!

a8m commented Jun 28, 2022

Uh oh!

a8m left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	{{- if $.ID.Type.ValueScanner }}
	} else if err := _node.ID.Scan(_spec.ID.Value); err != nil {
	return nil, err

	switch pt := reflect.PtrTo(t); {
	case pt.Implements(valueScannerType), t.Implements(valueScannerType),
	t.Kind() == expectType.Kind() && t.ConvertibleTo(expectType):
	default:
	d.Err = fmt.Errorf("GoType must be a %q type or ValueScanner", expectType)
	}

Fix problem when model maps integer id to a GoType #2657

Fix problem when model maps integer id to a GoType #2657

Uh oh!

Conversation

stoikheia commented Jun 16, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Errors

1.

2.

Fix

1.

2.

Uh oh!

giautm left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

a8m Jun 28, 2022

Choose a reason for hiding this comment

Uh oh!

stoikheia Jun 29, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stoikheia Jul 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

a8m Jun 28, 2022

Choose a reason for hiding this comment

Uh oh!

stoikheia Jun 29, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

stoikheia Jul 1, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

a8m commented Jun 28, 2022

Uh oh!

a8m left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

stoikheia commented Jun 16, 2022 •

edited

Loading

stoikheia Jun 29, 2022 •

edited

Loading

stoikheia Jul 1, 2022 •

edited

Loading

stoikheia Jun 29, 2022 •

edited

Loading

stoikheia Jul 1, 2022 •

edited

Loading