[Meteor 3] (server-first) Solving Meteor call Async stubValuePromise issue #12907

Grubba27 · 2023-11-29T19:13:35Z

As discussed in #12888 and #12897 we need to come to a decision in this API:

For this PR we have implemented the comment here:

const promise = Meteor.callAsync(...);

const resultFromStub = await promise.stub;
const resultFromServer = await promise;

// backwards compatible with existing code:
const resultFromServer = await Meteor.callAsync(...);

Fixes 'Error: A method named 'echo' is already defined' on Travis

Grubba27 · 2023-11-30T17:49:00Z

packages/ddp-client/test/livedata_tests.js

-    if (Meteor.isClient)
-      // client can fool itself by cheating, but only until the sync
-      // finishes
-      await checkBalances(test, -10, 160);
-    else await checkBalances(test, 90, 60);


@zodern maybe from this test, the api should be stub first, then the server result? because now it needs to await for the server result.
Not sure if this is correct

If you look at the old code for the test:

meteor/packages/ddp-client/test/livedata_tests.js

Lines 377 to 397 in e076317

function(test, expect) {

Meteor.call(

'ledger/transfer',

test.runId(),

'alice',

'bob',

100,

true,

expect(function(err, result) {

failure(test, 409)(err, result);

// Balances are reverted back to pre-stub values.

checkBalances(test, 90, 60);

})

);

if (Meteor.isClient)

// client can fool itself by cheating, but only until the sync

// finishes

checkBalances(test, -10, 160);

else checkBalances(test, 90, 60);

}

You could maybe implement it for Meteor 3 like this:

let promise = Meteor.callAsync( 'ledger/transfer', test.runId(), 'alice', 'bob', 100, true, ); if (Meteor.isClient) { // client can fool itself by cheating, but only until the sync // finishes await promise.stub await checkBalances(test, -10, 160); } await promise.catch(err => { failure(test, 409)(err); }); // Balances are reverted back to pre-stub values. await checkBalances(test, 90, 60);

I think I made a mistake somewhere in here

meteor/packages/ddp-client/common/livedata_connection.js

Lines 891 to 907 in fd2150c

// block waiting for the result.

if (future) {

future.stub = stubReturnValue;

future.serverResult = future;

if (options.returnServerPromise) {

return future;

}

if (options.returnStubValue) {

return future.then(() => stubReturnValue);

}

return future;

}

return options.returnStubValue ? stubReturnValue : undefined;

}

The test still does not pass

I took at look at this over the weekend. Unfortunately wasn't able to make the test pass. I noticed that checkBalances on the client is running before the stub promise has completed. When I added a timeout of 1, prior to checkBalances it passed.

if (Meteor.isClient) { // client can fool itself by cheating, but only until the sync // finishes await promise.stub // this isn't being awaited // if i add -- await sleep(1) -- it passes await checkBalances(test, -10, 160); }

Hope this can be of help. Maybe someone closer to the codebase will be able to instantly tell what the issue is. 🙂 If I get some time this week, I will take a closer look.

Based on what @jamauro found, it probably is because promise is the promise returned by .catch instead of the promise returned by callAsync. In the example code I shared, the .catch is done separately which avoids the issue.

Another issue is the code failure(test, 409)(err); needs to be after the .catch instead of inside it. Otherwise, the check will never run if the method succeeds.

jamauro · 2023-11-30T18:09:26Z

Will this solution enable getting the stub result before the server promise resolves? It's not clear to me if await promise.stub is awaiting the promise and then grabbing the stub or if it's somehow awaiting a stubPromise.

If not, is there a reason why something like this approach wouldn't work?

Grubba27 · 2023-11-30T18:50:23Z

If not, is there a reason why something like this approach wouldn't work?

It could work... this is more of a discussion of what the API should look like, and we could have both, as you suggested.

This PR focuses on being server-first.

because it defaults to be the server result, and now we are async. That test needed to be changed(not sure if I like this).

Not sure if I understood you question @jamauro, with this proposal we can do this.

const { stub } = Meteor.callAsync(...);
const resultFromStub = await stub;

And while coding this example I realize that makes sense as well to have server result as well in this object.

jamauro · 2023-11-30T19:08:50Z

Ok. I trust that y'all will come up with the best solution. 🙂

I mainly wanted to ensure that I could use the stub result without waiting on the server and then if the server errors, I could rollback the action I took with the stub result. I don't do this often but it can be kind of nice when working with a newly inserted id.

Grubba27 · 2023-11-30T19:19:48Z

@jamauro I've added your idea. You can also do this:

const { stub, serverResult }= Meteor.callAsync(...);
const whatServerDid = await serverResult;
const whatStubDid = await stub;

// or just

const whatServerDid2 = await Meteor.callAsync(...)

zodern · 2023-12-04T18:10:58Z

packages/ddp-client/common/livedata_connection.js

+      future.stub = stubReturnValue;
+      future.serverResult = future;


I would suggest renaming these to stubPromise and serverPromise for consistency, and to make it clear what the properties contain.

As far as I can tell, stubReturnValue is currently a value and not a promise that is passed into _apply from applyAsync. I think the code needs to be changed upstream of future.stubPromise = stubReturnValue in order for things to work as expected. I could be wrong though. There's a lot going on in this code :)

meteor/packages/ddp-client/common/livedata_connection.js

Line 721 in fd2150c

stubOptions.stubReturnValue = await stubInvocation();

Also it's not clear to me why stubInvocation is being awaited here when it is not a promise:

meteor/packages/ddp-client/common/livedata_connection.js

Lines 972 to 982 in fd2150c

const stubInvocation = () => {

if (Meteor.isServer) {

// Because saveOriginals and retrieveOriginals aren't reentrant,

// don't allow stubs to yield.

return Meteor._noYieldsAllowed(() => {

// re-clone, so that the stub can't affect our caller's values

return stub.apply(invocation, EJSON.clone(args));

});

} else {

return stub.apply(invocation, EJSON.clone(args));

}

rj-david · 2023-12-05T06:35:36Z

@jamauro I've added your idea. You can also do this:

const { stub, serverResult }= Meteor.callAsync(...);
const whatServerDid = await serverResult;
const whatStubDid = await stub;

// or just

const whatServerDid2 = await Meteor.callAsync(...)

This is stretching my developer mind. Every time I see this, I feel something is wrong.

Just a quick litmus test: Does the reason for having two interfaces for the result of Meteor.callAsync() outweigh all the hardships of explaining why there are two interfaces?

…c: stubPromise and serverPromise

…or the client

- bring zodern's solution to async stubs

…lpers - fix empty documents tests - fix database error reporting tests

- stop changing how Meteor.apply works

- bring back Meteor.apply in the stub queue

… into feature/solving-meteor-callasync-stubvaluepromise # Conflicts: # packages/ddp-client/client/client_convenience.js # packages/ddp-client/package.js

packages/ddp-client/test/async_stubs/client.js

packages/ddp-client/client/queueStubsHelpers.js

packages/ddp-client/client/client_convenience.js

packages/ddp-client/client/queueStubsHelpers.js

…valuepromise # Conflicts: # packages/accounts-2fa/.npm/package/npm-shrinkwrap.json # packages/email/.npm/package/npm-shrinkwrap.json # packages/npm-mongo/.npm/package/npm-shrinkwrap.json

- require Groups before Cleaning up

- inject new stub async helpers in the Connection.prototype - adjust tests to work with the new api

Grubba27 and others added 4 commits November 21, 2023 16:42

Add zodern solution

a5b0bb5

Removes duplicated echo method in tests of ddp-client

bd3e9c7

Fixes 'Error: A method named 'echo' is already defined' on Travis

Queue subscriptions

17d026f

Feat: zodern solution to stubvalue promise

f289678

Grubba27 requested a review from zodern November 29, 2023 19:13

github-actions bot added the Project:DDP label Nov 29, 2023

fredmaiaarantes requested review from radekmie and denihs November 29, 2023 19:16

Grubba27 mentioned this pull request Nov 29, 2023

Add zodern solution for Async stubs #12897

Merged

Grubba27 added 3 commits November 29, 2023 16:34

Tests: remove stubValuePromise from ddp-client

6665cf7

Tests: remove stubValuePromise from ddp-server

2c62fec

Tests: remove stubValuePromise from mongo_livedata

29522f1

github-actions bot added the Project:Mongo Driver label Nov 29, 2023

made compound methods isomorphic

2681977

Grubba27 commented Nov 30, 2023

View reviewed changes

add option to use serverResult

86425b9

update with zodern solution

fd2150c

StorytellerCZ added this to the Release 3.0 milestone Dec 1, 2023

zodern reviewed Dec 4, 2023

View reviewed changes

Grubba27 added 2 commits December 4, 2023 16:50

Update livedata with correct naming and explaining what is every prop

3b764fd

Update test with Zodern ideia

0e68376

denihs added 4 commits December 14, 2023 11:28

- expose in a proper way the new attributes for callAsync / applyAsyn…

5a34170

…c: stubPromise and serverPromise

- little adjust inside applyAsync to expose the new attributes only f…

c143fa7

…or the client

- fix some accounts tests

2b7a1df

- bring zodern's solution to async stubs

- Make sure Meteor.connect is already created before injecting the he…

c6e1754

…lpers - fix empty documents tests - fix database error reporting tests

denihs added 5 commits December 16, 2023 10:39

- Make sure Meteor.connect is already created before injecting the he…

6117340

…lpers - fix empty documents tests - fix database error reporting tests

- fix document with length

1a7cb65

- fix mongo-livedata - consistent _id generation tests

af137ba

- stop changing how Meteor.apply works

- remove unnecessary use of returnServerPromise

f38936a

- bring back Meteor.apply in the stub queue

- fix update EJSON

fcdbcaf

github-actions bot added Project:Accounts:Password Project:Tool labels Dec 18, 2023

denihs added 4 commits December 19, 2023 09:12

update blaze commit hash

29305e6

force CI tests to run again

52c2e25

Merge remote-tracking branch 'origin/zodern-solution-for-async-stubs'…

3a699bc

… into feature/solving-meteor-callasync-stubvaluepromise # Conflicts: # packages/ddp-client/client/client_convenience.js # packages/ddp-client/package.js

- adjusting applyAsync stub async method

95e4430

denihs mentioned this pull request Dec 19, 2023

Async Methods with Callback #12677

Closed

zodern reviewed Dec 19, 2023

View reviewed changes

denihs added 3 commits December 21, 2023 16:55

Merge branch 'release-3.0' into feature/solving-meteor-callasync-stub…

cdff03b

…valuepromise # Conflicts: # packages/accounts-2fa/.npm/package/npm-shrinkwrap.json # packages/email/.npm/package/npm-shrinkwrap.json # packages/npm-mongo/.npm/package/npm-shrinkwrap.json

- use circle ci image with node 20

1b7f2db

- require Groups before Cleaning up

- include new subscribe test

ad6dae7

- inject new stub async helpers in the Connection.prototype - adjust tests to work with the new api

denihs merged commit f0b4e16 into release-3.0 Dec 22, 2023

Grubba27 mentioned this pull request Dec 28, 2023

Release 3.0 #12359

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

[Meteor 3] (server-first) Solving Meteor call Async stubValuePromise issue #12907

[Meteor 3] (server-first) Solving Meteor call Async stubValuePromise issue #12907

Uh oh!

Grubba27 commented Nov 29, 2023

Uh oh!

Grubba27 Nov 30, 2023

Uh oh!

zodern Nov 30, 2023 •

edited

Loading

Uh oh!

Grubba27 Nov 30, 2023

Uh oh!

jamauro Dec 4, 2023

Uh oh!

zodern Dec 4, 2023

Uh oh!

jamauro commented Nov 30, 2023

Uh oh!

Grubba27 commented Nov 30, 2023

Uh oh!

jamauro commented Nov 30, 2023

Uh oh!

Grubba27 commented Nov 30, 2023

Uh oh!

zodern Dec 4, 2023

Uh oh!

jamauro Dec 4, 2023 •

edited

Loading

Uh oh!

rj-david commented Dec 5, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

	function(test, expect) {
	Meteor.call(
	'ledger/transfer',
	test.runId(),
	'alice',
	'bob',
	100,
	true,
	expect(function(err, result) {
	failure(test, 409)(err, result);
	// Balances are reverted back to pre-stub values.
	checkBalances(test, 90, 60);
	})
	);

	if (Meteor.isClient)
	// client can fool itself by cheating, but only until the sync
	// finishes
	checkBalances(test, -10, 160);
	else checkBalances(test, 90, 60);
	}

	// block waiting for the result.
	if (future) {
	future.stub = stubReturnValue;
	future.serverResult = future;

	if (options.returnServerPromise) {
	return future;
	}

	if (options.returnStubValue) {
	return future.then(() => stubReturnValue);
	}

	return future;
	}
	return options.returnStubValue ? stubReturnValue : undefined;
	}

	const stubInvocation = () => {
	if (Meteor.isServer) {
	// Because saveOriginals and retrieveOriginals aren't reentrant,
	// don't allow stubs to yield.
	return Meteor._noYieldsAllowed(() => {
	// re-clone, so that the stub can't affect our caller's values
	return stub.apply(invocation, EJSON.clone(args));
	});
	} else {
	return stub.apply(invocation, EJSON.clone(args));
	}

[Meteor 3] (server-first) Solving Meteor call Async stubValuePromise issue #12907

[Meteor 3] (server-first) Solving Meteor call Async stubValuePromise issue #12907

Uh oh!

Conversation

Grubba27 commented Nov 29, 2023

Uh oh!

Grubba27 Nov 30, 2023

Choose a reason for hiding this comment

Uh oh!

zodern Nov 30, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Grubba27 Nov 30, 2023

Choose a reason for hiding this comment

Uh oh!

jamauro Dec 4, 2023

Choose a reason for hiding this comment

Uh oh!

zodern Dec 4, 2023

Choose a reason for hiding this comment

Uh oh!

jamauro commented Nov 30, 2023

Uh oh!

Grubba27 commented Nov 30, 2023

Uh oh!

jamauro commented Nov 30, 2023

Uh oh!

Grubba27 commented Nov 30, 2023

Uh oh!

zodern Dec 4, 2023

Choose a reason for hiding this comment

Uh oh!

jamauro Dec 4, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rj-david commented Dec 5, 2023

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

zodern Nov 30, 2023 •

edited

Loading

jamauro Dec 4, 2023 •

edited

Loading