[Meteor 3] Sequential packages loading order #13095
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
✅ Deploy Preview for v3-meteor-api-docs canceled.
|
|
There's one test failing and I'm not sure if it's "failing correctly". The ordering is, of course, changed, but the final one looks OK to me. @zodern could you check it maybe? diff --git a/tools/tests/top-level-await.js b/tools/tests/top-level-await.js
index b2deb6ca11..26d2fa0d4c 100644
--- a/tools/tests/top-level-await.js
+++ b/tools/tests/top-level-await.js
@@ -23,18 +23,18 @@ selftest.define("xxxx top level await - order", async function (options) {
const lines = [
'package sync',
- 'package 1 - b before',
- 'package 2 - b',
- 'package 2 - a before',
'package sync - later',
+ 'package 1 - b before',
'package 1 - b after',
- 'package 2 - b later',
- 'package 2 - a after',
'package 1 - b later',
'package 1 - a before',
- 'package 2 - a later',
'package 1 - a after',
'package 1 - a later',
+ 'package 2 - b',
+ 'package 2 - a before',
+ 'package 2 - b later',
+ 'package 2 - a after',
+ 'package 2 - a later',
'app a.js - before',
'app b.js - before',
'app a.js - after', |
|
Well, considering that we now have a sequential load, I believe this new other is correct. Let's see what @zodern thinks. |
denihs
previously approved these changes
Apr 16, 2024
|
I'm a little surprised this works without removing the deps tracking, though it was designed to load in the original order when all packages are sync. A couple things:
Another thing is that the deps code isn't adding anything here. You should be able to move the queueing into |
|
I noticed I misunderstood the implementation. It seems it does rely on the deps code still, and only ensures sequential order for eager modules. It seems likely there are edge cases when mixing packages that are async with packages that don't have a dependency on In the logs, it needs an additional message for when a package finishes loading to check that each package doesn't start loading until the previous one finishes. I would also suggest making package 1 in the test use top level await to wait 100ms before logging |
|
The after logs don't show the packages loading in the correct order: The packages that have a dependency on I was expecting the implementation to look something like: var pending = [];
var processingQueue = false;
function queue(name, runImage) {
pending.push({name: name, runImage: runImage});
processNext();
}
function processNext() {
if (processingQueue) {
return;
}
var config = pending.shift();
if (!config) {
return;
}
processingQueue = true;
loadImage(config.name, config.runImage, function () {
processingQueue = false;
processNext();
});
}It has a simple queue and processes one item at a time. Since |
|
I've tried to implement such a queue @zodern, but no matter what I try, it fails with the following error while loading the Code// This file needs to work in old web browsers and old node versions
// It should not use js features newer than EcmaScript 5.
//
// Handles loading linked code for packages and apps
// Ensures packages and eager requires run in the correct order
// when there is code that uses top level await
var hasOwn = Object.prototype.hasOwnProperty;
var pending = [];
var isProcessing = false;
function queue(name, deps, runImage) {
console.log(`queue(${name})`);
pending.push({name: name, runImage: runImage});
processNext();
}
function processNext() {
if (isProcessing) {
return;
}
var next = pending.shift();
if (!next) {
return;
}
console.log(`load(${next.name})`);
isProcessing = true;
var config = next.runImage();
runEagerModules(config, function (mainModuleExports) {
// Get the exports after the eager code has been run
var exports = config.export ? config.export() : {};
if (config.mainModulePath) {
Package._define(next.name, mainModuleExports, exports);
} else {
Package._define(next.name, exports);
}
isProcessing = false;
processNext();
});
}
function runEagerModules(config, callback) {
if (!config.eagerModulePaths) {
return callback();
}
var index = -1;
var mainExports = {};
var mainModuleAsync = false;
function evaluateNextModule() {
index += 1;
if (index === config.eagerModulePaths.length) {
if (mainModuleAsync) {
// Now that the package has loaded, mark the main module as sync
// This allows other packages and the app to `require` the package
// and for it to work the same, regardless of if it uses TLA or not
// XXX: this is a temporary hack until we find a better way to do this
const reify = config.require('/node_modules/meteor/modules/node_modules/@meteorjs/reify/lib/runtime');
reify._requireAsSync(config.mainModulePath);
}
return callback(mainExports);
}
var path = config.eagerModulePaths[index];
var exports = config.require(path);
if (checkAsyncModule(exports)) {
if (path === config.mainModulePath) {
mainModuleAsync = true;
}
// Is an async module
exports.then(function (exports) {
if (path === config.mainModulePath) {
mainExports = exports;
}
evaluateNextModule();
})
// This also handles errors in modules and packages loaded sync
// afterwards since they are run within the `.then`.
.catch(function (error) {
if (
typeof process === 'object' &&
typeof process.nextTick === 'function'
) {
// Is node.js
process.nextTick(function () {
throw error;
});
} else {
// TODO: is there a faster way to throw the error?
setTimeout(function () {
throw error;
}, 0);
}
});
} else {
if (path === config.mainModulePath) {
mainExports = exports;
}
evaluateNextModule();
}
}
evaluateNextModule();
}
function checkAsyncModule (exports) {
var potentiallyAsync = exports && typeof exports === 'object' &&
hasOwn.call(exports, '__reifyAsyncModule');
if (!potentiallyAsync) {
return;
}
return typeof exports.then === 'function';
}
// For this to be accurate, all linked files must be queued before calling this
// If all are loaded, returns null. Otherwise, returns a promise
function waitUntilAllLoaded() {
if (pending.length === 0) {
// All packages are loaded
// If there were no async packages, then there might not be a promise
// polyfill loaded either, so we don't create a promise to return
return null;
}
return new Promise(function (resolve) {
queue('', [], function () {
resolve();
return {};
});
});
}
// Since the package.js doesn't export load or waitUntilReady
// these will never be globals in packages or apps that depend on core-runtime
Package['core-runtime'] = {
queue: queue,
waitUntilAllLoaded: waitUntilAllLoaded
};I'll try to make my |
denihs
approved these changes
Apr 18, 2024
zodern
reviewed
Apr 19, 2024
| var pendingNames = Object.keys(pending); | ||
|
|
||
| if (pendingNames.length === 0) { | ||
| if (pending.length === 0) { |
There was a problem hiding this comment.
This also needs to check if isProcessing is true. Otherwise, it won't wait when the only js image that is async is the last one.
Besides this, it looks good.
There was a problem hiding this comment.
@denihs can you add this check or should I file another PR?
There was a problem hiding this comment.
@radekmie please open a new PR. Then, next week, I can merge it and do another release with this.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Updated description
According to the @zodern's comments, I created a synchronous queue instead. It matches the expected loading order now (the tests are updated).
Previous description
This is a different attempt at solving https://forums.meteor.com/t/order-of-loading-packages/61112 (see #13074 for the first one). The idea is to keep the dependency loading order as it was, but maintain a queue to load at most one package in order.
How I tested it:
Logs before
Logs after
As you can see, the last few lines show that the packages were loaded in the expected order, i.e.,
one,two,three.