The new proposed changes look pretty simple to adopt 😀 As a suggestion it might be nice to show the new async code example along side the current fibers based code example.

// api/LinksCollectionAsync.js

// import { Mongo } from "meteor/mongo";  // current fibers
import { createAsyncCollection } from "meteor/mongo";

// export const LinksCollection = new Mongo.Collection("links");  // current fibers
export const LinksCollectionAsync = createAsyncCollection("links");

Any format that gives an easy side by side comparison makes it a lot easier to see the required format changes during a quick inspection 👍. We'd probably all be able to give a bit faster & better feedback that way!

For example from your example above, inside Meteor.startup, could we still use Meteor.call instead of the imported insertLink function?

Meteor.call('insertAsync' {
    title: "Do the Tutorial",
    url: "https://www.meteor.com/tutorials/react/creating-an-app"
});

await insertLink({
  title: "Do the Tutorial",
  url: "https://www.meteor.com/tutorials/react/creating-an-app"
});

From inspecting the code, it looks like this would be no problem, but it would help to see what is a random example vs a proposed change.

I'm really not a fan of having *Async on the name of all the methods. Already we have to use the await keyword which is kind of noisy. If the collection is declared async by virtue of why not simply allow the usage of the shorter names? I tend to use TypeScript in my projects so it would warn me if I was confused between a legacy sync collection and the new async ones...

I'm really not a fan of having *Async on the name of all the methods. Already we have to use the await keyword which is kind of noisy. If the collection is declared async by virtue of why not simply allow the usage of the shorter names? I tend to use TypeScript in my projects so it would warn me if I was confused between a legacy sync collection and the new async ones...

For sure shorter is better but the problem here is that we want to keep the old methods available yet, maybe forever in the client.

It would be impossible for the developers to know when they are using each type of collections and the code would be very hard to understand. And we can't just rely on type definitions to make sure people are not confused as not every project is going to use Typescript.

In the future we could migrate back to shorter names after all the Fibers dependencies are removed.

Another thing to discuss here, MongoDB Node.js driver already change many names and methods, like insertOne and insertMany instead of insert. So an option would be to already adopt the new names in this migration but maybe we could have some conflicts with the old MongoDB Node.js driver API in a few cases returning to the same problem. Also it would increase a lot the changes in this version as we would need to implement new code for some new options of MongoDB and I would like to try to keep this PR as small as possible to avoid breaking changes.

But again, open to ideas here.

For example from your example above, inside Meteor.startup, could we still use Meteor.call instead of the imported insertLink function?

Yes, you definitely don't need this intermediate function.

Any format that gives an easy side by side comparison makes it a lot easier

Good feedback, thanks!

I'm surprised that .findAsync() exists. .find() in the official MongoDB Node.js driver is a synchronous operation, and only the cursor methods are (and have to be) asynchronous. I know it'll require an additional await somewhere, but I can imagine a lot of people scratching their heads on this:

const old = X.find(query).fetch();
const new = await (await X.findAsync(query)).fetchAsync();

I wanted to check, what'd be needed to change it.

1. Instead of throwing an error here, do this instead.
2. To make it even better and more implicit, move this to here, in a .then(). This will make collection.pendingPromise a proper initializer. (If it's resolved, then the collection is initialized.)
3. If the point is that the initialization should be lazy (I guess it is), then the pendingPromise could be a getter (with the initialization in .then()).
   • Right now setCollectionInstance may get called multiple times, as there's no synchronization between different operations doing await this.pendingPromise, e.g., Array.from({ length: 1000 }, _ => collection.findAsync()).
4. The initialization of the driver into this getter as well. And that would require making MongoInternals.RemoteCollectionDriver#mongo a Promise of the connection instead.

Doing all of that would mean that the entire API relies on a single Promise.await in here or here. If that's not an option, the MongoInternals.RemoteCollectionDriver#mongo can be a MongoConnection | Promise<MongoConnection> instead of always a Promise, and every application could have something like this:

Meteor.startup(async () => {
  await Mongo.waitForConnection();
});

To make the transition easier, this could be a part of one of the official packages, or even a separate package.

On a side note, I'd vote against async constructors. Right now, there are three: MongoConnection, MongoInternals.RemoteCollectionDriver, OplogHandle. I wouldn't use a factory function (like createAsyncCollection) either, but either a static .create() method instead. Or move the initialization into a Promise that's stored on the instance and make people await it if needed. (With this one, the entire instance can be thenable.)

I'm surprised that .findAsync() exists. .find() in the official MongoDB Node.js driver is a synchronous operation, and only the cursor methods are (and have to be) asynchronous

I know you get the difference between a Meteor Collection and the usage of the driver directly but to be clear to all readers, in a regular usage of the driver usually there is a connect step first and a Meteor Collection does this internally that is why a second await is necessary.

In my second try I was not creating a findAsync method but it was hard to synchronize everything inside the collection to be proper initialized, including OplogHandle. I didn't test it but I believe if we postpone all the initialization it's probably that the OplogHandle will not be initialized and because of that changes directly to the database from different apps are not going to be consumed by OplogHandle until a first operation is performed in the database.

But again, I didn't go forward in this path in the second try and we could try it, maybe it could work but we need to be careful with side effects.

On a side note, I'd vote against async constructors

These constructors should not be used externally so we could change to anything later, personally I would avoid classes if I was rewriting this from scratch, as the idea here is to keep compatibility with the current version I tried to change as few code as possible.

I wouldn't use a factory function (like createAsyncCollection)

My main point with this function was to be able to lazy initialize the collections properly in the top level so we could still create our Collection instances in the top level of files and export them, but they are definitely other options.

As I said I did many iterations (you can see in my commits) and then in the end maybe we can indeed improve somethings because the flows are easier to visualize now 😉 .

I'll double check your specific proposed changes later and reply with more details and if they are possible or not and what would be the consequences, thank you for your detailed feedback. This is exactly what I'm looking for exposing these code changes here, thank you 😄 .

Great references and points in most of the comments. As a personal taste, I would try to stick with the naming convention proposed by the nodejs driver. I know insert is simpler than having to write insertOne or insertMany, but it's not such a big deal, and I think it would help newcomers to understand "easily" how to use the mongo driver functions in meteor.

Hi @radekmie, I'm back to this code and analyzing your ideas.

In this comment I'll focus on findAsync.

I'm surprised that .findAsync() exists. .find() in the official MongoDB Node.js driver is a synchronous operation, and only the cursor methods are (and have to be) asynchronous. I know it'll require an additional await somewhere, but I can imagine a lot of people scratching their heads on this:

We would need to connect to MongoDB in advance in order to create Cursors without requiring await before find.

The code below is not going to work in the current version of Meteor as startup callbacks are called in parallel.

Meteor.startup(async () => {
  await Mongo.waitForConnection();
});

This is a problem because we can use collection.find() inside another startup callback and maybe the connection won't be ready yet.

If we change this code to await each hook it would be a solution:

var callStartupHooks = Profile("Call Meteor.startup hooks", function () {
  // run the user startup hooks.  other calls to startup() during this can still
  // add hooks to the end.
  while (__meteor_bootstrap__.startupHooks.length) {
    var hook = __meteor_bootstrap__.startupHooks.shift();
    Profile.time(hook.stack || "(unknown)", () => Promise.await(hook()));
  }
  // Setting this to null tells Meteor.startup to call hooks immediately.
  __meteor_bootstrap__.startupHooks = null;
});

The only problem left is to make sure mongo package is always the first to add the callback, the declaration order in .meteor/packages file should take care of that, otherwise we could create a way to add priority startup callbacks.

Am I missing something or this change is really necessary for your approach to work?

If I'm correct we could also create a different startup phase and run then one by one so we don't change the behavior in the old startup. I believe we will need this sequencial way anyway to remove Fibers completely from startup code (I already did a PoC removing Fibers from startup and it was necessary to orchestrate some initial calls).

Something probably like Meteor.beforeStartupQueue(callback).

Hi @hexsprite, in this comment I'm going to focus on Async suffix.

Do you see any other alternatives after my comment here?

One goal here is to keep existent apps working in the same way and provide an easy way to migrate collections gradually.

I still don't see a better way than creating new methods to avoid confusion.

Hi @radekmie, in this comment I'll focus on async constructors.

On a side note, I'd vote against async constructors. Right now, there are three: MongoConnection, MongoInternals.RemoteCollectionDriver, OplogHandle. I wouldn't use a factory function (like createAsyncCollection) either, but either a static .create() method instead. Or move the initialization into a Promise that's stored on the instance and make people await it if needed. (With this one, the entire instance can be thenable.)

I was treating these constructors as internal APIs (as they are) so using async constructors is not a problem IMO. Do you think we should consider them external APIs? Because if not I don't think we should worry too much on ergonomics as developers are not going to use it.

What are the benefits that you see in using static methods instead of async constructors and factory functions?

The only one that I can think is to centralize the import in the class but do we want to keep this external API as a class? What is the point of using a class to use just a static method of it? The whole point of classes is to keep the state and using static methods is exactly the opposite, so I'm not sure if this is the best option here.

Side note: I'm not a fan of classes.

Update on findAsync.

I was able to use a sync find but using Promise.await (aka Fiber) in the top-level to initialize the connection during the code evaluation. Even using beforeStartupQueue was not enough as the collection is created in the top-level.

Not sure yet if this trade-off is worth it. It would require top-level await support in order to remove Fibers completely but maybe this is going to be necessary anyway. Comments?

I pushed the changes in the commit 9509c3a so we can see if all the tests are going to pass with this set up as well.

Of course, if we decide to move this change forward we could potentially simplify other parts of this PR as well.

Hi @filipenevola, thanks for the update!

The code below is not going to work in the current version of Meteor as startup callbacks are called in parallel.

Right, I forgot that Meteor.startup is a "fire and forget" one. Changing the behavior of it doesn't feel right, especially if we'll include Promise.await to the equation.

Something probably like Meteor.beforeStartupQueue(callback).

This would introduce the same problems down the road, as more and more code would rely on it.

I was treating these constructors as internal APIs (as they are) so using async constructors is not a problem IMO. Do you think we should consider them external APIs? Because if not I don't think we should worry too much on ergonomics as developers are not going to use it.

I agree that it is an internal API but it's been there for a long time and we're able to make it backward-compatible. And yes, no one should use it, but as I checked, I've used MongoConnection once in one of the projects to establish a second connection to the DB. (Yes, it can be done separately, without this code. I know, I know.)

What are the benefits that you see in using static methods instead of async constructors and factory functions?

I'd say it goes like this:

1. Constructors.
2. Static methods.
3. Factory functions.
4. async constructors.

If we could rely on standard constructors, that'd be ideal. First of all, due to the objective matter of the entire MongoDB API. If we'd say that everything but one call is a method, I see it as an inconsistency. But sure, factory functions are technically the same, but not bound to an object - I'm fine with that.

However, async constructors are really problematic. Most importantly, this code won't work:

class X { async constructor() {} }

// Uncaught SyntaxError: Class constructor may not be an async method

And because of that, TypeScript won't allow it either. (There's also this StackOverflow answer, but I link it only as a more comprehensive writeup rather than anything else.)

The only one that I can think is to centralize the import in the class but do we want to keep this external API as a class? What is the point of using a class to use just a static method of it? The whole point of classes is to keep the state and using static methods is exactly the opposite, so I'm not sure if this is the best option here.

I'd say that this is the main, not whole point of them. As we're in a (weirdly) typed language (I assume that this code will be in TypeScript sooner or later), making it a static method makes it possible for subclasses to extend. And again, I agree that composition is better than inheritance in general, but as the MongoDB API is already strongly object-oriented, I'd stick to it.

Side note: I'm not a fan of classes.

Oh, believe me - me neither...

I was able to use a sync find but using Promise.await (aka Fiber) in the top-level to initialize the connection during the code evaluation. Even using beforeStartupQueue was not enough as the collection is created in the top-level.

Not sure yet if this trade-off is worth it. It would require top-level await support in order to remove Fibers completely but maybe this is going to be necessary anyway. Comments?

I'd say it's a big step, as the entire MongoDB integration relies on a single await. Congratulations! If that's the case, Meteor could have a couple of "globally known promises to await before the server starts". Something like Meteor._storeMongoDBConnectionDriverSomewhereSoTheServerWillAwaitItOnStartup(driver) is more than fine by me.

Hi @radekmie, just to clarify a few things here:

I'd say it's a big step, as the entire MongoDB integration relies on a single await. Congratulations! If that's the case, Meteor could have a couple of "globally known promises to await before the server starts". Something like Meteor._storeMongoDBConnectionDriverSomewhereSoTheServerWillAwaitItOnStartup(driver) is more than fine by me.

Right now I'm starting the connection when mongo package is evaluated so you wouldn't need any call in your app code.

If you want to use an external the internal API would still accept an external driver.

The problem here is that I'm relying in a top-level Promise.await what will require top-level await support to migrate away from Fibers, but I think this is going to be necessary sooner or later.

Another point, we are depending on one await for the connection, but we still need one await for each MongoDB operation.

Something like this:

import { Meteor } from "meteor/meteor";
import { LinksCollectionAsync } from "../imports/api/LinksCollectionAsync";

Meteor.startup(async () => {
  if (await LinksCollectionAsync.find().countAsync()) {
    console.log(`No need to insert the first link`);
    console.log(`Links: `, await linksCursor.fetchAsync());
    return;
  }

  await LinksCollectionAsync.insertAsync({
    title: "Do the Tutorial",
    url: "https://www.meteor.com/tutorials/react/creating-an-app",
    createdAt: new Date(),
  });
});

Two test groups are breaking after this change, I'm going to check why soon.

We probably need to open a discussion about top-level await implementation in Meteor if we are going to proceed with this change.

Ah, right, we've slightly misunderstood @filipenevola 😅

Right now I'm starting the connection when mongo package is evaluated so you wouldn't need any call in your app code.
If you want to use an external the internal API would still accept an external driver.

That's what I meant and that's great.

The problem here is that I'm relying in a top-level Promise.await what will require top-level await support to migrate away from Fibers, but I think this is going to be necessary sooner or later.

1. I think it's fine, at least for the first step.
2. That's where my idea of "well-known promises" kicks in. Basically, I can imagine that somewhere deep in Meteor insides, there's a _promiseOfMongoDBDriver and it's awaited in the correct place (before the app starts, etc.). It couples the implementation slightly, but it's a nice workaround for getting rid of Promise.await before implementing top-level await.

Another point, we are depending on one await for the connection, but we still need one await for each MongoDB operation.

Yep, that's right. Plus, I'd add await before the LinksCollectionAsync.insertAsync to make it more clear.

We probably need to open a discussion about top-level await implementation in Meteor if we are going to proceed with this change.

Agree, but I think it's not that urgent if nobody raised it already (I think?).

The errors we are getting here (MongoServerSelectionError connect ECONNREFUSED) are somehow related to this top level connection, but I was not able yet to understand why they are happening.

I was also studying the top-level await specification to make sure our solution here is compatible with top-level await specification and it is compatible.

The examples even mention this case, where you want to connect to a resource in a top-level code to make sure all the consumer modules are ready to use the resource. So in terms of specifications we are safe to proceed but it's important to remember that Meteor doesn't implement top-level await yet.

The implementation here is using Promise.await that is powered by Fibers, but by my understanding this solution could be replaced by await in the top-level as soon as Meteor supports top-level await.

So I believe we can continue investing in this approach in order to avoid findAsync. I need to continue analyzing the error that is happening in the tests to understand why and how to fix it.

Can someone enlighten me on the use of wrapping MongoDB driver collection objects in a post-fiber universe?

EDIT: OH, right... The client.

I'm really not a fan of having *Async on the name of all the methods. Already we have to use the await keyword which is kind of noisy. If the collection is declared async by virtue of why not simply allow the usage of the shorter names? I tend to use TypeScript in my projects so it would warn me if I was confused between a legacy sync collection and the new async ones...

For sure shorter is better but the problem here is that we want to keep the old methods available yet, maybe forever in the client.

It would be impossible for the developers to know when they are using each type of collections and the code would be very hard to understand. And we can't just rely on type definitions to make sure people are not confused as not every project is going to use Typescript.

In the future we could migrate back to shorter names after all the Fibers dependencies are removed.

Another thing to discuss here, MongoDB Node.js driver already change many names and methods, like insertOne and insertMany instead of insert. So an option would be to already adopt the new names in this migration but maybe we could have some conflicts with the old MongoDB Node.js driver API in a few cases returning to the same problem. Also it would increase a lot the changes in this version as we would need to implement new code for some new options of MongoDB and I would like to try to keep this PR as small as possible to avoid breaking changes.

But again, open to ideas here.

Agree that it would be ideal to avoid *Async on all the mongo related operations.

Could there be a Meteor setting that you opt in to when starting up with meteor run? Then maybe one day it becomes the default and you can opt in to Fibers for some time before it’s completely dropped.

Agree that it would be ideal to avoid *Async on all the mongo related operations.

Could there be a Meteor setting that you opt in to when starting up with meteor run? Then maybe one day it becomes the default and you can opt in to Fibers for some time before it’s completely dropped.

Apologies if I'm misunderstanding this suggestion, but... To me, the benefit to having a suffixed version of Mongo methods is that it allows us to complete the migration over time by having both the Fiber and Fiber-free versions existing side-by-side in the same codebase.

If this behavior were controlled by a global setting that can only be either on or off for the whole application at any given time, then we'd be forced to migrate from Fibers -> async/await in one fell swoop, which can be much more challenging for a large, existing codebase. As someone maintaining a large, existing Meteor codebase, I very much appreciate the effort to make a gradual migration possible! I agree that it does make for some awkward naming in the meantime, but it seems like the lesser evil -- and as Filipe wrote above, perhaps the names could be swapped in a later version after the migration is complete.

Indeed, a gradual migration path is going to be important for my project as well. What I would do:

1. Add async versions of methods with *Async suffix. Maintain the two APIs in parallel for a while.
2. At some point, do a major version bump where you drop the non-async API, and make those names aliases for the *Async versions.
3. (optionally) at some further major version bump, drop the *Async-suffixed versions.

Ah, I hadn't considered that some might want to piecemeal migrate to their codebases. Still might be nice to have an option for those that want to migrate wholesale without having *Async everywhere.

My concern is that *Async never gets dropped and then the db operations are not in line with what you'd see in the mongo docs. Seems like a recipe for confusion and for folks to avoid adopting Meteor because it feels non-standard.

Ah, I hadn't considered that some might want to piecemeal migrate to their codebases. Still might be nice to have an option for those that want to migrate wholesale without having *Async everywhere.

Except its not just your code that will make these calls. Any meteor library you use will be expecting collection.findOne() to be fiber and not a promise. Making every library which does mongo calls handle this will likely be messy and prone to bugs.

While an Async suffix isnt the prettiest, it is the best way to not break the ecosystem or force lots of work upon users or library maintainers all at once, but instead let it be done over a couple of meteor releases

I would encourage everyone here to approach this huge change by assuming only one thing really:

It is the end of line of meteor in its first incarnation and all your code along with all packages will need to be substantially modified.

Every single call your application makes into meteor will need to be modified. And every single method in the call chain that has any of those calls needs to be changed too.

Yes, that’s right. There will be no free lunches and no automatic upgrade to async/await instead of fiber-synchronous calls. It will just be a lot of work and headaches.

You will save time by moving your code to Typescript and/or using some other tooling (e.g. SonarCloud) to detect where in your call chain you have now forgotten to convert the calling method to be async and add an await on invocations.

The Meteor team can facilitate the move by introducing runtime warnings for all fiber-based non-async methods along with the new *Async methods so that you can prepare for the big switch and that will require coordination.

I'm against changing to insertOne and insertMany vs the existing insert.

For our production system it would require a lot of changes as we're heavily relying on MongoDb transactions that we're accessing via raw collection.

Together with @fredmaiaarantes we've decided that I'll continue @filipenevola's work here. As of right now, I want to set up a project to see where the current implementation is and then add tests for the new async API.

I'm fully aware of the problems coming from using the async API on the client too (#11505 (comment)) and will put some thought into it as well.

🤞

A small update after a couple of hours.

Open topics:

1. I'm not sure about the createAsyncCollection function used instead of a constructor. It's not clear what the instance exactly is, and whether one can extend its prototype. I'd rather stick to new Mongo.Collection but with a { isAsync: true } option. Of course, it's possible to support both (that's the current state), but the question is which one should be recommended. Right now it's Mongo.Collection.create and I'm totally fine with it.
2. As the *Async methods throw an error on non-async collection, I was thinking about making non-*Async methods throw on async collections. The idea is to ensure that the async collections are already migrated to the new API, and the removal of sync API won't affect them.
   • This can be an option, e.g., throwOnSyncMethods.
   • It affects the cursors as well.
3. What kind of tests are needed to merge this PR? Rewriting all of the mongo_livedata_tests.js using the async API is going to require a lot of work.
4. Why was this test removed?
5. What is this change about?

If possible, I'd love to hear your opinion on these, @filipenevola. Of course, everyone else is welcome to comment as well.

What I've done so far:

1. Played around with the current implementation. Everything works well, and all tests are passing.
2. Added findOneAsync.
3. Replaced all but one Promise.await(MongoInternals.defaultRemoteCollectionDriver()).mongo instances with (await MongoInternals.defaultRemoteCollectionDriver()).mongo to reduce the number of Promise.await calls.
   • In my opinion supporting top-level await is the way to go.
4. Reverted whitespace-only changes to keep the PR as small as possible.
5. Added a sanity check for the presence of *Async methods on the collection and the cursor.

To do in the first place:

• Discuss the open topics.
• Merge in tests from the mongo-no-fiber-option-new-tests branch.
• Update the branch with the latest code.

You can find my code here: https://github.com/radekmie/meteor/tree/mongo-no-fiber-option

@fredmaiaarantes, @filipenevola, @anyone-interested-here If possible, please do comment on the topics above. In the meantime (this week) I'll work on more tests.

1. Right now it's Mongo.Collection.create and I'm totally fine with it.

I think that having a create method rather than using the constructor is good, as I think it likely that something async will want to be done during the creation. Even if it isnt needed today, having the create method return a Promise, and being the only way to create an async collection would be good future proofing

2. As the *Async methods throw an error on non-async collection, I was thinking about making non-*Async methods throw on async collections. The idea is to ensure that the async collections are already migrated to the new API, and the removal of sync API won't affect them.

I haven't checked the code, so I can only make assumptions for now.
It would be good if this property was named to make it clear that it only affects behaviour server-side, or if it does affect both server and client code, then it should be split into 2 properties so that they can be configured differently.
Perhaps this could have a warn mode, that logs a stack trace each time a non-async method is called, to help find stray calls without risking breaking your application

Personally, I am happy with *Async methods being added, while the method names are not as pretty it gives users time to gradually migrate code over rather than it needing to be done all at once before anything can be run.
At work we have been missing a Promise based mongo api, so made our own wrapper around Mongo.Collection. We found it crucial to have that to allow us to do mongo queries in parallel, as we could feed them into Promise.all() and await the result of that promise. That really helped us with performance when running a bunch of slow/large mongo queries at once

@Julusian

I think that having a create method rather than using the constructor is good, as I think it likely that something async will want to be done during the creation. Even if it isnt needed today, having the create method return a Promise, and being the only way to create an async collection would be good future proofing

The point is, it'll never do anything async; at least as long as the connection will already be there. I've already suggested that it can be done in #11605 (comment).

It would be good if this property was named to make it clear that it only affects behaviour server-side, or if it does affect both server and client code, then it should be split into 2 properties so that they can be configured differently.
Perhaps this could have a warn mode, that logs a stack trace each time a non-async method is called, to help find stray calls without risking breaking your application

It will affect both client and server. It is another topic that introducing async functions on the client is harder than on the backend...

@StorytellerCZ thanks for the input. Just keep in mind that it's not my branch - I haven't opened a PR yet. I'll do that later this week.

I see some mentions of relying on top-level awaits to be able to keep the existing pattern of creating collection instances at top level, but top-level awaits itself comes with a requirement to move from commonjs to ecmascript modules.

https://www.stefanjudis.com/today-i-learned/top-level-await-is-available-in-node-js-modules/

Without having done any work with ES modules I suspect that making that move will both break compatibility with older browsers and probably a lot of other breaking changes as well (hello reify). Can someone that knows more elaborate so we don’t just assume that it IS possible to move to ES modules and top-level await?

@fredmaiaarantes, @filipenevola, @anyone-interested-here If possible, please do comment on the topics above. In the meantime (this week) I'll work on more tests.

I will reread everything here today and share my thoughts. Thanks for pushing it!

Without having done any work with ES modules I suspect that making that move will both break compatibility with older browsers and probably a lot of other breaking changes as well (hello reify). Can someone that knows more elaborate so we don’t just assume that it IS possible to move to ES modules and top-level await?

We are planning to implement top-level await within our existing module system instead of switching the bundler's output to ES modules. Other bundlers have successfully done this, such as esbuild, so it is possible.

I just messaged @filipenevola to see if he can clarify these questions @radekmie.

Hi @radekmie sorry for the delay, my 2 cents in the open topics:

1. I'm not sure about the createAsyncCollection function used instead of a constructor. It's not clear what the instance exactly is, and whether one can extend its prototype. I'd rather stick to new Mongo.Collection but with a { isAsync: true } option. Of course, it's possible to support both (that's the current state), but the question is which one should be recommended. Right now it's Mongo.Collection.create and I'm totally fine with it.

Yes, I've changed to that after our discussions here.

2. As the *Async methods throw an error on non-async collection, I was thinking about making non-*Async methods throw on async collections. The idea is to ensure that the async collections are already migrated to the new API, and the removal of sync API won't affect them.
   * This can be an option, e.g., throwOnSyncMethods.
   * It affects the cursors as well.

I agree. It should be restrictive on mixing the two approaches in both sides (in the ideal World) but the idea was to make room for people to migrate slowly to the new async methods instead of migrating everything in a single batch in a single collection, I think this could be beneficial as the main challenge here is to provide a nice way for people to migrate their code.

But we are also allowing creating two collection instances for the same collection in MongoDB using different approaches so maybe we could prevent the mixed use case inside the same instance. It would be fine probably.

3. What kind of tests are needed to merge this PR? Rewriting all of the mongo_livedata_tests.js using the async API is going to require a lot of work.

We don't NEED anything else to make sure the old methods are still working. This work is done. What we SHOULD is add new tests to validate Async execution paths, but as they are going to be 100% new I'm not too afraid as we can get errors and problems in the beta phase and then add these tests to cover specific cases and avoid regressions, we did this in the MongoDB 5 oplog changes and in the end, the result was great.

4. Why was this test removed?

I explained in my commit message. TL;DR: it was testing a case that doesn't exist.

5. What is this change about?

Also explained in the commit. TL;DR: adding an option to test only one package tests on test-in-console/run.sh

I'm open to helping with other topics as well, feel free to send me an email ([email protected]) if I don't get back soon enough, I'm not tracking my GitHub notifications anymore.

As I opened #12028 just now, this one can be closed.