Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Emit 'drain' when protocol queue is emptied? #271

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
grncdr opened this issue Aug 10, 2012 · 7 comments
Closed

Emit 'drain' when protocol queue is emptied? #271

grncdr opened this issue Aug 10, 2012 · 7 comments

Comments

@grncdr
Copy link
Contributor

grncdr commented Aug 10, 2012

The pg module does this, and uses it for auto-releasing connections back into the pool. It also has pauseDrain and resumeDrain methods that allow you to prevent the event from being emitted (and thus keep a given connection out of the pool, which is useful for transactions).

I've written a monkey-patch to add this behaviour to mysql for my own needs, but if there's any interest I'd like to clean it up and get it merged so I don't have to do the evil monkey patching.

@felixge
Copy link
Collaborator

felixge commented Aug 10, 2012

Yes, I'd like to have a 'drain' event. I was thinking about it already when starting to work on a connection pool, but didn't have time to continue with it.

So if you want to add a 'drain' event for now, please send me a patch and I'll merge it.

I don't want to add 'pauseDrain' / 'resumeDrain' quite yet so - this use case would be better served by explicitly releasing connections from the pool. So my pool API would probably be like this:

Explicit alloc / free:

var connection = pool.alloc();
// do stuff
pool.free(connection);

Auto free:

pool.alloc({autoRelease: true});
// do stuff

What do you think?

@grncdr
Copy link
Contributor Author

grncdr commented Aug 10, 2012

Specifying the auto-release behaviour when checking out a connection seems reasonable, but I'd argue for {autoRelease: true} being the default, since the scenarios where you want to hang on to a specific connection are less common.

To be perfectly honest, my interest in having connection pooling be part of the mysql driver is pretty low, as I'm working on a very minimal layer over mysql, postgres, and sqlite3 that will provide a consistent interface for creating a connection pool, checking out connections, and making queries.

@felixge
Copy link
Collaborator

felixge commented Aug 15, 2012

Specifying the auto-release behaviour when checking out a connection seems reasonable, but I'd argue for {autoRelease: true} being the default, since the scenarios where you want to hang on to a specific connection are less common.

My concern is that this default will lead to a lot of subtle programming errors. Example:

a) User allocates a connection (set to auto release by default)
b) Performs a query
c) Inside the query callback performs an http request
d) In the http request callback performs another query
e) Oh shit, the connection was already auto-released after c)

To be perfectly honest, my interest in having connection pooling be part of the mysql driver is pretty low, as I'm working on a very minimal layer over mysql, postgres, and sqlite3 that will provide a consistent interface for creating a connection pool, checking out connections, and making queries.

I find connection pooling to be a surprisingly difficult feature. You're an ambitious man for trying to abstract it across drivers.

@grncdr
Copy link
Contributor Author

grncdr commented Aug 15, 2012

e) Oh shit, the connection was already auto-released after c)

Yeah, on second thought this is likely to be way more common than I imagined. Backing up a bit, I think the real issue is in trying to shoe-horn the pooling semantics into the connections themselves. What if your pool interface had a query method, and using that method directly on the pool is how a user indicates they want auto-release behaviour? e.g::

connection = pool.alloc() // I want a connection, I'll give it back when I'm done
pool.query(function (err, rows) {
  // the user never sees the connection used, so auto-releasing it is a non-issue.
})

That's quite similar to how gesundheit engines behave right now, and it's working pretty well for me there.

I find connection pooling to be a surprisingly difficult feature. You're an ambitious man for trying to abstract it across drivers.

Writing a DB agnostic library means I'm already committed to writing a bunch of shims for any driver I want to support, so factoring that into a separate module just seems like an obvious next step ;)

@felixge
Copy link
Collaborator

felixge commented Aug 22, 2012

Yes, I was planning on having a pool.query() method for exactly this use case. I'll soon have some time to work on node-mysql again, so please keep me posted on your pool.

@grncdr
Copy link
Contributor Author

grncdr commented Aug 22, 2012

I've started on it over here, but unfortunately there is very little documentation or tests so far, I'm just testing the feasibility of the idea.

@grncdr
Copy link
Contributor Author

grncdr commented Oct 31, 2012

FWIW, my main use case for this (querying with pooled connections and not having to manually return them) is now implemented by my any-db project. I need to start expanding the test suite now that I'm somewhat satisfied with the design, but for now it "works for me" and I no longer need the 'drain' event proposed here and in #272.

Looking at #306, I'm not sure that the use case as presented warrants increasing the API surface area of the driver, but I'll pick that discussion up over there.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Development

No branches or pull requests

3 participants