Thanks to visit codestin.com
Credit goes to github.com

Skip to content

bpo-9303: Migrate sqlite3 module to _v2 API to enhance performance #359

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 10 commits into from
Mar 3, 2017
Merged

bpo-9303: Migrate sqlite3 module to _v2 API to enhance performance #359

merged 10 commits into from
Mar 3, 2017

Conversation

palaviv
Copy link
Contributor

@palaviv palaviv commented Feb 28, 2017

Use the _v2 API when possible.

berkerpeksag
berkerpeksag reviewed Feb 28, 2017
View reviewed changes
Copy link
Member

@berkerpeksag berkerpeksag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can also avoid calling sqlite3_reset in _pysqlite_seterror if the v2 API is available.

@berkerpeksag
Copy link
Member

I haven't checked yet, but I also think that the check for SQLITE_SCHEMA can also be eliminated if the v2 API is available:

cpython/Modules/_sqlite/cursor.c

Line 553 in c92e604

if (rc == SQLITE_SCHEMA) {

berkerpeksag
berkerpeksag reviewed Feb 28, 2017
View reviewed changes
Modules/_sqlite/connection.c Outdated
@@ -118,8 +118,13 @@ int pysqlite_connection_init(pysqlite_Connection* self, PyObject* args, PyObject
return -1;
}
Py_BEGIN_ALLOW_THREADS
#ifdef HAVE_SQLITE3_OPEN_V2
rc = sqlite3_open_v2(database, &self->db,
SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE, NULL);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style nitpick: FOO|BAR is more common than FOO | BAR in sqlite3 code base.

berkerpeksag
berkerpeksag reviewed Feb 28, 2017
View reviewed changes
Modules/_sqlite/connection.c Outdated
@@ -118,8 +118,13 @@ int pysqlite_connection_init(pysqlite_Connection* self, PyObject* args, PyObject
return -1;
}
Py_BEGIN_ALLOW_THREADS
#ifdef HAVE_SQLITE3_OPEN_V2
rc = sqlite3_open_v2(database, &self->db,
Copy link
Member

@berkerpeksag berkerpeksag Feb 28, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that there is an opportunity to simplify this whole SQLITE_OPEN_URI and HAVE_SQLITE3_OPEN_V2 dance.

(The use of Py_BEGIN_ALLOW_THREADS is a bit weird in line 111 and line 120.)

@berkerpeksag berkerpeksag mentioned this pull request Mar 1, 2017
Closed
@palaviv
Copy link
Contributor Author

palaviv commented Mar 1, 2017 •
edited
Loading

I think we can also avoid calling sqlite3_reset in _pysqlite_seterror if the v2 API is available.

done

I haven't checked yet, but I also think that the check for SQLITE_SCHEMA can also be eliminated if the v2 API is available:

cpython/Modules/_sqlite/cursor.c

Line 553 in c92e604

if (rc == SQLITE_SCHEMA) {

I don't think we should do this because we can get SQLITE_SCHEMA if SQLITE_MAX_SCHEMA_RETRY tries to recompile will pass.

berkerpeksag
berkerpeksag reviewed Mar 2, 2017
View reviewed changes
Copy link
Member

@berkerpeksag berkerpeksag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks pretty good to me. We just need an entry in Misc/NEWS (please add "Patch by Aviv Palivoda." too)

Modules/_sqlite/util.c Outdated
if (st != NULL) {
(void)sqlite3_reset(st);
}
#endif

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: Please remove extra empty line.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

Modules/_sqlite/connection.c Outdated
@@ -118,6 +118,8 @@ int pysqlite_connection_init(pysqlite_Connection* self, PyObject* args, PyObject
return -1;
}
Py_BEGIN_ALLOW_THREADS
/* No need to use sqlite3_open_v2 as sqlite3_open(filename, db) is the
same as sqlite3_open_v2(filename, db, SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE, NULL)*/
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nitpick: FOO|BAR syntax instead of FOO | BAR.

Also, finish the line with a full stop and space:

/* [...] NULL). */

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

@berkerpeksag
Copy link
Member

I don't think we should do this because we can get SQLITE_SCHEMA if SQLITE_MAX_SCHEMA_RETRY tries to recompile will pass.

Right, we can still get SQLITE_SCHEMA from sqlite3_step() (in legacy API we need to call sqlite3_reset() to get it), but in this case rc is the return value of sqlite3_reset(). The latter call is not needed when we have the new API since sqlite3_step() calls sqlite3_reset() for us.

@palaviv
Copy link
Contributor Author

palaviv commented Mar 2, 2017

I have removed the reset but I left the recompile because we should recompile before calling sqlite3_step again.

berkerpeksag
berkerpeksag reviewed Mar 2, 2017
View reviewed changes
Modules/_sqlite/util.c Outdated
@@ -49,10 +49,13 @@ int _pysqlite_seterror(sqlite3* db, sqlite3_stmt* st)
{
int errorcode;

/* SQLite often doesn't report anything useful, unless you reset the statement first */
#if SQLITE_VERSION_NUMBER >= 3003009
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't >= be < here?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch!

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

serhiy-storchaka
serhiy-storchaka requested changes Mar 3, 2017
View reviewed changes
Copy link
Member

@serhiy-storchaka serhiy-storchaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix an error found by @berkerpeksag.

berkerpeksag
berkerpeksag approved these changes Mar 3, 2017
View reviewed changes
Copy link
Member

@berkerpeksag berkerpeksag left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, Aviv! I will wait for Serhiy's approval and merge this.

@berkerpeksag berkerpeksag merged commit 86a6705 into python:master Mar 3, 2017
@palaviv palaviv mentioned this pull request Mar 4, 2017
Closed
@coleifer coleifer mentioned this pull request Jul 22, 2019
Closed
jaraco pushed a commit that referenced this pull request Dec 2, 2022
@Mariatta
Use Python 3.7.8 (#359)
5f46257 
Closes #358
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@berkerpeksag berkerpeksag berkerpeksag approved these changes

@serhiy-storchaka serhiy-storchaka serhiy-storchaka approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
5 participants
@palaviv @berkerpeksag @serhiy-storchaka @brettcannon @the-knights-who-say-ni