Thanks to visit codestin.com
Credit goes to github.com

Skip to content

bpo-27645: Supporting native backup facility of SQLite #4238

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 52 commits into from
Mar 10, 2018
Merged

bpo-27645: Supporting native backup facility of SQLite #4238

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
52 commits
Select commit Hold shift + click to select a range
0c9a4e5
Expose the SQLite Online Backup API as Connection.backup()
lelit Jul 28, 2016
624849a
Preliminary documentation for sqlite3.Connection.backup()
lelit Jul 28, 2016
b17e572
Reduce code nesting depth by using a finally label, as other function…
lelit Mar 1, 2017
3a64654
Wrap SQLite calls between BEGIN_ALLOW_THREADS and END_ALLOW_THREADS
lelit Mar 1, 2017
8220ed6
Make pages and progress keyword-only args in sqlite3.Connection.backup()
lelit Mar 1, 2017
f7c6fc5
Remove reference to SQLite docs in Connection.backup() documentation
lelit Mar 1, 2017
df607df
Use f-string in the sqlite3.Connection.backup() example
lelit Mar 1, 2017
169369b
Explicitly mark sqlite3.Connection.backup() as added in v3.7
lelit Mar 1, 2017
7ee5341
Reduce chances of cluttering future “git blame”
lelit Mar 1, 2017
f88cd12
Parametrize the name of the database involved in the backup
lelit Mar 3, 2017
7ee018c
Test propagation of exception raised in backup's progress callback
lelit Mar 3, 2017
21bfc82
Use a better name for variable
lelit Mar 3, 2017
9574b91
Do not delay next iteration if the result was OK
lelit Mar 3, 2017
09407c8
Omit the backup method when underlying SQLite library is older than 3…
lelit Mar 3, 2017
f99e65e
Assert that the non-mandatory arguments are keyword-only
lelit Mar 3, 2017
aaa1508
Pass also the current status of the ongoing backup to the progress ca…
lelit Mar 4, 2017
13de3a1
Slightly different way handling backup step's error state
lelit Mar 4, 2017
9b2f47a
When an error occurs while the backup is going on, remove the target …
lelit Mar 4, 2017
ce55873
Add NEWS entry for issue 27645
lelit Oct 5, 2017
5a08168
New argument "sleep" to specify a different number of milliseconds
lelit Oct 24, 2017
960303f
Allow to copy the database to either an external file or to another C…
lelit Oct 24, 2017
4bd0b3e
Check that target backup DB is not in transaction
lelit Oct 24, 2017
7dc53f0
Use DeleteFileW() under MS Windows to delete incomplete backup
lelit Dec 5, 2017
a333639
Second attempt at properly calling DeleteFileW()
lelit Dec 5, 2017
4ea5ca4
Setup exception from sqlite error condition
lelit Dec 5, 2017
48fd04b
Close the backup connection in case of error
lelit Dec 5, 2017
3256d52
Attempt to chain the exception raised by incomplete backup deletion
lelit Dec 5, 2017
a04a86e
Fix double call to PyMem_Free()
lelit Dec 5, 2017
a4334a6
Remove leftover comment
lelit Dec 5, 2017
69d8996
Drop const qualifier to avoid compiler warning
lelit Dec 5, 2017
ac5c64b
Avoid using NamedTemporaryFile in test cases
lelit Dec 5, 2017
09671e4
Make the sleep parameter a double, accepting a value in seconds
lelit Dec 5, 2017
b5260e0
Replace gotos with explicit return, given that no cleanup is needed
lelit Dec 5, 2017
37f316f
Rename test methods using snake_style instead of CamelCase
lelit Dec 5, 2017
b7fcf9e
Use one single pointers declaration style in the new function
lelit Dec 6, 2017
6566166
Fix glitch in example code
lelit Dec 8, 2017
e6f8950
Rectify explanation of the pages parameter
lelit Dec 8, 2017
5b06a74
Add missing ending dot
lelit Dec 8, 2017
5c20723
Simplify code
lelit Dec 8, 2017
2155feb
Add missing const qualifier
lelit Dec 8, 2017
d21c9cb
Fix leak by DECREFing the result of progress callback call
lelit Dec 8, 2017
8e096c2
Restore the suite() function, used by outer Lib/test/test_sqlite.py
lelit Dec 8, 2017
a2f15bc
Simplify backup() implementation
lelit Dec 12, 2017
66df8b3
Remove now useless includes
lelit Dec 12, 2017
814ef4e
Recognize common error codes for old SQLite3 missing sqlite3_errstr()
lelit Dec 12, 2017
8fe5c30
Move argument type check to PyArg_ParseTupleAndKeywords()
lelit Dec 12, 2017
acc2f37
Use better names for local variables
lelit Dec 12, 2017
32285b1
Merge remote-tracking branch 'upstream/master' into pr/4238
berkerpeksag Mar 10, 2018
7943561
Cosmetic fixes
berkerpeksag Mar 10, 2018
824d5f8
Fix segfault in test_bad_target_closed_connection
berkerpeksag Mar 10, 2018
6fd854a
Relax an assert
berkerpeksag Mar 10, 2018
57f49c6
This will be included in 3.7
berkerpeksag Mar 10, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Parametrize the name of the database involved in the backup 
Suggested by Aviv Palivoda.
  • Loading branch information
@lelit
lelit committed Oct 5, 2017
commit f88cd12317d97dde1d7f644f89f8ad2aeceda23e
7 changes: 6 additions & 1 deletion Doc/library/sqlite3.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -521,7 +521,7 @@ Connection Objects
f.write('%s\n' % line)


.. method:: backup(filename, *, pages=0, progress=None)
.. method:: backup(filename, *, pages=0, progress=None, name="main")

This method makes a backup of a SQLite database into the mandatory argument
*filename*, even while it's being accessed by other clients, or concurrently by
Expand All @@ -535,6 +535,11 @@ Connection Objects
will be executed at each iteration with two integer arguments, respectively the
*remaining* number of pages still to be copied and the *total* number of pages.

The *name* argument specifies the database name that will be copied: it must be
a string containing either ``"main"``, the default, to indicate the main
database, ``"temp"`` to indicate the temporary database or the name specified
after the ``AS`` keyword in an ``ATTACH`` statement for an attached database.

Example::

# Copy an existing database into another file
Expand Down
14 changes: 14 additions & 0 deletions Lib/sqlite3/test/backup.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -93,6 +93,20 @@ def progress(remaining, total):
self.assertEqual(journal[1], 1)
self.assertEqual(journal[2], 0)

def CheckDatabaseSourceName(self):
with NamedTemporaryFile(suffix='.sqlite') as bckfn:
self.cx.backup(bckfn.name, name='main')
self.cx.backup(bckfn.name, name='temp')
with self.assertRaises(sqlite.OperationalError):
self.cx.backup(bckfn.name, name='non-existing')
self.cx.execute("ATTACH DATABASE ':memory:' AS attached_db")
self.cx.execute('CREATE TABLE attached_db.foo (key INTEGER)')
self.cx.executemany('INSERT INTO attached_db.foo (key) VALUES (?)', [(3,), (4,)])
self.cx.commit()
with NamedTemporaryFile(suffix='.sqlite') as bckfn:
self.cx.backup(bckfn.name, name='attached_db')
self.testBackup(bckfn.name)

def suite():
return unittest.TestSuite(unittest.makeSuite(BackupTests, "Check"))

Expand Down
9 changes: 5 additions & 4 deletions Modules/_sqlite/connection.c
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1448,14 +1448,15 @@ pysqlite_connection_backup(pysqlite_Connection* self, PyObject* args, PyObject*
char* filename;
int pages = -1;
PyObject* progress = Py_None;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PEP 7, the * come before the var name, not after the type. It's PyObject *progress. Same for char *name.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've used the same style as the whole unit, do you think I should follow PEP7 in only new function?
I personally prefer that style, but seemed better to follow the code around... the same reasoning I had with the backup.py (but that's completely new...)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you sure? :)

+    sqlite3 *bckconn;
+    sqlite3_backup *bckhandle;

I guess muscle memory is hard to beat...

Your remark is valid. I don't care that much about PEP 7, as long as you stick to one one style. It's really up to you, either obey PEP 7 for new code or stick to the existing style.

char* name = "main";
PyObject* retval = NULL;
int rc;
sqlite3 *bckconn;
sqlite3_backup *bckhandle;
static char *keywords[] = {"filename", "pages", "progress", NULL};
static char *keywords[] = {"filename", "pages", "progress", "name", NULL};

if (!PyArg_ParseTupleAndKeywords(args, kwds, "s|$iO:backup", keywords,
&filename, &pages, &progress)) {
if (!PyArg_ParseTupleAndKeywords(args, kwds, "s|$iOs:backup", keywords,
&filename, &pages, &progress, &name)) {
goto finally;
}

Expand All @@ -1477,7 +1478,7 @@ pysqlite_connection_backup(pysqlite_Connection* self, PyObject* args, PyObject*
}

Py_BEGIN_ALLOW_THREADS
bckhandle = sqlite3_backup_init(bckconn, "main", self->db, "main");
bckhandle = sqlite3_backup_init(bckconn, "main", self->db, name);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it worth to make the destination table name configurable?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The second parameter, "main", is the destination database name (see here): in principle yes, it could be exposed as a parameter, but while the source name (ie the fourth parameter) is useful because you may desire to take a snapshot of an attached database, using a target different from "main" seems seldom useful...

Py_END_ALLOW_THREADS

if (bckhandle) {
Expand Down