diff --git a/.github/ISSUE_TEMPLATE/bug_report.md b/.github/ISSUE_TEMPLATE/bug_report.md index e2f14929..9e0a7bb3 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.md +++ b/.github/ISSUE_TEMPLATE/bug_report.md @@ -3,7 +3,6 @@ name: Bug report about: Create a report to help us improve title: '' labels: bug, not qualified -assignees: gabor-boros --- @@ -23,7 +22,6 @@ If applicable, add screenshots to help explain your problem. **System info** - OS: [e.g. macOS Mojave 10.14.3] - RethinkDB Version: [e.g. 2.4.0] - - Python client version: [e.g. 2.4.1 **Additional context** Add any other context about the problem here. diff --git a/.github/ISSUE_TEMPLATE/feature_request.md b/.github/ISSUE_TEMPLATE/feature_request.md index 817faa03..0512b489 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.md +++ b/.github/ISSUE_TEMPLATE/feature_request.md @@ -3,7 +3,6 @@ name: Feature request about: Suggest an idea for this project title: '' labels: enhancement, not qualified, question -assignees: gabor-boros --- diff --git a/.github/PULL_REQUEST_TEMPLATE.md b/.github/PULL_REQUEST_TEMPLATE.md index 18768cbf..fc4a2e02 100644 --- a/.github/PULL_REQUEST_TEMPLATE.md +++ b/.github/PULL_REQUEST_TEMPLATE.md @@ -8,8 +8,7 @@ A clear and concise description of what did you changed and why. If applicable, add code examples to help explain your changes. **Checklist** -- [ ] Unit tests created/modified -- [ ] Integration tests created/modified +- [ ] I have read and agreed to the [RethinkDB Contributor License Agreement](http://rethinkdb.com/community/cla/) **References** Anything else related to the change e.g. documentations, RFCs, etc. diff --git a/CODE_OF_CONDUCT.md b/CODE_OF_CONDUCT.md new file mode 100644 index 00000000..9c628288 --- /dev/null +++ b/CODE_OF_CONDUCT.md @@ -0,0 +1,71 @@ +# Contributor Covenant Code of Conduct + +## Our Pledge + +In the interest of fostering an open and welcoming environment, we as +contributors and maintainers pledge to making participation in our project and +our community a harassment-free experience for everyone, regardless of age, body +size, disability, ethnicity, sex characteristics, gender identity and expression, +level of experience, education, socio-economic status, nationality, personal +appearance, race, religion, or sexual identity and orientation. + +## Our Standards + +Examples of behaviour that contributes to creating a positive environment +include: + +* Using welcoming and inclusive language +* Being respectful of differing viewpoints and experiences +* Gracefully accepting constructive criticism +* Focusing on what is best for the community +* Showing empathy towards other community members + +Examples of unacceptable behaviour by participants include: + +* The use of sexualized language or imagery and unwelcome sexual attention or advances +* Trolling, insulting/derogatory comments, and personal or political attacks +* Public or private harassment +* Publishing others' private information, such as a physical or electronic address, without explicit permission +* Other conduct which could reasonably be considered inappropriate in a professional setting + +## Our Responsibilities + +Project maintainers are responsible for clarifying the standards of acceptable +behaviour and are expected to take appropriate and fair corrective action in +response to any instances of unacceptable behaviour. + +Project maintainers have the right and responsibility to remove, edit, or +reject comments, commits, code, wiki edits, issues, and other contributions +that are not aligned to this Code of Conduct, or to ban temporarily or +permanently any contributor for other behaviors that they deem inappropriate, +threatening, offensive, or harmful. + +## Scope + +This Code of Conduct applies both within project spaces and in public spaces +when an individual is representing the project or its community. Examples of +representing a project or community include using an official project e-mail +address, posting via an official social media account, or acting as an appointed +representative at an online or offline event. Representation of a project may be +further defined and clarified by project maintainers. + +## Enforcement + +Instances of abusive, harassing, or otherwise unacceptable behaviour may be +reported by contacting the project team at open@rethinkdb.com. All +complaints will be reviewed and investigated and will result in a response that +is deemed necessary and appropriate to the circumstances. The project team is +obligated to maintain confidentiality with regard to the reporter of an incident. +Further details of specific enforcement policies may be posted separately. + +Project maintainers who do not follow or enforce the Code of Conduct in good +faith may face temporary or permanent repercussions as determined by other +members of the project's leadership. + +## Attribution + +This Code of Conduct is adapted from the Contributor Covenant, version 1.4, +available at https://www.contributor-covenant.org/version/1/4/code-of-conduct.html + +For answers to common questions about this code of conduct, see +https://www.contributor-covenant.org/faq diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 00000000..e8c48beb --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,45 @@ +# Contributing + +Contributions are welcome, and they are greatly appreciated! Every little bit helps! You can contribute in many ways, not limited to this document. + +## Types of Contributions + +### Report Bugs + +First of all, please check that the bug is not reported yet. If that's already reported then upvote the existing bug instead of opening a new bug report. + +Report bugs at https://github.com/rethinkdb/rethinkdb-python/issues. If you are reporting a bug, please include: + +- Your operating system name and version. +- Any details about your local setup that might be helpful in troubleshooting. +- Detailed steps to reproduce the bug. + +### Fix Bugs + +Look through the GitHub issues for bugs. Anything tagged with "bug", "good first issue" and "help wanted" is open to whoever wants to implement it. + +### Implement Features + +Look through the GitHub issues for features. Anything tagged with "enhancement", "good first issue" and "help wanted" is open to whoever wants to implement it. In case you added a new Rule or Precondition, do not forget to add them to the docs as well. + +### Write Documentation + +RethinkDB could always use more documentation, whether as part of the official docs, in docstrings, or even on the web in blog posts, articles, and such. To extend the documentation on the website, visit the [www](https://github.com/rethinkdb/www) repo. For extending the docs, you can check the [docs](https://github.com/rethinkdb/docs) repo. + +### Submit A Feature + +First of all, please check that the feature request is not reported yet. If that's already reported then upvote the existing request instead of opening a new one. + +If you are proposing a feature: + +- Check if there is an opened feature request for the same idea. +- Explain in detail how it would work. +- Keep the scope as narrow as possible, to make it easier to implement. +- Remember that this is an open-source project, and that contributions are welcome :) + +## Pull Request Guidelines + +Before you submit a pull request, check that it meets these guidelines: + +1. The pull request should include tests (if applicable) +2. If the pull request adds functionality, the docs should be updated too. diff --git a/LICENSE b/LICENSE index a3a6a42f..261eeb9e 100644 --- a/LICENSE +++ b/LICENSE @@ -1,13 +1,201 @@ -Copyright 2018 RethinkDB + Apache License + Version 2.0, January 2004 + http://www.apache.org/licenses/ -Licensed under the Apache License, Version 2.0 (the "License"); -you may not use this file except in compliance with the License. -You may obtain a copy of the License at + TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION - http://www.apache.org/licenses/LICENSE-2.0 + 1. Definitions. -Unless required by applicable law or agreed to in writing, software -distributed under the License is distributed on an "AS IS" BASIS, -WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. -See the License for the specific language governing permissions and -limitations under the License. + "License" shall mean the terms and conditions for use, reproduction, + and distribution as defined by Sections 1 through 9 of this document. + + "Licensor" shall mean the copyright owner or entity authorized by + the copyright owner that is granting the License. + + "Legal Entity" shall mean the union of the acting entity and all + other entities that control, are controlled by, or are under common + control with that entity. For the purposes of this definition, + "control" means (i) the power, direct or indirect, to cause the + direction or management of such entity, whether by contract or + otherwise, or (ii) ownership of fifty percent (50%) or more of the + outstanding shares, or (iii) beneficial ownership of such entity. + + "You" (or "Your") shall mean an individual or Legal Entity + exercising permissions granted by this License. + + "Source" form shall mean the preferred form for making modifications, + including but not limited to software source code, documentation + source, and configuration files. + + "Object" form shall mean any form resulting from mechanical + transformation or translation of a Source form, including but + not limited to compiled object code, generated documentation, + and conversions to other media types. + + "Work" shall mean the work of authorship, whether in Source or + Object form, made available under the License, as indicated by a + copyright notice that is included in or attached to the work + (an example is provided in the Appendix below). + + "Derivative Works" shall mean any work, whether in Source or Object + form, that is based on (or derived from) the Work and for which the + editorial revisions, annotations, elaborations, or other modifications + represent, as a whole, an original work of authorship. For the purposes + of this License, Derivative Works shall not include works that remain + separable from, or merely link (or bind by name) to the interfaces of, + the Work and Derivative Works thereof. + + "Contribution" shall mean any work of authorship, including + the original version of the Work and any modifications or additions + to that Work or Derivative Works thereof, that is intentionally + submitted to Licensor for inclusion in the Work by the copyright owner + or by an individual or Legal Entity authorized to submit on behalf of + the copyright owner. For the purposes of this definition, "submitted" + means any form of electronic, verbal, or written communication sent + to the Licensor or its representatives, including but not limited to + communication on electronic mailing lists, source code control systems, + and issue tracking systems that are managed by, or on behalf of, the + Licensor for the purpose of discussing and improving the Work, but + excluding communication that is conspicuously marked or otherwise + designated in writing by the copyright owner as "Not a Contribution." + + "Contributor" shall mean Licensor and any individual or Legal Entity + on behalf of whom a Contribution has been received by Licensor and + subsequently incorporated within the Work. + + 2. Grant of Copyright License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + copyright license to reproduce, prepare Derivative Works of, + publicly display, publicly perform, sublicense, and distribute the + Work and such Derivative Works in Source or Object form. + + 3. Grant of Patent License. Subject to the terms and conditions of + this License, each Contributor hereby grants to You a perpetual, + worldwide, non-exclusive, no-charge, royalty-free, irrevocable + (except as stated in this section) patent license to make, have made, + use, offer to sell, sell, import, and otherwise transfer the Work, + where such license applies only to those patent claims licensable + by such Contributor that are necessarily infringed by their + Contribution(s) alone or by combination of their Contribution(s) + with the Work to which such Contribution(s) was submitted. If You + institute patent litigation against any entity (including a + cross-claim or counterclaim in a lawsuit) alleging that the Work + or a Contribution incorporated within the Work constitutes direct + or contributory patent infringement, then any patent licenses + granted to You under this License for that Work shall terminate + as of the date such litigation is filed. + + 4. Redistribution. You may reproduce and distribute copies of the + Work or Derivative Works thereof in any medium, with or without + modifications, and in Source or Object form, provided that You + meet the following conditions: + + (a) You must give any other recipients of the Work or + Derivative Works a copy of this License; and + + (b) You must cause any modified files to carry prominent notices + stating that You changed the files; and + + (c) You must retain, in the Source form of any Derivative Works + that You distribute, all copyright, patent, trademark, and + attribution notices from the Source form of the Work, + excluding those notices that do not pertain to any part of + the Derivative Works; and + + (d) If the Work includes a "NOTICE" text file as part of its + distribution, then any Derivative Works that You distribute must + include a readable copy of the attribution notices contained + within such NOTICE file, excluding those notices that do not + pertain to any part of the Derivative Works, in at least one + of the following places: within a NOTICE text file distributed + as part of the Derivative Works; within the Source form or + documentation, if provided along with the Derivative Works; or, + within a display generated by the Derivative Works, if and + wherever such third-party notices normally appear. The contents + of the NOTICE file are for informational purposes only and + do not modify the License. You may add Your own attribution + notices within Derivative Works that You distribute, alongside + or as an addendum to the NOTICE text from the Work, provided + that such additional attribution notices cannot be construed + as modifying the License. + + You may add Your own copyright statement to Your modifications and + may provide additional or different license terms and conditions + for use, reproduction, or distribution of Your modifications, or + for any such Derivative Works as a whole, provided Your use, + reproduction, and distribution of the Work otherwise complies with + the conditions stated in this License. + + 5. Submission of Contributions. Unless You explicitly state otherwise, + any Contribution intentionally submitted for inclusion in the Work + by You to the Licensor shall be under the terms and conditions of + this License, without any additional terms or conditions. + Notwithstanding the above, nothing herein shall supersede or modify + the terms of any separate license agreement you may have executed + with Licensor regarding such Contributions. + + 6. Trademarks. This License does not grant permission to use the trade + names, trademarks, service marks, or product names of the Licensor, + except as required for reasonable and customary use in describing the + origin of the Work and reproducing the content of the NOTICE file. + + 7. Disclaimer of Warranty. Unless required by applicable law or + agreed to in writing, Licensor provides the Work (and each + Contributor provides its Contributions) on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or + implied, including, without limitation, any warranties or conditions + of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A + PARTICULAR PURPOSE. You are solely responsible for determining the + appropriateness of using or redistributing the Work and assume any + risks associated with Your exercise of permissions under this License. + + 8. Limitation of Liability. In no event and under no legal theory, + whether in tort (including negligence), contract, or otherwise, + unless required by applicable law (such as deliberate and grossly + negligent acts) or agreed to in writing, shall any Contributor be + liable to You for damages, including any direct, indirect, special, + incidental, or consequential damages of any character arising as a + result of this License or out of the use or inability to use the + Work (including but not limited to damages for loss of goodwill, + work stoppage, computer failure or malfunction, or any and all + other commercial damages or losses), even if such Contributor + has been advised of the possibility of such damages. + + 9. Accepting Warranty or Additional Liability. While redistributing + the Work or Derivative Works thereof, You may choose to offer, + and charge a fee for, acceptance of support, warranty, indemnity, + or other liability obligations and/or rights consistent with this + License. However, in accepting such obligations, You may act only + on Your own behalf and on Your sole responsibility, not on behalf + of any other Contributor, and only if You agree to indemnify, + defend, and hold each Contributor harmless for any liability + incurred by, or claims asserted against, such Contributor by reason + of your accepting any such warranty or additional liability. + + END OF TERMS AND CONDITIONS + + APPENDIX: How to apply the Apache License to your work. + + To apply the Apache License to your work, attach the following + boilerplate notice, with the fields enclosed by brackets "[]" + replaced with your own identifying information. (Don't include + the brackets!) The text should be enclosed in the appropriate + comment syntax for the file format. We also recommend that a + file or class name and description of purpose be included on the + same "printed page" as the copyright notice for easier + identification within third-party archives. + + Copyright [yyyy] [name of copyright owner] + + Licensed under the Apache License, Version 2.0 (the "License"); + you may not use this file except in compliance with the License. + You may obtain a copy of the License at + + http://www.apache.org/licenses/LICENSE-2.0 + + Unless required by applicable law or agreed to in writing, software + distributed under the License is distributed on an "AS IS" BASIS, + WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + See the License for the specific language governing permissions and + limitations under the License. diff --git a/README.md b/README.md index 99844043..049c1e65 100644 --- a/README.md +++ b/README.md @@ -57,36 +57,35 @@ for hero in marvel_heroes.run(connection): ``` ### Asyncio mode -Asyncio mode is compatible with Python ≥ 3.4, which is when asyncio was -introduced into the standard library. +Asyncio mode is compatible with Python ≥ 3.5. ```python import asyncio from rethinkdb import r -# Native coroutines are supported in Python ≥ 3.5. In Python 3.4, you should -# use the @asyncio.couroutine decorator instead of "async def", and "yield from" -# instead of "await". async def main(): - r.set_loop_type('asyncio') - connection = await r.connect(db='test') - - await r.table_create('marvel').run(connection) - - marvel_heroes = r.table('marvel') - await marvel_heroes.insert({ - 'id': 1, - 'name': 'Iron Man', - 'first_appearance': 'Tales of Suspense #39' - }).run(connection) - - # "async for" is supported in Python ≥ 3.6. In earlier versions, you should - # call "await cursor.next()" in a loop. - cursor = await marvel_heroes.run(connection) - async for hero in cursor: - print(hero['name']) - -asyncio.get_event_loop().run_until_complete(main()) + async with await r.connect(db='test') as connection: + await r.table_create('marvel').run(connection) + + marvel_heroes = r.table('marvel') + await marvel_heroes.insert({ + 'id': 1, + 'name': 'Iron Man', + 'first_appearance': 'Tales of Suspense #39' + }).run(connection) + + # "async for" is supported in Python ≥ 3.6. In earlier versions, you should + # call "await cursor.next()" in a loop. + cursor = await marvel_heroes.run(connection) + async for hero in cursor: + print(hero['name']) + # The `with` block performs `await connection.close(noreply_wait=False)`. + +r.set_loop_type('asyncio') + +# "asyncio.run" was added in Python 3.7. In earlier versions, you +# might try asyncio.get_event_loop().run_until_complete(main()). +asyncio.run(main()) ``` ### Gevent mode @@ -253,8 +252,5 @@ $ export DO_TOKEN= $ make test-remote ``` -## New features -Github's Issue tracker is **ONLY** used for reporting bugs. NO NEW FEATURE ACCEPTED! Use [spectrum](https://spectrum.chat/rethinkdb) for supporting features. - ## Contributing Hurray! You reached this section which means, that you would like to contribute. Please read our contributing guide lines and feel free to open a pull request. diff --git a/requirements.txt b/requirements.txt index b6833250..4d2e981d 100644 --- a/requirements.txt +++ b/requirements.txt @@ -1,14 +1,18 @@ async-generator==1.10; python_version>="3.6" +coverage<=4.5.4; python_version<"3.5" +coverage==5.5; python_version>="3.5" codacy-coverage==1.3.11 +looseversion==1.3.0 mock==3.0.5 -pytest-cov==2.8.1 +pytest-cov==2.10.1 pytest-tornasync==0.6.0.post2; python_version >= '3.5' -pytest-trio==0.5.2; python_version>="3.6" +pytest-trio==0.6.0; python_version>="3.6" pytest==4.6.6; python_version<"3.5" -pytest==5.4.1; python_version>="3.5" -six==1.14.0 +pytest==6.1.2; python_version>="3.5" +six==1.15.0 tornado==5.1.1; python_version<"3.6" tornado==6.0.4; python_version>="3.6" -trio==0.13.0; python_version>="3.6" -outcome==1.0.1; python_version>="3.5" -attrs==19.3.0; python_version>="3.5" +trio==0.16.0; python_version>="3.6" +outcome==1.1.0; python_version>="3.6" +outcome==1.0.1; python_version<="3.5" +attrs==20.3.0; python_version>="3.5" diff --git a/rethinkdb/__init__.py b/rethinkdb/__init__.py index 49eef611..70b1661a 100644 --- a/rethinkdb/__init__.py +++ b/rethinkdb/__init__.py @@ -11,10 +11,6 @@ # WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. # See the License for the specific language governing permissions and # limitations under the License. -import imp -import os - -import pkg_resources from rethinkdb import errors, version @@ -50,38 +46,45 @@ def __init__(self): self._index_rebuild = _index_rebuild self._restore = _restore + # Re-export internal modules for backward compatibility + self.ast = ast + self.errors = errors + self.net = net + self.query = query + net.Connection._r = self - for module in (net, query, ast, errors): + for module in (self.net, self.query, self.ast, self.errors): for function_name in module.__all__: setattr(self, function_name, getattr(module, function_name)) self.set_loop_type(None) def set_loop_type(self, library=None): - if library is None: - self.connection_type = net.DefaultConnection - return - - # find module file - manager = pkg_resources.ResourceManager() - libPath = "%(library)s_net/net_%(library)s.py" % {"library": library} - if not manager.resource_exists(__name__, libPath): - raise ValueError("Unknown loop type: %r" % library) - - # load the module - modulePath = manager.resource_filename(__name__, libPath) - moduleName = "net_%s" % library - moduleFile, pathName, desc = imp.find_module( - moduleName, [os.path.dirname(modulePath)] - ) - module = imp.load_module("rethinkdb." + moduleName, moduleFile, pathName, desc) + if library == "asyncio": + from rethinkdb.asyncio_net import net_asyncio + self.connection_type = net_asyncio.Connection + + if library == "gevent": + from rethinkdb.gevent_net import net_gevent + self.connection_type = net_gevent.Connection + + if library == "tornado": + from rethinkdb.tornado_net import net_tornado + self.connection_type = net_tornado.Connection + + if library == "trio": + from rethinkdb.trio_net import net_trio + self.connection_type = net_trio.Connection + + if library == "twisted": + from rethinkdb.twisted_net import net_twisted + self.connection_type = net_twisted.Connection - # set the connection type - self.connection_type = module.Connection + if library is None or self.connection_type is None: + self.connection_type = self.net.DefaultConnection - # cleanup - manager.cleanup_resources() + return def connect(self, *args, **kwargs): return self.make_connection(self.connection_type, *args, **kwargs) diff --git a/rethinkdb/_restore.py b/rethinkdb/_restore.py index 23633f94..2c29eb1c 100755 --- a/rethinkdb/_restore.py +++ b/rethinkdb/_restore.py @@ -180,7 +180,7 @@ def parse_options(argv, prog=None): "Temporary directory doesn't exist or is not a directory: %s" % options.temp_dir ) - if not os.access(options["temp_dir"], os.W_OK): + if not os.access(options.temp_dir, os.W_OK): parser.error("Temporary directory inaccessible: %s" % options.temp_dir) # - create_args @@ -244,7 +244,7 @@ def do_unzip(temp_dir, options): ) # filter out tables we are not looking for - table = os.path.splitext(file_name) + table = os.path.splitext(file_name)[0] if tables_to_export and not ( (db, table) in tables_to_export or (db, None) in tables_to_export ): diff --git a/rethinkdb/ast.py b/rethinkdb/ast.py index 3b9fddc6..3623cbf5 100644 --- a/rethinkdb/ast.py +++ b/rethinkdb/ast.py @@ -20,13 +20,21 @@ import base64 import binascii -import collections import datetime import json +import sys import threading from rethinkdb import ql2_pb2 -from rethinkdb.errors import QueryPrinter, ReqlDriverCompileError, ReqlDriverError, T +from rethinkdb.errors import (QueryPrinter, ReqlDriverCompileError, + ReqlDriverError, T) + +if sys.version_info < (3, 3): + # python < 3.3 uses collections + import collections +else: + # but collections is deprecated from python >= 3.3 + import collections.abc as collections P_TERM = ql2_pb2.Term.TermType @@ -74,7 +82,7 @@ def clear(cls): def expr(val, nesting_depth=20): """ - Convert a Python primitive into a RQL primitive value + Convert a Python primitive into a RQL primitive value """ if not isinstance(nesting_depth, int): raise ReqlDriverCompileError("Second argument to `r.expr` must be a number.") @@ -639,7 +647,7 @@ def compose(self, args, optargs): ] if self.infix: - return T("(", T(*t_args, intsp=[" ", self.statement_infix, " "]), ")") + return T("(", T(*t_args, intsp=[" ", self.st_infix, " "]), ")") else: return T("r.", self.statement, "(", T(*t_args, intsp=", "), ")") @@ -759,7 +767,7 @@ def recursively_make_hashable(obj): class ReQLEncoder(json.JSONEncoder): """ - Default JSONEncoder subclass to handle query conversion. + Default JSONEncoder subclass to handle query conversion. """ def __init__(self): @@ -779,7 +787,7 @@ def default(self, obj): class ReQLDecoder(json.JSONDecoder): """ - Default JSONDecoder subclass to handle pseudo-type conversion. + Default JSONDecoder subclass to handle pseudo-type conversion. """ def __init__(self, reql_format_opts=None): diff --git a/rethinkdb/asyncio_net/net_asyncio.py b/rethinkdb/asyncio_net/net_asyncio.py index 781081e5..e0058c4d 100644 --- a/rethinkdb/asyncio_net/net_asyncio.py +++ b/rethinkdb/asyncio_net/net_asyncio.py @@ -20,6 +20,7 @@ import socket import ssl import struct +import sys from rethinkdb import ql2_pb2 from rethinkdb.errors import ( @@ -39,13 +40,12 @@ pQuery = ql2_pb2.Query.QueryType -@asyncio.coroutine -def _read_until(streamreader, delimiter): +async def _read_until(streamreader, delimiter): """Naive implementation of reading until a delimiter""" buffer = bytearray() while True: - c = yield from streamreader.read(1) + c = await streamreader.read(1) if c == b"": break # EOF buffer.append(c[0]) @@ -69,13 +69,12 @@ def reusable_waiter(loop, timeout): else: deadline = None - @asyncio.coroutine - def wait(future): + async def wait(future): if deadline is not None: new_timeout = max(deadline - loop.time(), 0) else: new_timeout = None - return (yield from asyncio.wait_for(future, new_timeout, loop=loop)) + return (await asyncio.wait_for(future, new_timeout)) return wait @@ -101,20 +100,18 @@ def __init__(self, *args, **kwargs): def __aiter__(self): return self - @asyncio.coroutine - def __anext__(self): + async def __anext__(self): try: - return (yield from self._get_next(None)) + return (await self._get_next(None)) except ReqlCursorEmpty: raise StopAsyncIteration - @asyncio.coroutine - def close(self): + async def close(self): if self.error is None: self.error = self._empty_error() if self.conn.is_open(): self.outstanding_requests += 1 - yield from self.conn._parent._stop(self) + await self.conn._parent._stop(self) def _extend(self, res_buf): Cursor._extend(self, res_buf) @@ -123,8 +120,7 @@ def _extend(self, res_buf): # Convenience function so users know when they've hit the end of the cursor # without having to catch an exception - @asyncio.coroutine - def fetch_next(self, wait=True): + async def fetch_next(self, wait=True): timeout = Cursor._wait_to_timeout(wait) waiter = reusable_waiter(self.conn._io_loop, timeout) while len(self.items) == 0 and self.error is None: @@ -132,7 +128,7 @@ def fetch_next(self, wait=True): if self.error is not None: raise self.error with translate_timeout_errors(): - yield from waiter(asyncio.shield(self.new_response)) + await waiter(asyncio.shield(self.new_response)) # If there is a (non-empty) error to be received, we return True, so the # user will receive it on the next `next` call. return len(self.items) != 0 or not isinstance(self.error, RqlCursorEmpty) @@ -142,15 +138,14 @@ def _empty_error(self): # with mechanisms to return from a coroutine. return RqlCursorEmpty() - @asyncio.coroutine - def _get_next(self, timeout): + async def _get_next(self, timeout): waiter = reusable_waiter(self.conn._io_loop, timeout) while len(self.items) == 0: self._maybe_fetch_batch() if self.error is not None: raise self.error with translate_timeout_errors(): - yield from waiter(asyncio.shield(self.new_response)) + await waiter(asyncio.shield(self.new_response)) return self.items.popleft() def _maybe_fetch_batch(self): @@ -162,6 +157,8 @@ def _maybe_fetch_batch(self): self.outstanding_requests += 1 asyncio.ensure_future(self.conn._parent._continue(self)) +# Python <3.7's StreamWriter has no wait_closed(). +DO_WAIT_CLOSED = sys.version_info >= (3, 7) class ConnectionInstance(object): _streamreader = None @@ -186,8 +183,7 @@ def client_address(self): if self.is_open(): return self._streamwriter.get_extra_info("sockname")[0] - @asyncio.coroutine - def connect(self, timeout): + async def connect(self, timeout): try: ssl_context = None if len(self._parent.ssl) > 0: @@ -199,10 +195,9 @@ def connect(self, timeout): ssl_context.check_hostname = True # redundant with match_hostname ssl_context.load_verify_locations(self._parent.ssl["ca_certs"]) - self._streamreader, self._streamwriter = yield from asyncio.open_connection( + self._streamreader, self._streamwriter = await asyncio.open_connection( self._parent.host, self._parent.port, - loop=self._io_loop, ssl=ssl_context, ) self._streamwriter.get_extra_info("socket").setsockopt( @@ -227,26 +222,25 @@ def connect(self, timeout): break # This may happen in the `V1_0` protocol where we send two requests as # an optimization, then need to read each separately - if request is not "": + if request != "": self._streamwriter.write(request) - response = yield from asyncio.wait_for( + response = await asyncio.wait_for( _read_until(self._streamreader, b"\0"), timeout, - loop=self._io_loop, ) response = response[:-1] except ReqlAuthError: - yield from self.close() + await self.close() raise except ReqlTimeoutError as err: - yield from self.close() + await self.close() raise ReqlDriverError( "Connection interrupted during handshake with %s:%s. Error: %s" % (self._parent.host, self._parent.port, str(err)) ) except Exception as err: - yield from self.close() + await self.close() raise ReqlDriverError( "Could not connect to %s:%s. Error: %s" % (self._parent.host, self._parent.port, str(err)) @@ -260,8 +254,7 @@ def connect(self, timeout): def is_open(self): return not (self._closing or self._streamreader.at_eof()) - @asyncio.coroutine - def close(self, noreply_wait=False, token=None, exception=None): + async def close(self, noreply_wait=False, token=None, exception=None): self._closing = True if exception is not None: err_message = "Connection is closed (%s)." % str(exception) @@ -281,38 +274,39 @@ def close(self, noreply_wait=False, token=None, exception=None): if noreply_wait: noreply = Query(pQuery.NOREPLY_WAIT, token, None, None) - yield from self.run_query(noreply, False) + await self.run_query(noreply, False) self._streamwriter.close() + # Python <3.7 has no wait_closed(). + if DO_WAIT_CLOSED: + await self._streamwriter.wait_closed() # We must not wait for the _reader_task if we got an exception, because that # means that we were called from it. Waiting would lead to a deadlock. if self._reader_task and exception is None: - yield from self._reader_task + await self._reader_task return None - @asyncio.coroutine - def run_query(self, query, noreply): + async def run_query(self, query, noreply): self._streamwriter.write(query.serialize(self._parent._get_json_encoder(query))) if noreply: return None response_future = asyncio.Future() self._user_queries[query.token] = (query, response_future) - return (yield from response_future) + return (await response_future) # The _reader coroutine runs in parallel, reading responses # off of the socket and forwarding them to the appropriate Future or Cursor. # This is shut down as a consequence of closing the stream, or an error in the # socket/protocol from the server. Unexpected errors in this coroutine will # close the ConnectionInstance and be passed to any open Futures or Cursors. - @asyncio.coroutine - def _reader(self): + async def _reader(self): try: while True: - buf = yield from self._streamreader.readexactly(12) + buf = await self._streamreader.readexactly(12) (token, length,) = struct.unpack("= %s got: %s" @@ -170,7 +170,7 @@ def check_minimum_version(options, minimum_version="1.6", raise_exception=True): DbTable = collections.namedtuple("DbTable", ["db", "table"]) -_tableNameRegex = re.compile(r"^(?P\w+)(\.(?P\w+))?$") +_tableNameRegex = re.compile(r"^(?P[\w-]+)(\.(?P
[\w-]+))?$") class CommonOptionsParser(optparse.OptionParser, object): @@ -308,7 +308,7 @@ def take_action(self, action, dest, opt, value, values, parser): values.ensure_value(dest, {})[self.metavar.lower()] = value elif action == "get_password": - values[dest] = getpass.getpass("Password for `admin`: ") + values.ensure_value('password', getpass.getpass("Password for `admin`: ")) else: super(CommonOptionChecker, self).take_action( action, dest, opt, value, values, parser diff --git a/rethinkdb/version.py b/rethinkdb/version.py index 9c4ac378..572868c2 100644 --- a/rethinkdb/version.py +++ b/rethinkdb/version.py @@ -15,4 +15,4 @@ # This file incorporates work covered by the following copyright: # Copyright 2010-2016 RethinkDB, all rights reserved. -VERSION = "2.4.0+source" +VERSION = "2.4.10.post1+source" diff --git a/scripts/convert_protofile.py b/scripts/convert_protofile.py index 98f676e3..ec80b3e8 100644 --- a/scripts/convert_protofile.py +++ b/scripts/convert_protofile.py @@ -86,9 +86,9 @@ def convertFile(inputFile, outputFile, language): assert(outputFile is not None and hasattr(outputFile, 'write')) assert(language in languageDefs) - messageRegex = re.compile('\s*(message|enum) (?P\w+) \{') - valueRegex = re.compile('\s*(?P\w+)\s*=\s*(?P\w+)') - endRegex = re.compile('\s*\}') + messageRegex = re.compile(r'\s*(message|enum) (?P\w+) \{') + valueRegex = re.compile(r'\s*(?P\w+)\s*=\s*(?P\w+)') + endRegex = re.compile(r'\s*\}') indentLevel = languageDefs[language]["initialIndentLevel"] lastIndentLevel = languageDefs[language]["initialIndentLevel"] - 1 diff --git a/scripts/install-db.sh b/scripts/install-db.sh index d80307e8..69d7a355 100755 --- a/scripts/install-db.sh +++ b/scripts/install-db.sh @@ -5,8 +5,8 @@ set -u export DISTRIB_CODENAME=$(lsb_release -sc) -echo "deb https://download.rethinkdb.com/apt $DISTRIB_CODENAME main" | sudo tee /etc/apt/sources.list.d/rethinkdb.list -wget -qO- https://download.rethinkdb.com/apt/pubkey.gpg | sudo apt-key add - +sudo apt-key adv --keyserver keys.gnupg.net --recv-keys "539A 3A8C 6692 E6E3 F69B 3FE8 1D85 E93F 801B B43F" +echo "deb https://download.rethinkdb.com/repository/ubuntu-xenial xenial main" | sudo tee /etc/apt/sources.list.d/rethinkdb.list sudo apt-get update --option Acquire::Retries=100 --option Acquire::http::Timeout="300" sudo apt-get install -y --option Acquire::Retries=100 --option Acquire::http::Timeout="300" rethinkdb diff --git a/setup.py b/setup.py index 62aa03c3..ded5627f 100644 --- a/setup.py +++ b/setup.py @@ -21,8 +21,6 @@ import setuptools -from rethinkdb.version import VERSION - try: import asyncio @@ -32,26 +30,22 @@ RETHINKDB_VERSION_DESCRIBE = os.environ.get("RETHINKDB_VERSION_DESCRIBE") -VERSION_RE = r"^v(?P\d+\.\d+)\.(?P\d+)?(\.(?P\w+))?$" - -if RETHINKDB_VERSION_DESCRIBE: - MATCH = re.match(VERSION_RE, RETHINKDB_VERSION_DESCRIBE) +VERSION_RE = r"(?P0|[1-9]\d*)\.(?P0|[1-9]\d*)\.(?P0|[1-9]\d*)\.(?Ppost[1-9]\d*)" - if MATCH: - VERSION = MATCH.group("version") +with open("rethinkdb/version.py", "r") as f: + version_parts = re.search(VERSION_RE, f.read()).groups() + VERSION = ".".join(filter(lambda x: x is not None, version_parts)) - if MATCH.group("patch"): - VERSION += "." + MATCH.group("patch") - if MATCH.group("post"): - VERSION += "." + MATCH.group("post") +if RETHINKDB_VERSION_DESCRIBE: + version_parts = re.match(VERSION_RE, RETHINKDB_VERSION_DESCRIBE) - with open("rethinkdb/version.py", "w") as f: - f.write('VERSION = {0}'.format(repr(VERSION))) - else: + if not version_parts: raise RuntimeError("{!r} does not match version format {!r}".format( RETHINKDB_VERSION_DESCRIBE, VERSION_RE)) + VERSION = ".".join(filter(lambda x: x is not None, version_parts.groups())) + setuptools.setup( name='rethinkdb', @@ -73,6 +67,10 @@ 'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.7', 'Programming Language :: Python :: 3.8', + 'Programming Language :: Python :: 3.9', + 'Programming Language :: Python :: 3.10', + 'Programming Language :: Python :: 3.11', + 'Programming Language :: Python :: 3.12', ], packages=[ 'rethinkdb', @@ -95,9 +93,10 @@ 'rethinkdb-repl = rethinkdb.__main__:startInterpreter' ] }, - python_requires=">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*", + python_requires=">=2.7, !=3.0.*, !=3.1.*, !=3.2.*, !=3.3.*, >=3.5", install_requires=[ - 'six' + 'six', + 'looseversion' ], test_suite='tests' ) diff --git a/tests/integration/test_cursor.py b/tests/integration/test_cursor.py index 54ec631d..684b89c4 100644 --- a/tests/integration/test_cursor.py +++ b/tests/integration/test_cursor.py @@ -1,6 +1,6 @@ import pytest -from rethinkdb.errors import ReqlCursorEmpty +from rethinkdb.errors import ReqlCursorEmpty, ReqlTimeoutError from tests.helpers import IntegrationTestCaseBase @@ -55,6 +55,23 @@ def test_stop_iteration(self): for i in range(0, len(self.documents) + 1): cursor.next() + def test_iteration_after_timeout(self): + """Getting a `ReqlTimeoutError` while using a cursor, should not + close the underlying connection to the server. + """ + # Note that this cursor is different to the others - it uses `.changes()` + cursor = self.r.table(self.table_name).changes().run(self.conn) + + # Attempting to set `wait=False` on this changes query will timeout, + # as data is not available yet + with pytest.raises(ReqlTimeoutError): + cursor.next(wait=False) + + # We should be able to call the cursor again after a timeout, + # such a timeout should not cause the underlying connection to close + with pytest.raises(ReqlTimeoutError): + cursor.next(wait=False) + def test_for_loop(self): self.r.table(self.table_name).insert(self.documents).run(self.conn)