Thanks to visit codestin.com
Credit goes to github.com

Skip to content

Conversation

@tonycosentini
Copy link

This is opened after noticing description is much slower in 3.x. I'm pretty unfamiliar with psycopg (and even more so with cython) so this might be more of a starting point for discussion.

Changes

  • Fetching the encoding (even though it's in a Python LRU cahce) was kind of expensive. Now it's cached in the cursor once its fetched once until a reset().
  • Cythonified the Column class so libpq can be called directly.
  • Made the work to get names + types as lazy as possible.

Benchmarking

Benchmark code (table has ~20 columns):

def benchmark_description(conn, iterations=10000):
    """Benchmark cursor.description access and return elapsed nanoseconds"""
    start_ns = time.perf_counter_ns()
    cur = conn.cursor()
    cur.execute("SELECT * FROM benchmark_table LIMIT 1")

    # Access description many times
    for _ in range(iterations):
        desc = cur.description

    cur.close()
    return time.perf_counter_ns() - start_ns

Rough timing from my machine:

  • psycopg2 (2.9.11) - 0.004170051 seconds (4170051 ns)
  • psycopg[c] (3.3.2) - 2.072374879 seconds (2072374879 ns)
  • psycopg[c] (with this PR) - 0.395435509 seconds (395435509 ns)

Testing

There are no API changes so the existing tests should be sufficient.

@tonycosentini
Copy link
Author

I'm closing this since while the performance change is noticeable - it's still orders of magnitude off from what psycopgp2 is doing. I don't think the the additional overhead/complexity is worth it for these small gains (but if anyone else wants to take it over, by all means go for it).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant