Checking SQL code with SQLGlot #193
Replies: 3 comments 8 replies
-
|
Hmm |
Beta Was this translation helpful? Give feedback.
-
|
I guess ...
from dataclasses import dataclass
@dataclass
class PartClass:
pnum: int
qoh: int
...
def silly1(parts: list[PartClass]) -> list[tuple[int]]:
stmt = """
select parts.pnum
from parts
"""
dict_parts = [
{"pnum": p.pnum, "qoh": p.qoh}
for p in parts
]
return execute(
stmt,
tables=dict(parts=dict_parts),
schema=dict(parts=dict(pnum="int", qoh="int")),
).rows
def silly2(parts: list[PartClass]) -> list[tuple[int]]:
stmt = """
select parts.pnum
from parts
where false
"""
dict_parts = [
{"pnum": p.pnum, "qoh": p.qoh}
for p in parts
]
return execute(
stmt,
tables=dict(parts=dict_parts),
schema=dict(parts=dict(pnum="int", qoh="int")),
).rowsOutput: |
Beta Was this translation helpful? Give feedback.
-
|
So thanks to @tobymao we get functionality similar to @akcheung's Cosette. Cosette is probably a little more powerful than this but it only supports integer types and requires a special form of SQL. I wonder what other languages have pure-Python interpreters. |
Beta Was this translation helpful? Give feedback.
-
|
cool! fyi you shouldn’t need to pass schema |
Beta Was this translation helpful? Give feedback.
-
|
Oh wow. It would be amazing to get this working. Indeed, you've run into an issue with TypedDicts, specifically when inside containers. It'll take me a day or two to make sure I fix that correctly. I'm sort of shocked to see that the rest of the process works out. (even a small difference like |
Beta Was this translation helpful? Give feedback.
-
|
Yup! I should be able to get the regular |
Beta Was this translation helpful? Give feedback.
-
|
the execution engine is new, but we can pass around 17 tpc-h queries |
Beta Was this translation helpful? Give feedback.
-
Very cool! To clarify, though, my curiosity was mostly about CrossHair's ability (or inability) to effectively reason about SQLGlot's SQL execution code. To the extent it doesn't, I'd love to add examples to CrossHair's benchmark suite. |
Beta Was this translation helpful? Give feedback.
-
|
I think this is almost equivalent to the dataframe discussion and the associated issue in terms of performance. Though you are starting from sql equality instead of pre and post conditions, I think it ultimately boils down improve the same parts (mainly how to scale when you add a bigger number of columns and different types). So If one is solved the other is solved, I will keep an eye on this. |
Beta Was this translation helpful? Give feedback.
-
|
A note in regards to "realistic SQL": These two queries come from the "count bug" here. from typing import TypedDict
from sqlglot.executor import execute
class Part(TypedDict):
pnum: int
qoh: int
class Supply(TypedDict):
pnum: int
shipdate: int
schema = dict(
parts=dict(pnum="int", qoh="int"),
supplies=dict(pnum="int", shipdate="int"),
)
def q1td(parts: list[Part], supplies: list[Supply]) -> list[tuple[int]]:
stmt = """
select parts.pnum
from parts
where parts.qoh = (
select count(supplies.shipdate)
from supplies
where supplies.pnum = parts.pnum and supplies.shipdate < 10
)
"""
return execute(
stmt,
tables=dict(parts=parts, supplies=supplies),
schema=schema,
).rows
def q2td(parts: list[Part], supplies: list[Supply]) -> list[tuple[int]]:
stmt = """
select parts.pnum
from parts, (
select supplies.pnum as pnum, count(supplies.shipdate) as cnt
from supplies
where supplies.shipdate < 10
group by supplies.pnum
) temp
where parts.qoh = temp.cnt and parts.pnum = temp.pnum
"""
return execute(
stmt,
tables=dict(parts=parts, supplies=supplies),
schema=schema,
).rowsThe which is awesome (although With the default which is confusing because when you run the functions on those inputs they don't raise those exceptions. |
Beta Was this translation helpful? Give feedback.
-
|
Thanks for the example @amacfie! While improving performance is a priority, false positives are always bugs, so I've filed #196. (as you may have guessed, sqlglot's exception handling is handling CrossHair's timeout exception) Solving that particular problem shouldn't be a huge lift, though I am spread a little thin these days - hopefully soon. |
Beta Was this translation helpful? Give feedback.
-
SQLGlot has a pure Python SQL engine. So it would be cool if we could use CrossHair to check SQL code.
However,
crosshair diffbehavioronsilly1andsilly2gives "All paths exhausted, functions are likely the same!". What would we need to do to make this work?Beta Was this translation helpful? Give feedback.
All reactions