dshape does not stick in bz.Data(...) #1674
Description
Hello,
I am having trouble passing in a dshape to bz.Data(...) for an sqlite3 table. The minimal reproducible error is:
import blaze as bz
from datashape import dshape
import numpy as np
import pandas as pd
import sqlite3
np.random.seed(100)
start = pd.Timestamp('2015-01-01')
end = pd.Timestamp('2015-12-31')
dates = pd.date_range(start, end)
sids = np.arange(0, 100)
df = pd.DataFrame(
data={'value': np.random.random(size=len(dates)*len(sids))},
index = pd.MultiIndex.from_tuples(list(itertools.product(dates,sids)), names=('asof_date', 'sid'))
)
df = df.reset_index()
df.to_sql('my_table', con=sqlite3.connect('temp.db'), if_exists='replace', index=False)
my_dshape = dshape("var*{asof_date: datetime, sid: int64, value: float64}")
expr = bz.Data(
'sqlite:///temp.db::my_table',
dshape=my_dshape
)If I then inspect expr with expr.dshape I get
dshape("var * {asof_date: ?datetime, sid: ?int32, value: ?float32}")
The question is why does the dshape not stick?
The table looks fine:
con = sqlite3.connect('temp.db')
cursor = con.cursor()
cursor.execute("SELECT * FROM my_table LIMIT 3")
cursor.fetchall()gives
[(u'2015-01-01 00:00:00', 0, 0.5434049417909654),
(u'2015-01-01 00:00:00', 1, 0.27836938509379616),
(u'2015-01-01 00:00:00', 2, 0.4245175907491331)]
and cursor.execute("PRAGMA table_info('my_table')").fetchall() gives
[(0, u'asof_date', u'TIMESTAMP', 0, None, 0),
(1, u'sid', u'INTEGER', 0, None, 0),
(2, u'value', u'REAL', 0, None, 0)]
pip freeze is
alembic==0.7.7
alphalens==0.3.2
-e git+https://[email protected]/marketneutral/alphatools.git@458bc866f5a31978986e2cdf49b4f2cf8f54aa87#egg=alphatools
appnope==0.1.0
asn1crypto==0.24.0
backports-abc==0.5
backports.functools-lru-cache==1.5
backports.shutil-get-terminal-size==1.0.0
bcolz==0.12.1
blaze==0.10.1
Bottleneck==1.2.1
certifi==2018.8.13
cffi==1.11.5
chardet==3.0.4
click==6.7
contextlib2==0.5.5
cryptography==2.3.1
cryptography-vectors==2.3.1
-e git+https://[email protected]/marketneutral/cvxportfolio.git@152188adac72c1bce85f1aa526e85b4f611cf497#egg=cvxportfolio
cvxpy==1.0.8
cycler==0.10.0
cyordereddict==0.2.2
Cython==0.28.5
datashape==0.5.2
decorator==4.3.0
dill==0.2.8.2
ecos==2.0.5
empyrical==0.5.0
enum34==1.1.6
fastcache==1.0.2
Flask==1.0.2
Flask-Cors==3.0.6
future==0.16.0
futures==3.2.0
idna==2.7
intervaltree==2.1.0
ipaddress==1.0.22
ipykernel==4.8.2
ipython==5.8.0
ipython-genutils==0.2.0
itsdangerous==0.24
Jinja2==2.10
jupyter-client==5.2.3
jupyter-core==4.4.0
kiwisolver==1.0.1
Logbook==0.12.5
lru-dict==1.1.4
lxml==4.2.4
Mako==1.0.7
MarkupSafe==1.0
matplotlib==2.2.3
multipledispatch==0.5.0
multiprocess==0.70.6.1
networkx==1.11
numexpr==2.6.6
numpy==1.14.3
odo==0.5.0
osqp==0.4.0
pandas==0.22.0
pandas-datareader==0.6.0
pathlib2==2.3.2
patsy==0.5.0
pexpect==4.6.0
pickleshare==0.7.4
pkginfo==1.4.2
prompt-toolkit==1.0.15
psutil==5.4.7
ptyprocess==0.6.0
pycparser==2.18
Pygments==2.2.0
pyOpenSSL==18.0.0
pyparsing==2.2.0
PySocks==1.6.8
python-dateutil==2.7.3
pytz==2018.5
pyzmq==17.1.2
requests==2.19.1
requests-file==1.4.3
requests-ftp==0.3.1
requests-toolbelt==0.8.0
scandir==1.9.0
scikit-learn==0.19.2
scipy==1.1.0
scs==2.0.2
seaborn==0.9.0
simplegeneric==0.8.1
singledispatch==3.4.0.3
six==1.11.0
sortedcontainers==1.4.4
SQLAlchemy==1.2.10
statsmodels==0.9.0
subprocess32==3.5.2
tables==3.4.4
toolz==0.9.0
tornado==5.1
tqdm==4.25.0
trading-calendars==1.0.1
traitlets==4.3.2
twine==1.11.0
urllib3==1.23
wcwidth==0.1.7
Werkzeug==0.14.1
wrapt==1.10.11
zipline==1.3.0