I ingested a multindex dataframe as follows:
tiledb.from_pandas('Binance_VOptions',df.reset_index().set_index(['time','pair','type','strike','expiry',]))
This resulted in the following schema:
ArraySchema(
domain=Domain(*[
Dim(name='time', domain=(numpy.datetime64('2021-01-15T07:52:12.000000000'), numpy.datetime64('2021-02-15T22:14:31.000000000')), tile=10 nanoseconds, dtype='datetime64[ns]'),
Dim(name='pair', domain=(None, None), tile=None, dtype='|S0'),
Dim(name='type', domain=(None, None), tile=None, dtype='|S0'),
Dim(name='strike', domain=(24000.0, 64000.0), tile=10.0, dtype='float64'),
Dim(name='expiry', domain=(numpy.datetime64('2021-02-19T00:00:00.000000000'), numpy.datetime64('2021-03-26T00:00:00.000000000')), tile=10 nanoseconds, dtype='datetime64[ns]'),
]),
attrs=[
Attr(name='id', dtype='int64', var=False, filters=FilterList([ZstdFilter(level=1), ])),
Attr(name='price', dtype='float64', var=False, filters=FilterList([ZstdFilter(level=1), ])),
Attr(name='qty', dtype='float64', var=False, filters=FilterList([ZstdFilter(level=1), ])),
Attr(name='quoteQty', dtype='float64', var=False, filters=FilterList([ZstdFilter(level=1), ])),
Attr(name='side', dtype='int64', var=False, filters=FilterList([ZstdFilter(level=1), ])),
Attr(name='contract', dtype='<U0', var=True, filters=FilterList([ZstdFilter(level=1), ])),
Attr(name='CVD', dtype='float64', var=False, filters=FilterList([ZstdFilter(level=1), ])),
],
cell_order='row-major',
tile_order='row-major',
capacity=10000,
sparse=True,
allows_duplicates=True,
coords_filters=FilterList([ZstdFilter(level=-1), ])
)
But it looks like dimensions of type ‘|S0’ can’t be read, using the .df[:] method (A[:] works):
with tiledb.open('Binance_VOptions','r') as A:
data = A.query(use_arrow=True,dims=('time','pair','type','strike','expiry',)).df[:]
data
By removing ‘pair’, and ‘type’ from the above query, the code works. Otherwise the script crashes without errors.
tiledb version: 0.8.2