Internal TileDB uncaught exception; basic_string::compare:

1

Hi,

I’m trying to port a Python program from an HDFS back end to S3. Running some simple tests I’m getting an uncaught exception from TileDB. As s3 back ends, I have tried both minio and the Ceph object store. Here’s an example with a stack trace from pytest:

    def test_basic_tiledb_s3_operativity(clean_s3):
        import tiledb

        uri =  's3://firstbucket/basic-tiledb-s3-operativity'
        config = tiledb.Config(params={
            'vfs.s3.aws_access_key_id': 'tdm-user',
            'vfs.s3.aws_secret_access_key': 'tdm-user-s3',
            'vfs.s3.endpoint_override': 'minio:9000',
            'vfs.s3.scheme': 'http',
            'vfs.s3.region': '',
            'vfs.s3.verify_ssl': 'false',
            'vfs.s3.use_virtual_addressing': 'false',
            'vfs.s3.use_multipart_upload': 'false'})
        ctx = tiledb.Ctx(config=config)
    
        a = np.arange(5)
        logging.debug("trying to write array to s3: %s", uri)
        schema = tiledb.schema_like(a, ctx=ctx)
>       tiledb.DenseArray.create(uri, schema)

tests/client/test_non_scalar_source.py:131:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
tiledb/libtiledb.pyx:3697: in tiledb.libtiledb.Array.create
    ???
tiledb/libtiledb.pyx:481: in tiledb.libtiledb._raise_ctx_err
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

>   ???
E   tiledb.libtiledb.TileDBError: Error: Internal TileDB uncaught exception; basic_string::compare: __pos (which is 18446744073709551615) > this->size() (which is 4)

tiledb/libtiledb.pyx:466: TileDBError

In other test, the same unhandled exception is raised from tiledb.object_type(uri, context). While the stack trace in that case is different, the exception message is identical:

tiledb.libtiledb.TileDBError: Error: Internal TileDB uncaught exception; basic_string::compare: __pos (which is 18446744073709551615) > this->size() (which is 4)

TileDB version: 0.7.0

I’m running in docker. I have built my own image and installed tiledb with pip.

Any hints?

Thanks,

Luca

2

I’ll add a detail that seems relevant: sometimes, this test passes.

My complete test scenario first executes a function that uses boto to delete any objects in the s3 bucket (that’s the clean_s3 pytest fixture left in the code I pasted), then executes the test function. If I launch the test a few times in quick succession, the first time it’ll pass; subsequent attempts all fail with the same exception.

3

Hi @ilveroluca –

Thanks for the report. I’m working on reproducing this to see what’s going wrong. I’ll let you know once I have an update.

4

Hi Joe,

I opened an issue on Github with a smaller code snipped sufficient to reproduce the problem. Here’s the link: https://github.com/TileDB-Inc/TileDB-Py/issues/409

Since posting, I have confirmed that TileDB 0.6.1 doesn’t have the issue (I tried both your docker image on DockerHub and I rebuilt my own). I’m currently working on testing TileDB 0.6.6.

5

0.6.6 also works. It looks like a regression in 0.7.0.