Can't Consolidate - "Got Multiple Arguments for URI"

1

Hey,

I’m trying to consolidate the fragments of an array but am getting an error message I can’t get to the bottom of.I’m carrying out the following operation:

config = tiledb.Config({'sm.consolidation.mode': 'fragments'})

with tiledb.open('path/to/array', 'w') as TileDB_array:
    TileDB_array.consolidate(config)

And get returned:

TypeError: consolidate() got multiple values for keyword argument ‘uri’

Does anyone know how I can solve this?

I’m using v2 of TileDB and the latest Python version

2

Hi @jake,

We’ll fix the array.consolidate(...) version in the next point release, but the top-level function will also work for now:

tiledb.consolidate("/path/to/array", config=config)

HTH,
Isaiah

3

Hi @ihnorton,

Thanks for response and suggestions.

I’ve just tried …

array_dir = f'/path/to/array'.replace('/', '\\')
config = tiledb.Config({'sm.consolidation.mode': 'fragments'})

tiledb.consolidate(array_dir, config=config)

… and it appears to run succesfully with no errors raised but the fragments are still fragmented. It also returns the path of the array which I assume is related to the operation being carried out successfully?

I’ve tried this on multiple arrays with a few different config parameters to no luck.

Thanks!
Jake

4

Hi @Jake,

TileDB provides versioned arrays to facilitate “time-traveling” over different data views, which can be useful in a number of different applications. Due to a number of requests, the latest core TileDB release does not remove consolidated fragments by default during the consolidation step. This preserves the capability to open the array at an earlier timepoint while still providing the improved performance of a consolidated view.

In order to remove the prior fragments, there is a new “vacuum” step, which is documented here. Consolidation and vacuuming may be restricted to array metadata or fragment footers (range information) with the sm.consolidation.mode configuration parameter.

Best,
Isaiah