How to consolidate to a certain timepoint

1

Currently I don’t see tiledb::Array::consolidate function takes the optional argument of timestamp, at which I’d like to consolidate.
Or put it this way, do we have api to drop the latest layers of fragments as a mechanism to roll-back to the certain historical point. I know I can open the array at a given timestamp, but not sure to permanently walk back to it.

Here is the use case:
I have a raw data written to tiledb, it goes through two stages of data transformation: trans1, trans2.
If I decide trans2 isn’t done properly, I’d like to strip trans2 fragment/timestamp, and perform another run of transformation on top of trans1. Or I’d like to cancel both trans layers and go back to the raw data stage and start all over again.

Also If there is a native support to tag the timestamp with a string , e.g. ‘raw’, ‘trans1’, ‘trans2’, so that I can easily identify and access each fragment layer.

2

Hi Mike, you are raising two nice issues. Would you mind adding them as a feature suggestions at https://feedback.tiledb.com/?

  • Undo functionality, i.e., “cancel/delete” fragments based on time.
  • Tag fragments with strings/labels so that you can use those when opening the array, or consolidating or undoing.

We’ll try to work on these soon. Thanks!

3

I’ve submitted feature suggestions.

4

Thank you Mike! We’ll work on those features soon.