First off, I just would like to acknowledge how awesome this project is! There’s some very cool technology here, and I’m excited to see the project develop in the future!
As a bit of background I’m doing a deep dive with TileDB-py right now, primarily dealing with single attribute DenseArrays. My application issues multiple writes to the same array (with both complete re-assignment for every cell, and with sub-array assignment at arbitrary indices - which I am not able to predict in advance). Reads are issued for arbitrary (and unpredictable) subarrays as they existed at particular timesteps.
I’m wondering if there is any way to have tiledb record some hash (sha1, md5, etc) of the data stored in a DenseArray after a new fragment has been written? To clarify, I am referring to the hash of the full consolidated/superimposed data set (for every cell) which would be generated after writing some fragment.
Is there any built-in utility that would let me achieve this? If not, do you have any suggestions which might achieve this in a more sane way than following each write with an immediate read of the full array into memory so the data can be hashed manually?
Let me know, and thank you in advance!