TileDB for petabyte images / video storage and fast retrieval?

I was wondering if TileDB was a good fit for our use case.

We need to query images and videos and load them as fast as possible into the GPU for display. We’re building a massive visualization of terabytes of video archives.

We extract features from frames and need to create different level of details (texture atlas pyramids) e.g 4k image to 2k to 256px to 128px to 32px and build texture atlas so we can load as many thumbnails as possible very quickly.

These atlases would be ideally created on the fly from user queries: e.g create 4 x 8k square textures for all the images with the most red from their 32px representation.

8192px / 32 = 256, 256x256 = 65536, 4x65536 = 262144 thumbnails.
So we could basically load 262’144 thumbnails in one query from 4 images.

Writing speed is irrelevant since it is mostly to read data back. Database will be local to the system.

It’s a very specific usecase and very challenging technically, but maybe an expert could weigh in?

Thank you very much



Thanks for reaching out! This seems to be right up our alley (we’ve been constructing such pyramidal structures for various biomedical and geospatial imaging applications for interactive visualization purposes). I suggest we get on a call with our imaging experts (@ihnorton on biomedical and @Norman_Barker on geospatial). Please feel free to email me at stavros@tiledb.com and we’ll get a quick demo working.