TileDB Resource Requirements for Image Processing

1

Hi TileDB Community,

We are evaluating TileDB for slicing image data stored in arrays. According to the documentation, it’s recommended to have nodes with at least 24GB of memory and 8 CPUs, which translates to a 1:3 CPU-memory ratio. However, we noticed that the default configuration in the TileDB Helm chart uses a closer-to 1:2 ratio.

Given our use case—processing images of around 1.5GB in size, which expand to about 10GB in TileDB array —we are wondering about the optimal resource allocation. Specifically:

  • Does TileDB have similar memory requirements as databases like MongoDB or PostgreSQL, or does it rely more on CPU performance for tasks like image slicing?
  • Is it crucial to stick to the 1:3 ratio, or would the 1:2 ratio provided in the Helm chart suffice for our image data processing needs?

Any insights or recommendations on CPU and memory allocation based on similar workloads would be greatly appreciated!

Thanks in advance for your help!

2

Hi @Andor_Markus, apologies for the delayed response here – this depends on your use-case and data configuration, but in this circumstance for compressed data I would suggest 1:3.