# Tiledb cache management

**URL:** <https://forum.tiledb.com/t/tiledb-cache-management/320>\
**Category:** Uncategorized\
**Created:** [March 23, 2021, 4:52pm UTC](https://forum.tiledb.com/t/tiledb-cache-management/320 "2021-03-23T16:52:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![tiphaineruy](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.tiledb.com/tiphaineruy/32/126_2.png) [@tiphaineruy](https://forum.tiledb.com/u/tiphaineruy)\
**Post date:** [March 23, 2021, 4:52pm UTC](https://forum.tiledb.com/t/tiledb-cache-management/320/1 "2021-03-23T16:52:55Z")

</div>

Hello !  
I was wondering a bit on how to optimize tile fetching and caching on arrays.

So from what I’ve gathered. The only way to cache a cloud array for now is by playing with the parameter `sm.tile_cache_size`. Which impact an uncompressed in memory LRU cache of fetched tiles by the current session.

The Issue is that, sometimes, you might want to keep a “generated” cache from one session to another ( as in, commit the cache to the client filesystem) ( For example on compute instances, on long lived sessions that you might have to restart at some point without loosing the cache, or have multiple tiledb clients opening the same cache etc… )

I found an old issue relating to that problem.

> <https://github.com/TileDB-Inc/TileDB/issues/1366>
>
> When a read query causes a tile to be read from disk, TileDB places the uncompressed tile in an in-memory LRU...

But I’m not sure if you had adressed it at some point and I’m missing something.

My current workaround is using minio gateway:

```auto
docker run \
--name minio-s3-gateway \
-d \
-e MINIO_ROOT_USER=$PUB_KEY \
-e MINIO_ROOT_PASSWORD=$PRIV_KEY \
-e MINIO_CACHE="on" \
-e MINIO_CACHE_DRIVES="/data" \
-p 9000:9000 \
-v minio-cache:/data \
minio/minio gateway s3

```

(You can tune some more params for the caching policy )  
It uses minio as a local cache for the fetched objects.  
This has the benefit of faster metadata fetch on array opening.

I guess the question is: Is there a way to create a longer lived cache of fetched tiles that isn’t cleared when you close the client / can reuse between tiledb embeded clients ?

Thanks !

---

<div class="post-metadata">

**Author:** ![stavros](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@stavros](https://forum.tiledb.com/u/stavros)\
**Post date:** [March 25, 2021, 5:49pm UTC](https://forum.tiledb.com/t/tiledb-cache-management/320/2 "2021-03-25T17:49:40Z")

</div>

Hi @tiphaineruy, this indeed would be a nice feature (and your workaround is valid). It’s not in our roadmap currently but we could work on it in the future.

If you don’t mind, could you please add it on  
[our feature request site](https://feedback.tiledb.com/tiledb-core)? Our team will add a ticket to our backlog.

---

<div class="post-metadata">

**Author:** ![tiphaineruy](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.tiledb.com/tiphaineruy/32/126_2.png) [@tiphaineruy](https://forum.tiledb.com/u/tiphaineruy)\
**Post date:** [March 25, 2021, 7:12pm UTC](https://forum.tiledb.com/t/tiledb-cache-management/320/3 "2021-03-25T19:12:28Z")

</div>

Done @stravos : [Support Array caching | Voters | TileDB](https://feedback.tiledb.com/tiledb-core/p/support-array-caching)

Though indeed it’s not a really high priority issue as the minio gateway workaround works. But I’m sure long term some edge cases might require a tighter integration with tiledb embeded.

cheers.

---

<div class="post-metadata">

**Author:** ![ghpu](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.tiledb.com/ghpu/32/252_2.png) [@ghpu](https://forum.tiledb.com/u/ghpu)\
**Post date:** [March 6, 2023, 4:05pm UTC](https://forum.tiledb.com/t/tiledb-cache-management/320/4 "2023-03-06T16:05:21Z")

</div>

Minio gateway is now deprecated, it might be a good time to raise the caching issue again.