# Concurrent read tile arrays from s3 **URL:** \ **Category:** Uncategorized\ **Created:** [March 25, 2021, 11:45pm UTC](https://forum.tiledb.com/t/concurrent-read-tile-arrays-from-s3/322 "2021-03-25T23:45:43Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![mikejiang](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.tiledb.com/mikejiang/32/102_2.png) [@mikejiang](https://forum.tiledb.com/u/mikejiang)\ **Post date:** [March 25, 2021, 11:45pm UTC](https://forum.tiledb.com/t/concurrent-read-tile-arrays-from-s3/322/1 "2021-03-25T23:45:44Z")
Can’t seem to get concurrent read work (in order to speed up ) here is my config ```auto cfg["sm.compute_concurrency_level"] = 10; cfg["sm.io_concurrency_level"] = 10; cfg["vfs.min_parallel_size"] = 10485760; // set it to default for now ,yet to figure out how to use it // (set it too small as 100k seems to slow down read) ``` and here is my tile 2d array layout ```auto domain.add_dimension(tiledb::Dimension::create( ctx, "cell", {1, ncell}, ncell)); // @suppress("Invalid arguments") // @suppress("Symbol is not // resolved") domain.add_dimension(tiledb::Dimension::create( ctx, "channel", {1, nch == 0 ? 1 : nch}, 1)); // @suppress("Invalid arguments") // @suppress("Symbol is not // resolved") tiledb::ArraySchema schema(ctx, TILEDB_DENSE); schema.set_domain(domain); schema.add_attribute(tiledb::Attribute::create(ctx, "mat")); schema.set_tile_order(TILEDB_COL_MAJOR).set_cell_order(TILEDB_COL_MAJOR); ``` basically store cells x channels 2d array, each row is a cell, each col is a channel, and access pattern is reading by channel, thus I am tiling by channel. I wonder how the concurrent read request can be applied here to optimize reading multi-channel at once, say using 10 threads to fetch 10 channels concurrently. here is what my query looks like ```auto tiledb::Query query(*ctxptr_, *mat_array_ptr_); query.set_layout(TILEDB_COL_MAJOR); int ncol = 0; int nrow = 0; int dim_idx = 0; ncol = cidx.size(); nrow = dims[0]; dim_idx = 1; query.add_range(0, 1, nrow); // select all rows // tiledb idx starting from 1 for (int i : cidx) { query.add_range(dim_idx, i + 1, i + 1); } arma::Mat buf(nrow, ncol); query.set_buffer("mat", buf.memptr(), nrow * ncol); query.submit(); query.finalize(); ``` Hope someone can point me to the right direction here ---
**Author:** ![stavros](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@stavros](https://forum.tiledb.com/u/stavros)\ **Post date:** [March 30, 2021, 5:37pm UTC](https://forum.tiledb.com/t/concurrent-read-tile-arrays-from-s3/322/2 "2021-03-30T17:37:01Z")
Thanks @mikejiang! Just I notice that we don’t parallelize over attributes. We will open a ticket and address this very soon. ---
**Author:** ![mikejiang](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.tiledb.com/mikejiang/32/102_2.png) [@mikejiang](https://forum.tiledb.com/u/mikejiang)\ **Post date:** [March 30, 2021, 7:22pm UTC](https://forum.tiledb.com/t/concurrent-read-tile-arrays-from-s3/322/3 "2021-03-30T19:22:12Z")
> parallelize over attributes > what does it mean?So currently tiledb only parallel over different arrays but not concurrent read different tiles within the same array? ---
**Author:** ![stavros](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@stavros](https://forum.tiledb.com/u/stavros)\ **Post date:** [March 30, 2021, 7:32pm UTC](https://forum.tiledb.com/t/concurrent-read-tile-arrays-from-s3/322/4 "2021-03-30T19:32:06Z")
Apologies, I’ve misread your comment. You currently seem to have one attribute (`mat`). I think you might be having just a single tile and a single attribute in the array (and no filters per attribute), therefore TileDB can’t seem to find anything to parallelize on :). Typically TileDB parallelizes over attributes, over tiles per attribute and over chunks per tile when there are filters. Can you define a tile extent per dimension which does not cover the whole dimension domain? In other words, use a last argument in `add_dimension` that is smaller than `ncell` or `nch`. Also to be sure, can you please [enable and dump the stats](https://docs.tiledb.com/main/solutions/tiledb-embedded/api-usage/using-performance-statistics) for your reads? Thanks! ---
**Author:** ![mikejiang](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.tiledb.com/mikejiang/32/102_2.png) [@mikejiang](https://forum.tiledb.com/u/mikejiang)\ **Post date:** [April 3, 2021, 1:15am UTC](https://forum.tiledb.com/t/concurrent-read-tile-arrays-from-s3/322/5 "2021-04-03T01:15:52Z")
No. I do have multiple tiles for ‘mat’ attributes. As you see above, for `cell` dims(i.e. row) I defined the extent as ncell, which make all cells as one tile, for `channel` dim (i.e. col), I defined the tile extent as 1, so basically our access pattern is fetching 1 or more columns(channels) each time. With this 2d layout, I expect tiledb will perform concurrent read upon multi-columns IO requests. But based on my testing, using 1 thread or 17 threads doesn’t make any time difference when I reading 17-column data from s3. (each column is about 4.5MB). ---
**Author:** ![stavros](https://avatars.discourse-cdn.com/v4/letter/s/51bf81/32.png) [@stavros](https://forum.tiledb.com/u/stavros)\ **Post date:** [April 5, 2021, 4:16pm UTC](https://forum.tiledb.com/t/concurrent-read-tile-arrays-from-s3/322/6 "2021-04-05T16:16:59Z")
TileDB should be parallelizing across tiles as long as `nch > 1`. Could you please enable and dump the stats as I suggest above for your reads in the case of `nthreads = 1` and `nthreads = 17` (assigning appropriately your concurrency config options, and setting `cfg` to the context in the very beginning of your program)? We should be able to see what’s happening from the stats.