# Runtime error in accessing s3 vcf data file

**URL:** <https://forum.tiledb.com/t/runtime-error-in-accessing-s3-vcf-data-file/288>\
**Category:** Uncategorized\
**Created:** [December 16, 2020, 12:18pm UTC](https://forum.tiledb.com/t/runtime-error-in-accessing-s3-vcf-data-file/288 "2020-12-16T12:18:00Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![Veerendar\_Konda](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.tiledb.com/veerendar_konda/32/113_2.png) [@Veerendar\_Konda](https://forum.tiledb.com/u/Veerendar_Konda)\
**Post date:** [December 16, 2020, 12:18pm UTC](https://forum.tiledb.com/t/runtime-error-in-accessing-s3-vcf-data-file/288/1 "2020-12-16T12:18:00Z")

</div>

Runtime error in accessing s3 vcf data file

Code:  
import tiledbvcf  
import boto3  
import os  
import tempfile  
import glob  
import pandas as pd

small\_ds = tiledbvcf.Dataset(‘small\_dataset2’, mode = “w”)

with open(“s3-vcf-samples.txt”) as f:  
sample\_uris = [l.rstrip("\n") for l in f.readlines()]

small\_ds.ingest\_samples(  
sample\_uris,  
scratch\_space\_path = tempfile.gettempdir(),  
scratch\_space\_size=10  
)

## Error message:

RuntimeError Traceback (most recent call last)  
 in ()  
4 sample\_uris,  
5 scratch\_space\_path = tempfile.gettempdir(),  
----\> 6 scratch\_space\_size=10  
7 )

/home/ec2-user/SageMaker/tileDB-new/tiledbvcf/lib/python3.7/site-packages/tiledbvcf/dataset.py in ingest\_samples(self, sample\_uris, extra\_attrs, checksum\_type, allow\_duplicates, scratch\_space\_path, scratch\_space\_size)  
212 # Create is a no-op if the dataset already exists.  
213 self.writer.create\_dataset()  
–\> 214 self.writer.register\_samples()  
215 self.writer.ingest\_samples()  
216

RuntimeError: TileDB-VCF exception: Error processing sample; URI ‘s3://bucket/data\_path/chr1-prefix.vcf.gz’ does not exist.

I tried exporting the access key and secret key as mentioned in the docs but still getting the same error.

Thanks.

---

<div class="post-metadata">

**Author:** ![aaron](https://avatars.discourse-cdn.com/v4/letter/a/e19adc/32.png) [@aaron](https://forum.tiledb.com/u/aaron)\
**Post date:** [December 16, 2020, 5:15pm UTC](https://forum.tiledb.com/t/runtime-error-in-accessing-s3-vcf-data-file/288/2 "2020-12-16T17:15:26Z")

</div>

Since you already have `boto3` imported could you try using that to verify you’re able to access the specified VCF file?

This should do it:

```auto
s3 = boto3.client('s3')
s3.list_objects_v2(Bucket = "bucket", Prefix = "data_path")

```

Alternatively you could use the aws-cli and attempt to list the bucket’s contents with  
`aws s3 ls `.

---

<div class="post-metadata">

**Author:** ![Veerendar\_Konda](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.tiledb.com/veerendar_konda/32/113_2.png) [@Veerendar\_Konda](https://forum.tiledb.com/u/Veerendar_Konda)\
**Post date:** [December 16, 2020, 5:33pm UTC](https://forum.tiledb.com/t/runtime-error-in-accessing-s3-vcf-data-file/288/3 "2020-12-16T17:33:36Z")

</div>

Yes, I’m able to access it using boto3.

---

<div class="post-metadata">

**Author:** ![aaron](https://avatars.discourse-cdn.com/v4/letter/a/e19adc/32.png) [@aaron](https://forum.tiledb.com/u/aaron)\
**Post date:** [December 16, 2020, 5:47pm UTC](https://forum.tiledb.com/t/runtime-error-in-accessing-s3-vcf-data-file/288/4 "2020-12-16T17:47:51Z")

</div>

Thanks for checking.

One other thought: is `sample_uris` a string with a single sample URI? Or is it a list of strings?  
If it’s the former, the string needs to be converted to a list, _ie_

```auto
small_ds.ingest_samples(
  [sample_uris],
  scratch_space_path = tempfile.gettempdir(),
  scratch_space_size=10
)

```

---

<div class="post-metadata">

**Author:** ![Veerendar\_Konda](https://yyz1.discourse-cdn.com/flex035/user_avatar/forum.tiledb.com/veerendar_konda/32/113_2.png) [@Veerendar\_Konda](https://forum.tiledb.com/u/Veerendar_Konda)\
**Post date:** [December 16, 2020, 6:18pm UTC](https://forum.tiledb.com/t/runtime-error-in-accessing-s3-vcf-data-file/288/5 "2020-12-16T18:18:04Z")

</div>

sample\_uris is a list of URIs.

---

<div class="post-metadata">

**Author:** ![aaron](https://avatars.discourse-cdn.com/v4/letter/a/e19adc/32.png) [@aaron](https://forum.tiledb.com/u/aaron)\
**Post date:** [December 16, 2020, 6:45pm UTC](https://forum.tiledb.com/t/runtime-error-in-accessing-s3-vcf-data-file/288/6 "2020-12-16T18:45:42Z")

</div>

Could you share what version of `tiledbvcf` you’re using (`tiledbvcf.version`) and how you installed it?
