[TileDB::Writer] Error: Write failed; Coordinate (1, 42049031, A) comes before last written coordinate in the global order

1

Hi all, I have VCF files obtained from imputation software (Gencove). I am indexing the files, creating a dataset and storing the VCFs as suggested in the HowTo:

for f in *.vcf.gz; do bcftools index -c $f; done

tiledbvcf create --uri my_vcf_dataset

tiledbvcf store --uri my_vcf_dataset *.vcf.gz

However, I am getting the following error:

 [critical] Exception: [TileDB::Writer] Error: Write failed; Coordinate (1, 42049031, A) comes before last written coordinate in the global order

Could you give me a suggestion of how to troubleshoot this issue? Thank you!

2

Hi @aheritas,

Thank you for sharing this issue. The error looks like it is related to an issue we have fixed but not release yet.

To confirm, can you please rerun the store command with debug messages turned on:

tiledbvcf store --uri my_vcf_dataset --log-level debug *.vcf.gz

and share the log messages that come before the [critical] Exception?

3

Dear @George_Powley, thank you for the reply. I tried pasting all the log messages in this reply but I exceed allowed characters, so I am pasting here the first part of all the log messages and in the next reply I will paste the rest. Let me know if there is anything else you need. Many thanks.

Log messages (1/3) 
[2022-07-22 18:33:39.717] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [info] Version:
TileDB-VCF version 0.17.0
TileDB version 2.10.1
htslib version 1.15.1
[2022-07-22 18:33:39.717] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [info] Command options:
uri="my_vcf_dataset"
threads=16
total-memory-budget-mb=48248
total-memory-percentage=0
resume=false
sample-batch-size=10
samples-file=""
paths=["6147765_impute-vcf.vcf.gz", "GRPO10091961_impute-vcf.vcf.gz", "LABL26121975_impute-vcf.vcf.gz", "LAVE25011958_impute-vcf.vcf.gz", "LOAC16031984_impute-vcf.vcf.gz", "LUSE03081974_impute-vcf.vcf.gz", "NOBR12101949_impute-vcf.vcf.gz", "RASA31011956_impute-vcf.vcf.gz"]
remove-sample-file=false
scratch-dir=""
scratch-mb=0
s3-part-size=50
tiledb-config="[]"
stats=false
stats-vcf-header-array=false
ratio-tiledb-memory=0.5
max-tiledb-memory-mb=4096
input-record-buffer-mb=1
avg-vcf-record-size=512
ratio-task-size=0.75
ratio-output-flush=0.75
disable-contig-fragment-merging=false
contigs-to-keep-separate=""
contigs-to-allow-merging="[]"
contig-mode="all"
log-level="debug"
verbose=false

[2022-07-22 18:33:39.717] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Open file limit = 1024
[2022-07-22 18:33:39.717] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] New open file limit = 1048576
[2022-07-22 18:33:39.717] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [info] Memory budget: total=48248 MiB = tiledb=4096 MiB + input=160 MiB + output=21996 MiB + stats=21996 MiB
[2022-07-22 18:33:39.717] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [info] Input buffers = 16 threads * 10 samples * 1 MiB
[2022-07-22 18:33:39.717] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [info] Output buffers = 16 threads * 1374 MiB (flush = 1031 MiB)
[2022-07-22 18:33:39.723] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Lookup member name='data' in group uri='my_vcf_dataset'
[2022-07-22 18:33:39.723] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Found group member name='data' uri='file:///data/tiledbvcf/my_vcf_dataset/data'
[2022-07-22 18:33:39.723] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Open 'data' using group uri 'file:///data/tiledbvcf/my_vcf_dataset/data'
[2022-07-22 18:33:39.724] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Lookup member name='vcf_headers' in group uri='my_vcf_dataset'
[2022-07-22 18:33:39.724] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Found group member name='vcf_headers' uri='file:///data/tiledbvcf/my_vcf_dataset/metadata/vcf_headers'
[2022-07-22 18:33:39.724] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Open 'vcf_headers' using group uri 'file:///data/tiledbvcf/my_vcf_dataset/metadata/vcf_headers'
[2022-07-22 18:33:39.727] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: Open array
[2022-07-22 18:33:40.114] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Initialization completed in 0.396799391 seconds.
[2022-07-22 18:33:40.114] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:40.202] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:40.299] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:40.390] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:40.482] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:40.573] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:40.664] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:40.755] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:40.846] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:40.937] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:41.029] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:41.120] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:41.211] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:41.302] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:41.394] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:41.485] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:41.577] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:41.686] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:41.778] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:41.869] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:41.961] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:42.052] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:42.144] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:42.235] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:42.327] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:42.418] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:42.509] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:42.600] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:42.692] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:42.783] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:42.875] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:42.966] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:43.057] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:43.148] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:43.240] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:43.334] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:43.425] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:43.517] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:43.611] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:43.702] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:43.794] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:43.885] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:43.977] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:44.069] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:44.160] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:44.252] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:44.344] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:44.438] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:44.529] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:44.621] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:44.712] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:44.803] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:44.895] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:44.987] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:45.078] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:45.170] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:45.262] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:45.354] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:45.445] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:45.537] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:45.629] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:45.720] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:45.812] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:45.904] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:45.995] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:46.087] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:46.179] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:46.270] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:46.362] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:46.454] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:46.545] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:46.636] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:46.728] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:46.819] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:46.911] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:47.002] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:47.099] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:47.190] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:47.282] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:47.373] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:47.464] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:47.555] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:47.647] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:47.738] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:47.829] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:47.920] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:48.012] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:48.103] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:48.194] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:48.285] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:48.377] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:48.468] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:48.559] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:48.651] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:48.742] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:48.833] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:48.924] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:49.016] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:49.107] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:49.198] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:49.290] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:49.381] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:49.472] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:49.563] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:49.654] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:49.744] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:49.835] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:49.926] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:50.017] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:50.108] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:50.198] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:50.292] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:50.383] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:50.474] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:50.564] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:50.656] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:50.746] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:50.837] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:50.928] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:51.019] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:51.115] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:51.206] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:51.297] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:51.388] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:51.479] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:51.570] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:51.661] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:51.751] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:51.842] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:51.933] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:52.023] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:52.114] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:52.205] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:52.296] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:52.387] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:52.477] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:52.568] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header 6147765_impute-vcf.vcf.gz
[2022-07-22 18:33:52.713] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header GRPO10091961_impute-vcf.vcf.gz
[2022-07-22 18:33:52.851] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LABL26121975_impute-vcf.vcf.gz
[2022-07-22 18:33:52.990] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LAVE25011958_impute-vcf.vcf.gz
[2022-07-22 18:33:53.128] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LOAC16031984_impute-vcf.vcf.gz
[2022-07-22 18:33:53.267] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header LUSE03081974_impute-vcf.vcf.gz
[2022-07-22 18:33:53.405] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header NOBR12101949_impute-vcf.vcf.gz
[2022-07-22 18:33:53.543] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Reading VCF header RASA31011956_impute-vcf.vcf.gz
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Output buffer records = 2111488
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 1: task size = 130.788.872 tasks = 33
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 2: task size = 119.446.872 tasks = 36
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 3: task size = 145.016.400 tasks = 30
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 4: task size = 147.506.864 tasks = 30
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 5: task size = 160.594.432 tasks = 27
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 6: task size = 168.296.624 tasks = 26
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 7: task size = 179.302.336 tasks = 24
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 8: task size = 183.918.432 tasks = 24
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 9: task size = 237.487.552 tasks = 19
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 10: task size = 211.831.024 tasks = 21
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 11: task size = 209.035.760 tasks = 21
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 12: task size = 218.604.416 tasks = 20
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 13: task size = 295.924.224 tasks = 15
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 14: task size = 318.502.432 tasks = 14
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 15: task size = 348.794.624 tasks = 13
[2022-07-22 18:33:53.682] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 16: task size = 313.272.256 tasks = 14
[2022-07-22 18:33:53.683] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 17: task size = 363.047.040 tasks = 12
[2022-07-22 18:33:53.683] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 18: task size = 373.015.808 tasks = 12
[2022-07-22 18:33:53.683] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 19: task size = 461.240.320 tasks = 10
[2022-07-22 18:33:53.683] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 20: task size = 466.560.256 tasks = 10
[2022-07-22 18:33:53.683] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 21: task size = 764.409.216 tasks = 6
[2022-07-22 18:33:53.683] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig 22: task size = 765.780.096 tasks = 6
4
Log messages (2/3) 
[2022-07-22 18:33:53.683] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Contig X: task size = 243.517.728 tasks = 18
[2022-07-22 18:33:53.770] [tiledb-vcf] [Process: 2648005] [Thread: 2648074] [debug] WriteWorker4: parse 1:0-130788871
[2022-07-22 18:33:53.770] [tiledb-vcf] [Process: 2648005] [Thread: 2648075] [debug] WriteWorker4: parse 1:130788872-261577743
[2022-07-22 18:33:53.770] [tiledb-vcf] [Process: 2648005] [Thread: 2648076] [debug] WriteWorker4: parse 1:261577744-392366615
[2022-07-22 18:33:53.770] [tiledb-vcf] [Process: 2648005] [Thread: 2648077] [debug] WriteWorker4: parse 1:392366616-523155487
[2022-07-22 18:33:53.770] [tiledb-vcf] [Process: 2648005] [Thread: 2648078] [debug] WriteWorker4: parse 1:523155488-653944359
[2022-07-22 18:33:53.770] [tiledb-vcf] [Process: 2648005] [Thread: 2648080] [debug] WriteWorker4: parse 1:784733232-915522103
[2022-07-22 18:33:53.770] [tiledb-vcf] [Process: 2648005] [Thread: 2648079] [debug] WriteWorker4: parse 1:653944360-784733231
[2022-07-22 18:33:53.770] [tiledb-vcf] [Process: 2648005] [Thread: 2648081] [debug] WriteWorker4: parse 1:915522104-1046310975
[2022-07-22 18:33:53.771] [tiledb-vcf] [Process: 2648005] [Thread: 2648082] [debug] WriteWorker4: parse 1:1046310976-1177099847
[2022-07-22 18:33:53.771] [tiledb-vcf] [Process: 2648005] [Thread: 2648086] [debug] WriteWorker4: parse 1:1569466464-1700255335
[2022-07-22 18:33:53.771] [tiledb-vcf] [Process: 2648005] [Thread: 2648084] [debug] WriteWorker4: parse 1:1307888720-1438677591
[2022-07-22 18:33:53.771] [tiledb-vcf] [Process: 2648005] [Thread: 2648085] [debug] WriteWorker4: parse 1:1438677592-1569466463
[2022-07-22 18:33:53.771] [tiledb-vcf] [Process: 2648005] [Thread: 2648083] [debug] WriteWorker4: parse 1:1177099848-1307888719
[2022-07-22 18:33:53.772] [tiledb-vcf] [Process: 2648005] [Thread: 2648087] [debug] WriteWorker4: parse 1:1700255336-1831044207
[2022-07-22 18:33:53.772] [tiledb-vcf] [Process: 2648005] [Thread: 2648088] [debug] WriteWorker4: parse 1:1831044208-1961833079
[2022-07-22 18:33:53.772] [tiledb-vcf] [Process: 2648005] [Thread: 2648089] [debug] WriteWorker4: parse 1:1961833080-2092621951
[2022-07-22 18:33:54.189] [tiledb-vcf] [Process: 2648005] [Thread: 2648075] [debug] VariantStats: new contig = 1
[2022-07-22 18:33:54.219] [tiledb-vcf] [Process: 2648005] [Thread: 2648074] [debug] VariantStats: new contig = 1
[2022-07-22 18:34:24.850] [tiledb-vcf] [Process: 2648005] [Thread: 2648075] [debug] Worker 1: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:34:24.906] [tiledb-vcf] [Process: 2648005] [Thread: 2648074] [debug] Worker 0: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:34:26.615] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4138453 cells from 1:10176-16778461 (task 1 / 16)
[2022-07-22 18:34:26.615] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 566121 records from 1:10176-16778461
[2022-07-22 18:34:26.693] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 0 not complete, resuming
[2022-07-22 18:34:57.376] [tiledb-vcf] [Process: 2648005] [Thread: 2648112] [debug] Worker 0: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:34:59.033] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4141002 cells from 1:16778461-34531934 (task 1 / 16)
[2022-07-22 18:34:59.033] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 565401 records from 1:16778461-34531934
[2022-07-22 18:34:59.110] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 0 not complete, resuming
[2022-07-22 18:35:29.875] [tiledb-vcf] [Process: 2648005] [Thread: 2648114] [debug] Worker 0: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:35:31.501] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4143656 cells from 1:34531934-53202740 (task 1 / 16)
[2022-07-22 18:35:31.501] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 562011 records from 1:34531934-53202740
[2022-07-22 18:35:31.599] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 0 not complete, resuming
[2022-07-22 18:36:02.498] [tiledb-vcf] [Process: 2648005] [Thread: 2648122] [debug] Worker 0: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:36:04.113] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4146812 cells from 1:53202740-71176771 (task 1 / 16)
[2022-07-22 18:36:04.113] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 570240 records from 1:53202740-71176771
[2022-07-22 18:36:04.200] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 0 not complete, resuming
[2022-07-22 18:36:35.054] [tiledb-vcf] [Process: 2648005] [Thread: 2648126] [debug] Worker 0: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:36:36.616] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4155776 cells from 1:71176771-89368239 (task 1 / 16)
[2022-07-22 18:36:36.616] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 571080 records from 1:71176771-89368239
[2022-07-22 18:36:36.719] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 0 not complete, resuming
[2022-07-22 18:37:07.619] [tiledb-vcf] [Process: 2648005] [Thread: 2648130] [debug] Worker 0: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:37:09.168] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4156028 cells from 1:89368239-107527833 (task 1 / 16)
[2022-07-22 18:37:09.168] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 573320 records from 1:89368239-107527833
[2022-07-22 18:37:09.256] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 0 not complete, resuming
[2022-07-22 18:37:33.590] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 3082825 cells from 1:107527833-121485142 (task 1 / 16)
[2022-07-22 18:37:33.590] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 424007 records from 1:107527833-121485142
[2022-07-22 18:37:33.650] [tiledb-vcf] [Process: 2648005] [Thread: 2648135] [debug] WriteWorker4: parse 1:2092621952-2223410823
[2022-07-22 18:37:35.112] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4139994 cells from 1:142535438-164009045 (task 2 / 16)
[2022-07-22 18:37:35.112] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 566395 records from 1:142535438-164009045
[2022-07-22 18:37:35.184] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 1 not complete, resuming
[2022-07-22 18:38:05.794] [tiledb-vcf] [Process: 2648005] [Thread: 2648136] [debug] Worker 1: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:38:07.328] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4148424 cells from 1:164009045-182077114 (task 2 / 16)
[2022-07-22 18:38:07.328] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 571354 records from 1:164009045-182077114
[2022-07-22 18:38:07.412] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 1 not complete, resuming
[2022-07-22 18:38:38.104] [tiledb-vcf] [Process: 2648005] [Thread: 2648140] [debug] Worker 1: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:38:39.529] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4159834 cells from 1:182077114-199948249 (task 2 / 16)
[2022-07-22 18:38:39.529] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 570154 records from 1:182077114-199948249
[2022-07-22 18:38:39.607] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 1 not complete, resuming
[2022-07-22 18:39:10.461] [tiledb-vcf] [Process: 2648005] [Thread: 2648142] [debug] Worker 1: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:39:11.923] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4146518 cells from 1:199948249-218111666 (task 2 / 16)
[2022-07-22 18:39:11.923] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 569723 records from 1:199948249-218111666
[2022-07-22 18:39:12.001] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 1 not complete, resuming
[2022-07-22 18:39:42.985] [tiledb-vcf] [Process: 2648005] [Thread: 2648145] [debug] Worker 1: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:39:44.442] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4147003 cells from 1:218111666-235914685 (task 2 / 16)
[2022-07-22 18:39:44.442] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 571253 records from 1:218111666-235914685
[2022-07-22 18:39:44.527] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 1 not complete, resuming
[2022-07-22 18:40:10.301] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 3297939 cells from 1:235914685-249240542 (task 2 / 16)
[2022-07-22 18:40:10.301] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 461512 records from 1:235914685-249240542
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648151] [debug] WriteWorker4: parse 1:2223410824-2354199695
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648152] [debug] WriteWorker4: parse 1:2354199696-2484988567
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648153] [debug] WriteWorker4: parse 1:2484988568-2615777439
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648154] [debug] WriteWorker4: parse 1:2615777440-2746566311
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648155] [debug] WriteWorker4: parse 1:2746566312-2877355183
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648156] [debug] WriteWorker4: parse 1:2877355184-3008144055
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648157] [debug] WriteWorker4: parse 1:3008144056-3138932927
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648159] [debug] WriteWorker4: parse 1:3269721800-3400510671
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648158] [debug] WriteWorker4: parse 1:3138932928-3269721799
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648161] [debug] WriteWorker4: parse 1:3531299544-3662088415
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648163] [debug] WriteWorker4: parse 1:3792877288-3923666159
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648160] [debug] WriteWorker4: parse 1:3400510672-3531299543
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648164] [debug] WriteWorker4: parse 1:3923666160-4054455031
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [info] Ingestion rate = 138090.471 records/sec
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648162] [debug] WriteWorker4: parse 1:3662088416-3792877287
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648165] [debug] WriteWorker4: parse 1:4054455032-4185243903
[2022-07-22 18:40:10.367] [tiledb-vcf] [Process: 2648005] [Thread: 2648166] [debug] WriteWorker4: parse 1:4185243904-21065479
[2022-07-22 18:40:10.485] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.485] [tiledb-vcf] [Process: 2648005] [Thread: 2648167] [debug] WriteWorker4: parse 10:0-211831023
[2022-07-22 18:40:10.493] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648168] [debug] WriteWorker4: parse 10:211831024-423662047
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648169] [debug] WriteWorker4: parse 10:423662048-635493071
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648170] [debug] WriteWorker4: parse 10:635493072-847324095
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648171] [debug] WriteWorker4: parse 10:847324096-1059155119
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648172] [debug] WriteWorker4: parse 10:1059155120-1270986143
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648173] [debug] WriteWorker4: parse 10:1270986144-1482817167
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648175] [debug] WriteWorker4: parse 10:1694648192-1906479215
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648176] [debug] WriteWorker4: parse 10:1906479216-2118310239
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648174] [debug] WriteWorker4: parse 10:1482817168-1694648191
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648177] [debug] WriteWorker4: parse 10:2118310240-2330141263
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648179] [debug] WriteWorker4: parse 10:2541972288-2753803311
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648178] [debug] WriteWorker4: parse 10:2330141264-2541972287
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648180] [debug] WriteWorker4: parse 10:2753803312-2965634335
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 1
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648182] [debug] WriteWorker4: parse 10:3177465360-3389296383
[2022-07-22 18:40:10.494] [tiledb-vcf] [Process: 2648005] [Thread: 2648181] [debug] WriteWorker4: parse 10:2965634336-3177465359
[2022-07-22 18:40:11.226] [tiledb-vcf] [Process: 2648005] [Thread: 2648167] [debug] VariantStats: new contig = 10
[2022-07-22 18:40:42.554] [tiledb-vcf] [Process: 2648005] [Thread: 2648167] [debug] Worker 1: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:40:42.554] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Previous contig 1 found to NOT be mergeable, finalizing previous fragment and starting new write for 10
[2022-07-22 18:40:42.554] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [info] Finalizing contig batch [1, 1]
[2022-07-22 18:40:42.555] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: Finalize query with 7142571 records
[2022-07-22 18:40:43.185] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: fragment_num = 1 uri = file:///data/tiledbvcf/my_vcf_dataset/variant_stats/__fragments/__1658525666621_1658525666621_ca7b09d2d02c4591b474d20807603091_14 samples = 6147765,GRPO10091961,LABL26121975,LAVE25011958,LOAC16031984,LUSE03081974,NOBR12101949,RASA31011956
[2022-07-22 18:40:44.815] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4132417 cells from 10:60493-15000553 (task 2 / 16)
[2022-07-22 18:40:44.815] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 576040 records from 10:60493-15000553
[2022-07-22 18:40:44.905] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 1 not complete, resuming
[2022-07-22 18:41:16.363] [tiledb-vcf] [Process: 2648005] [Thread: 2648185] [debug] Worker 1: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:41:17.786] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4138137 cells from 10:15000553-31414112 (task 2 / 16)
[2022-07-22 18:41:17.786] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 571946 records from 10:15000553-31414112
[2022-07-22 18:41:17.877] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 1 not complete, resuming
[2022-07-22 18:41:49.369] [tiledb-vcf] [Process: 2648005] [Thread: 2648190] [debug] Worker 1: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:41:50.855] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4128107 cells from 10:31414112-53326567 (task 2 / 16)
[2022-07-22 18:41:50.855] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 567428 records from 10:31414112-53326567
[2022-07-22 18:41:50.939] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 1 not complete, resuming
[2022-07-22 18:42:22.481] [tiledb-vcf] [Process: 2648005] [Thread: 2648191] [debug] Worker 1: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:42:24.026] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4142711 cells from 10:53326567-70482748 (task 2 / 16)
[2022-07-22 18:42:24.026] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 576659 records from 10:53326567-70482748
[2022-07-22 18:42:24.133] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 1 not complete, resuming
[2022-07-22 18:42:55.691] [tiledb-vcf] [Process: 2648005] [Thread: 2648195] [debug] Worker 1: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:42:57.141] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4129432 cells from 10:70482748-87806725 (task 2 / 16)
[2022-07-22 18:42:57.142] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 566004 records from 10:70482748-87806725
[2022-07-22 18:42:57.218] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 1 not complete, resuming
[2022-07-22 18:43:28.686] [tiledb-vcf] [Process: 2648005] [Thread: 2648197] [debug] Worker 1: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:43:30.204] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4132075 cells from 10:87806725-106491565 (task 2 / 16)
[2022-07-22 18:43:30.204] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 567278 records from 10:87806725-106491565
[2022-07-22 18:43:30.301] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 1 not complete, resuming
[2022-07-22 18:44:01.739] [tiledb-vcf] [Process: 2648005] [Thread: 2648201] [debug] Worker 1: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:44:03.253] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4132075 cells from 10:106491565-123453522 (task 2 / 16)
[2022-07-22 18:44:03.253] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 568638 records from 10:106491565-123453522
[2022-07-22 18:44:03.333] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 1 not complete, resuming
[2022-07-22 18:44:28.718] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 3173558 cells from 10:123453522-135524372 (task 2 / 16)
[2022-07-22 18:44:28.718] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 442653 records from 10:123453522-135524372
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648206] [debug] WriteWorker4: parse 10:3389296384-3601127407
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648207] [debug] WriteWorker4: parse 10:3601127408-3812958431
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648208] [debug] WriteWorker4: parse 10:3812958432-4024789455
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648209] [debug] WriteWorker4: parse 10:4024789456-4236620479
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648210] [debug] WriteWorker4: parse 10:4236620480-153484207
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648211] [debug] WriteWorker4: parse 11:0-209035759
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648212] [debug] WriteWorker4: parse 11:209035760-418071519
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648214] [debug] WriteWorker4: parse 11:627107280-836143039
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648215] [debug] WriteWorker4: parse 11:836143040-1045178799
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648217] [debug] WriteWorker4: parse 11:1254214560-1463250319
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648218] [debug] WriteWorker4: parse 11:1463250320-1672286079
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648213] [debug] WriteWorker4: parse 11:418071520-627107279
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648219] [debug] WriteWorker4: parse 11:1672286080-1881321839
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648216] [debug] WriteWorker4: parse 11:1045178800-1254214559
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [info] Ingestion rate = 124313.259 records/sec
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.781] [tiledb-vcf] [Process: 2648005] [Thread: 2648220] [debug] WriteWorker4: parse 11:1881321840-2090357599
[2022-07-22 18:44:28.797] [tiledb-vcf] [Process: 2648005] [Thread: 2648221] [debug] WriteWorker4: parse 11:2090357600-2299393359
[2022-07-22 18:44:28.856] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.856] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.856] [tiledb-vcf] [Process: 2648005] [Thread: 2648222] [debug] WriteWorker4: parse 11:2299393360-2508429119
[2022-07-22 18:44:28.856] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.856] [tiledb-vcf] [Process: 2648005] [Thread: 2648223] [debug] WriteWorker4: parse 11:2508429120-2717464879
[2022-07-22 18:44:28.856] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.856] [tiledb-vcf] [Process: 2648005] [Thread: 2648224] [debug] WriteWorker4: parse 11:2717464880-2926500639
[2022-07-22 18:44:28.856] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 10
[2022-07-22 18:44:28.856] [tiledb-vcf] [Process: 2648005] [Thread: 2648226] [debug] WriteWorker4: parse 11:3135536400-3344572159
[2022-07-22 18:44:28.856] [tiledb-vcf] [Process: 2648005] [Thread: 2648225] [debug] WriteWorker4: parse 11:2926500640-3135536399
[2022-07-22 18:44:29.536] [tiledb-vcf] [Process: 2648005] [Thread: 2648211] [debug] VariantStats: new contig = 11
[2022-07-22 18:45:00.535] [tiledb-vcf] [Process: 2648005] [Thread: 2648211] [debug] Worker 6: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:45:00.535] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Previous contig 10 found to NOT be mergeable, finalizing previous fragment and starting new write for 11
[2022-07-22 18:45:00.535] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [info] Finalizing contig batch [10, 10]
[2022-07-22 18:45:00.535] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: Finalize query with 4436646 records
[2022-07-22 18:45:01.442] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: fragment_num = 2 uri = file:///data/tiledbvcf/my_vcf_dataset/variant_stats/__fragments/__1658526044820_1658526044820_82a54652b4b94784bb6e2019fe9f4be3_14 samples = 6147765,GRPO10091961,LABL26121975,LAVE25011958,LOAC16031984,LUSE03081974,NOBR12101949,RASA31011956
[2022-07-22 18:45:02.964] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4126708 cells from 11:61394-15965420 (task 7 / 16)
[2022-07-22 18:45:02.964] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 575664 records from 11:61394-15965420
[2022-07-22 18:45:03.045] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 6 not complete, resuming
[2022-07-22 18:45:34.152] [tiledb-vcf] [Process: 2648005] [Thread: 2648232] [debug] Worker 6: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:45:35.697] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4136649 cells from 11:15965420-33137924 (task 7 / 16)
[2022-07-22 18:45:35.697] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 574249 records from 11:15965420-33137924
[2022-07-22 18:45:35.773] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 6 not complete, resuming
[2022-07-22 18:46:06.954] [tiledb-vcf] [Process: 2648005] [Thread: 2648235] [debug] Worker 6: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:46:08.459] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4134050 cells from 11:33137924-49973273 (task 7 / 16)
[2022-07-22 18:46:08.459] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 568329 records from 11:33137924-49973273
[2022-07-22 18:46:08.557] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 6 not complete, resuming
[2022-07-22 18:46:39.913] [tiledb-vcf] [Process: 2648005] [Thread: 2648239] [debug] Worker 6: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:46:41.374] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4131084 cells from 11:49973273-69355115 (task 7 / 16)
[2022-07-22 18:46:41.374] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 561575 records from 11:49973273-69355115
[2022-07-22 18:46:41.463] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 6 not complete, resuming
[2022-07-22 18:47:12.855] [tiledb-vcf] [Process: 2648005] [Thread: 2648241] [debug] Worker 6: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:47:14.364] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4128104 cells from 11:69355115-86266514 (task 7 / 16)
[2022-07-22 18:47:14.364] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 562436 records from 11:69355115-86266514
5
Last part log messages (3/3) 
[2022-07-22 18:47:14.447] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 6 not complete, resuming
[2022-07-22 18:47:45.933] [tiledb-vcf] [Process: 2648005] [Thread: 2648245] [debug] Worker 6: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:47:47.397] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4139358 cells from 11:86266514-103113711 (task 7 / 16)
[2022-07-22 18:47:47.397] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 571958 records from 11:86266514-103113711
[2022-07-22 18:47:47.488] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 6 not complete, resuming
[2022-07-22 18:48:18.986] [tiledb-vcf] [Process: 2648005] [Thread: 2648247] [debug] Worker 6: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:48:20.591] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4131792 cells from 11:103113711-120585310 (task 7 / 16)
[2022-07-22 18:48:20.591] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 566648 records from 11:103113711-120585310
[2022-07-22 18:48:20.676] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 6 not complete, resuming
[2022-07-22 18:48:49.518] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 3610127 cells from 11:120585310-134946452 (task 7 / 16)
[2022-07-22 18:48:49.518] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 495888 records from 11:120585310-134946452
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648253] [debug] WriteWorker4: parse 11:3344572160-3553607919
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648254] [debug] WriteWorker4: parse 11:3553607920-3762643679
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648255] [debug] WriteWorker4: parse 11:3762643680-3971679439
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648256] [debug] WriteWorker4: parse 11:3971679440-4180715199
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648257] [debug] WriteWorker4: parse 11:4180715200-94783663
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648258] [debug] WriteWorker4: parse 12:0-218604415
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648259] [debug] WriteWorker4: parse 12:218604416-437208831
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648260] [debug] WriteWorker4: parse 12:437208832-655813247
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [info] Ingestion rate = 124758.472 records/sec
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648262] [debug] WriteWorker4: parse 12:874417664-1093022079
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648263] [debug] WriteWorker4: parse 12:1093022080-1311626495
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648264] [debug] WriteWorker4: parse 12:1311626496-1530230911
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648266] [debug] WriteWorker4: parse 12:1748835328-1967439743
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648267] [debug] WriteWorker4: parse 12:1967439744-2186044159
[2022-07-22 18:48:49.588] [tiledb-vcf] [Process: 2648005] [Thread: 2648261] [debug] WriteWorker4: parse 12:655813248-874417663
[2022-07-22 18:48:49.589] [tiledb-vcf] [Process: 2648005] [Thread: 2648268] [debug] WriteWorker4: parse 12:2186044160-2404648575
[2022-07-22 18:48:49.589] [tiledb-vcf] [Process: 2648005] [Thread: 2648265] [debug] WriteWorker4: parse 12:1530230912-1748835327
[2022-07-22 18:48:49.667] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.667] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.667] [tiledb-vcf] [Process: 2648005] [Thread: 2648269] [debug] WriteWorker4: parse 12:2404648576-2623252991
[2022-07-22 18:48:49.667] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.667] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.667] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 11
[2022-07-22 18:48:49.667] [tiledb-vcf] [Process: 2648005] [Thread: 2648272] [debug] WriteWorker4: parse 12:3060461824-3279066239
[2022-07-22 18:48:49.667] [tiledb-vcf] [Process: 2648005] [Thread: 2648270] [debug] WriteWorker4: parse 12:2623252992-2841857407
[2022-07-22 18:48:49.667] [tiledb-vcf] [Process: 2648005] [Thread: 2648271] [debug] WriteWorker4: parse 12:2841857408-3060461823
[2022-07-22 18:48:49.667] [tiledb-vcf] [Process: 2648005] [Thread: 2648273] [debug] WriteWorker4: parse 12:3279066240-3497670655
[2022-07-22 18:48:50.357] [tiledb-vcf] [Process: 2648005] [Thread: 2648258] [debug] VariantStats: new contig = 12
[2022-07-22 18:49:21.401] [tiledb-vcf] [Process: 2648005] [Thread: 2648258] [debug] Worker 11: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:49:21.401] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Previous contig 11 found to NOT be mergeable, finalizing previous fragment and starting new write for 12
[2022-07-22 18:49:21.401] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [info] Finalizing contig batch [11, 11]
[2022-07-22 18:49:21.401] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: Finalize query with 4476747 records
[2022-07-22 18:49:21.999] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: fragment_num = 3 uri = file:///data/tiledbvcf/my_vcf_dataset/variant_stats/__fragments/__1658526302968_1658526302968_899b782998654ee5bd886e536aecc7f1_14 samples = 6147765,GRPO10091961,LABL26121975,LAVE25011958,LOAC16031984,LUSE03081974,NOBR12101949,RASA31011956
[2022-07-22 18:49:23.659] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4132417 cells from 12:60180-17021919 (task 12 / 16)
[2022-07-22 18:49:23.659] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 572847 records from 12:60180-17021919
[2022-07-22 18:49:23.774] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 11 not complete, resuming
[2022-07-22 18:49:54.839] [tiledb-vcf] [Process: 2648005] [Thread: 2648278] [debug] Worker 11: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:49:56.321] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4140908 cells from 12:17021919-34234768 (task 12 / 16)
[2022-07-22 18:49:56.321] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 579209 records from 12:17021919-34234768
[2022-07-22 18:49:56.419] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 11 not complete, resuming
[2022-07-22 18:50:27.551] [tiledb-vcf] [Process: 2648005] [Thread: 2648279] [debug] Worker 11: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:50:29.080] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4135453 cells from 12:34234768-53824915 (task 12 / 16)
[2022-07-22 18:50:29.080] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 564912 records from 12:34234768-53824915
[2022-07-22 18:50:29.155] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 11 not complete, resuming
[2022-07-22 18:51:00.496] [tiledb-vcf] [Process: 2648005] [Thread: 2648284] [debug] Worker 11: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:51:01.987] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4137410 cells from 12:53824915-71952518 (task 12 / 16)
[2022-07-22 18:51:01.987] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 569712 records from 12:53824915-71952518
[2022-07-22 18:51:02.088] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 11 not complete, resuming
[2022-07-22 18:51:33.332] [tiledb-vcf] [Process: 2648005] [Thread: 2648285] [debug] Worker 11: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:51:34.826] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4142820 cells from 12:71952518-90091916 (task 12 / 16)
[2022-07-22 18:51:34.826] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 568771 records from 12:71952518-90091916
[2022-07-22 18:51:34.906] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 11 not complete, resuming
[2022-07-22 18:52:06.364] [tiledb-vcf] [Process: 2648005] [Thread: 2648289] [debug] Worker 11: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:52:07.894] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4136039 cells from 12:90091935-108181266 (task 12 / 16)
[2022-07-22 18:52:07.894] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 568244 records from 12:90091935-108181266
[2022-07-22 18:52:07.972] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 11 not complete, resuming
[2022-07-22 18:52:39.432] [tiledb-vcf] [Process: 2648005] [Thread: 2648292] [debug] Worker 11: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:52:40.948] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4130784 cells from 12:108181266-125820508 (task 12 / 16)
[2022-07-22 18:52:40.948] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 565521 records from 12:108181266-125820508
[2022-07-22 18:52:41.027] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 11 not complete, resuming
[2022-07-22 18:52:58.277] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 2157809 cells from 12:125820508-133841814 (task 12 / 16)
[2022-07-22 18:52:58.277] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 301864 records from 12:125820508-133841814
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648297] [debug] WriteWorker4: parse 12:3497670656-3716275071
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648298] [debug] WriteWorker4: parse 12:3716275072-3934879487
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648300] [debug] WriteWorker4: parse 12:4153483904-77121023
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648299] [debug] WriteWorker4: parse 12:3934879488-4153483903
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [info] Ingestion rate = 125089.221 records/sec
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648301] [debug] WriteWorker4: parse 13:0-295924223
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648302] [debug] WriteWorker4: parse 13:295924224-591848447
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648303] [debug] WriteWorker4: parse 13:591848448-887772671
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648304] [debug] WriteWorker4: parse 13:887772672-1183696895
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648306] [debug] WriteWorker4: parse 13:1479621120-1775545343
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648307] [debug] WriteWorker4: parse 13:1775545344-2071469567
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648305] [debug] WriteWorker4: parse 13:1183696896-1479621119
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648308] [debug] WriteWorker4: parse 13:2071469568-2367393791
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648309] [debug] WriteWorker4: parse 13:2367393792-2663318015
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648310] [debug] WriteWorker4: parse 13:2663318016-2959242239
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.320] [tiledb-vcf] [Process: 2648005] [Thread: 2648311] [debug] WriteWorker4: parse 13:2959242240-3255166463
[2022-07-22 18:52:58.328] [tiledb-vcf] [Process: 2648005] [Thread: 2648312] [debug] WriteWorker4: parse 13:3255166464-3551090687
[2022-07-22 18:52:58.363] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.363] [tiledb-vcf] [Process: 2648005] [Thread: 2648313] [debug] WriteWorker4: parse 13:3551090688-3847014911
[2022-07-22 18:52:58.369] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.369] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.369] [tiledb-vcf] [Process: 2648005] [Thread: 2648314] [debug] WriteWorker4: parse 13:3847014912-4142939135
[2022-07-22 18:52:58.369] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] No records found for 12
[2022-07-22 18:52:58.370] [tiledb-vcf] [Process: 2648005] [Thread: 2648316] [debug] WriteWorker4: parse 14:0-318502431
[2022-07-22 18:52:58.369] [tiledb-vcf] [Process: 2648005] [Thread: 2648315] [debug] WriteWorker4: parse 13:4142939136-143896063
[2022-07-22 18:52:59.197] [tiledb-vcf] [Process: 2648005] [Thread: 2648316] [debug] VariantStats: new contig = 14
[2022-07-22 18:52:59.284] [tiledb-vcf] [Process: 2648005] [Thread: 2648301] [debug] VariantStats: new contig = 13
[2022-07-22 18:53:29.987] [tiledb-vcf] [Process: 2648005] [Thread: 2648316] [debug] Worker 14: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:53:30.304] [tiledb-vcf] [Process: 2648005] [Thread: 2648301] [debug] Worker 15: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:53:30.304] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Previous contig 12 found to NOT be mergeable, finalizing previous fragment and starting new write for 13
[2022-07-22 18:53:30.304] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [info] Finalizing contig batch [12, 12]
[2022-07-22 18:53:30.305] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: Finalize query with 4291080 records
[2022-07-22 18:53:30.418] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: fragment_num = 4 uri = file:///data/tiledbvcf/my_vcf_dataset/variant_stats/__fragments/__1658526563663_1658526563663_2ba4cb762b9a48499a2d094045cda396_14 samples = 6147765,GRPO10091961,LABL26121975,LAVE25011958,LOAC16031984,LUSE03081974,NOBR12101949,RASA31011956
[2022-07-22 18:53:32.023] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4136001 cells from 13:19020046-35470220 (task 16 / 16)
[2022-07-22 18:53:32.023] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 579506 records from 13:19020046-35470220
[2022-07-22 18:53:32.122] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 15 not complete, resuming
[2022-07-22 18:54:03.353] [tiledb-vcf] [Process: 2648005] [Thread: 2648324] [debug] Worker 15: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:54:04.876] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4136265 cells from 13:35470220-53775327 (task 16 / 16)
[2022-07-22 18:54:04.876] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 569719 records from 13:35470220-53775327
[2022-07-22 18:54:04.965] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 15 not complete, resuming
[2022-07-22 18:54:36.136] [tiledb-vcf] [Process: 2648005] [Thread: 2648326] [debug] Worker 15: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:54:37.618] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4148529 cells from 13:53775327-71571187 (task 16 / 16)
[2022-07-22 18:54:37.618] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 572977 records from 13:53775327-71571187
[2022-07-22 18:54:37.699] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Work for 15 not complete, resuming
[2022-07-22 18:55:09.153] [tiledb-vcf] [Process: 2648005] [Thread: 2648329] [debug] Worker 15: buffer full, output buffer size = 1032 MiB
[2022-07-22 18:55:10.614] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] Recorded 4144394 cells from 13:71571187-89259699 (task 16 / 16)
[2022-07-22 18:55:10.614] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: flushing 572182 records from 13:71571187-89259699
[2022-07-22 18:55:11.883] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [debug] VariantStats: Close array
[2022-07-22 18:55:12.560] [tiledb-vcf] [Process: 2648005] [Thread: 2648005] [critical] Exception: [TileDB::Writer] Error: Write failed; Coordinate (13, 71571187, C) comes before last written coordinate in the global order
6

Thank you for the log messages @aheritas. That is exactly what we needed and it confirms your issue should be solved in our next TileDB-VCF release.

A PR with the fix is here:

You can build TileDB-VCF from source with this fix today, if you like. We will release a version with the fix early next week.

7

@aheritas , one more idea to help you make progress in case you do not want to build TileDB-VCF from source.

The following steps will work around the issue. This will disable ingestion of the variant_stats array, which is an optional array and not required to work with VCF data.

# remove the old dataset
rm -rf my_vcf_dataset

# create a new dataset
tiledbvcf create --uri my_vcf_dataset

# remove the variant_stats array
rm -rf my_vcf_dataset/variant_stats

# store the VCFs
tiledbvcf store --uri my_vcf_dataset --log-level debug *.vcf.gz

This debug message will confirm the VariantStats ingestion is disabled:

[2022-07-22 19:51:28.616] [tiledb-vcf] [Process: 226379] [Thread: 226379] [debug] VariantStats: Variant stat ingestion disabled, because stat array not found in dataset.
8

Dear @George_Powley, tried the second approach and indeed it solved the issue. Thank you very much for your time and suggestions!!