Installation error

1

I have tried to install tileDB on EC2 instance by following these instructions.(https://docs.tiledb.com/genomics/installation/python)

I’m facing below error during python setup.py install step.

Makefile:83: recipe for target 'all' failedmake: *** [all] Error 2Traceback (most recent call last):File "setup.py", line 297, in <module>'Programming Language :: Python :: 3.7',File "/home/ubuntu/env/lib/python3.7/site-packages/setuptools/__init__.py", line 163, in setupreturn distutils.core.setup(**attrs)
File "/home/ubuntu/env/lib/python3.7/distutils/core.py", line 148, in setupdist.run_commands()File "/home/ubuntu/env/lib/python3.7/distutils/dist.py", line 966, in run_commandsself.run_command(cmd)
File "/home/ubuntu/env/lib/python3.7/distutils/dist.py", line 985, in run_commandcmd_obj.run()File "/home/ubuntu/env/lib/python3.7/site-packages/setuptools/command/install.py", line 67, in runself.do_egg_install()File "/home/ubuntu/env/lib/python3.7/site-packages/setuptools/command/install.py", line 109, in do_egg_installself.run_command('bdist_egg')
File "/home/ubuntu/env/lib/python3.7/distutils/cmd.py", line 313, in run_command

self.distribution.run_command(command)
File "/home/ubuntu/env/lib/python3.7/distutils/dist.py", line 985, in run_commandcmd_obj.run()File "setup.py", line 240, in runfind_or_build_libtiledbvcf(self)
File "setup.py", line 168, in find_or_build_libtiledbvcfbuild_libtiledbvcf()
File "setup.py", line 153, in build_libtiledbvcfsubprocess.check_call(build_cmd, cwd=build_dir)
File "/home/ubuntu/env/lib/python3.7/subprocess.py", line 363, in check_callraise CalledProcessError(retcode, cmd)subprocess.CalledProcessError: Command '['make', '-j4']' returned non-zero exit status 2

Then I tried Standalone installation(https://docs.tiledb.com/genomics/installation/standalone-tiledb-vcf) too and faced same issue executing this command: cmake … && make -j8.

CMake Error at /home/ubuntu/tiledb/TileDB-VCF/libtiledbvcf/build/externals/src/ep_tiledb-stamp/ep_tiledb-build-RELEASE.cmake:16 (message):Command failed: 2'make'

See also
/home/ubuntu/tiledb/TileDB-VCF/libtiledbvcf/build/externals/src/ep_tiledb-stamp/ep_tiledb-build-*.logCMakeFiles/ep_tiledb.dir/build.make:111: recipe for target 'externals/src/ep_tiledb-stamp/ep_tiledb-build' failedmake[2]: *** [externals/src/ep_tiledb-stamp/ep_tiledb-build] Error 1
CMakeFiles/Makefile2:99: recipe for target 'CMakeFiles/ep_tiledb.dir/all' failedmake[1]: *** [CMakeFiles/ep_tiledb.dir/all] Error 2
Makefile:83: recipe for target 'all' failed
make: *** [all] Error 2

Environment:
ubuntu 18.04 on EC2 machine.
Created Conda environment based on the instructions provided on tileDB website.

Thanks.

2

Hi @Veerendar_Konda, thanks for posting!

I was able to reproduce the issue and we’re currently looking into it. In the meantime, you can install both the CLI and Python package from our conda channel:

conda install -c tiledb libtiledbvcf
conda install -c tiledb tiledbvcf-py

We also provide up to date Docker images:

3

Hi @aaron , thanks for reply,

I have tried installing tiledbvcf-py using conda channel that you provided above, but facing UnsatisfiableError issue. tried with multiple version of python 3.5, 3.6, 3.7, 3.8 the issue is persisted across the above mentioned python versions. can you please provide more information on the environment and python version.

Error message:
conda install -c tiledb tiledbvcf-py
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Solving environment: /
Found conflicts! Looking for incompatible packages.
This can take several minutes. Press CTRL-C to abort.
failed

UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:

Specifications:

  • tiledbvcf-py -> python[version=’>=2.7,<2.8.0a0|>=3.8,<3.9.0a0|>=3.5,<3.6.0a0|>=3.9,<3.10.0a0’]

Your python: python=3.7

If python is on the left-most side of the chain, that’s the version you’ve asked for.
When python appears to the right, that indicates that the thing on the left is somehow
not available for the python version you are constrained to. Note that conda will not
change your python version to a different minor version unless you explicitly specify
that.

using python 3.6.10 and AWS Linux1
conda install -c tiledb tiledbvcf-py
Collecting package metadata (current_repodata.json): done
Solving environment: /
The environment is inconsistent, please check the package plan carefully
The following packages are causing the inconsistency:

  • defaults/linux-64::pandas==1.0.1=py36h0573a6f_0
  • defaults/noarch::jupyterlab==1.2.6=pyhf63ae98_0
  • defaults/linux-64::scikit-learn==0.22.1=py36hd81dba3_0
  • defaults/linux-64::python-language-server==0.31.7=py36_0
  • defaults/linux-64::bkcharts==0.2=py36_0
  • defaults/linux-64::nb_conda==2.2.1=py36_0
  • defaults/noarch::numpydoc==0.9.2=py_0
  • defaults/linux-64::pytest-arraydiff==0.3=py36h39e3cac_0
  • defaults/linux-64::bottleneck==1.3.2=py36heb32a55_0
  • defaults/linux-64::pywavelets==1.1.1=py36h7b6447c_0
  • defaults/noarch::pytest-astropy==0.8.0=py_0
  • defaults/linux-64::numexpr==2.7.1=py36h423224d_0
  • defaults/noarch::anaconda-project==0.8.4=py_0
  • defaults/linux-64::nbconvert==5.6.1=py36_0
  • defaults/linux-64::h5py==2.10.0=py36h7918eee_0
  • defaults/linux-64::bokeh==1.4.0=py36_0
  • defaults/noarch::jupyterlab_server==1.0.6=py_0
  • defaults/linux-64::numpy-base==1.18.1=py36hde5b4d6_1
  • defaults/linux-64::jupyter==1.0.0=py36_7
  • defaults/linux-64::astropy==4.0=py36h7b6447c_0
  • defaults/linux-64::patsy==0.5.1=py36_0
  • defaults/linux-64::scikit-image==0.16.2=py36h0573a6f_0
  • defaults/linux-64::matplotlib-base==3.1.3=py36hef1b27d_0
  • defaults/linux-64::imageio==2.6.1=py36_0
  • defaults/linux-64::pytables==3.6.1=py36h71ec239_0
  • defaults/linux-64::nb_conda_kernels==2.2.4=py36_0
  • defaults/linux-64::mkl_fft==1.0.15=py36ha843d7b_0
  • defaults/linux-64::statsmodels==0.11.0=py36h7b6447c_0
  • defaults/linux-64::spyder==4.0.1=py36_0
  • defaults/noarch::seaborn==0.10.0=py_0
  • defaults/linux-64::requests==2.22.0=py36_1
  • defaults/linux-64::numba==0.48.0=py36h0573a6f_0
  • defaults/linux-64::scipy==1.4.1=py36h0b6359f_0
  • defaults/noarch::pytest-doctestplus==0.5.0=py_0
  • defaults/linux-64::mkl_random==1.1.0=py36hd6b4f25_0
  • defaults/noarch::dask==2.11.0=py_0
  • defaults/noarch::ipywidgets==7.5.1=py_0
  • defaults/linux-64::widgetsnbextension==3.5.1=py36_0
  • defaults/noarch::s3fs==0.4.2=py_0
  • defaults/linux-64::notebook==6.0.3=py36_0
  • defaults/linux-64::matplotlib==3.1.3=py36_0
  • defaults/linux-64::anaconda-client==1.7.2=py36_0
  • defaults/linux-64::numpy==1.18.1=py36h4f9e942_0
    failed with initial frozen solve. Retrying with flexible solve.
    Solving environment: failed with repodata from current_repodata.json, will retry with next repodata source.
    Collecting package metadata (repodata.json): done
    Solving environment: -
    The environment is inconsistent, please check the package plan carefully
    The following packages are causing the inconsistency:
    \
  • defaults/noarch::jupyterlab==1.2.6=pyhf63ae98_0
  • defaults/linux-64::python-language-server==0.31.7=py36_0
  • defaults/linux-64::nb_conda==2.2.1=py36_0
  • defaults/noarch::numpydoc==0.9.2=py_0
  • defaults/noarch::anaconda-project==0.8.4=py_0
  • defaults/linux-64::nbconvert==5.6.1=py36_0
  • defaults/linux-64::bokeh==1.4.0=py36_0
  • defaults/noarch::jupyterlab_server==1.0.6=py_0
  • defaults/linux-64::jupyter==1.0.0=py36_7
  • defaults/linux-64::scikit-image==0.16.2=py36h0573a6f_0
  • defaults/linux-64::imageio==2.6.1=py36_0
  • defaults/linux-64::nb_conda_kernels==2.2.4=py36_0
  • defaults/linux-64::spyder==4.0.1=py36_0
  • defaults/linux-64::requests==2.22.0=py36_1
  • defaults/noarch::dask==2.11.0=py_0
  • defaults/noarch::ipywidgets==7.5.1=py_0
  • defaults/linux-64::widgetsnbextension==3.5.1=py36_0
  • defaults/noarch::s3fs==0.4.2=py_0
  • defaults/linux-64::notebook==6.0.3=py36_0
  • defaults/linux-64::anaconda-client==1.7.2=py36_0
    failed with initial frozen solve. Retrying with flexible solve.
    Solving environment: \
    Found conflicts! Looking for incompatible packages.
    This can take several minutes. Press CTRL-C to abort.
    failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

Output in format: Requested package -> Available versions

We want to try tileDB using docker though but we are looking more interactive way, like using Jupyter notebooks.

Thanks.

4

Apologies, I should have have mentioned conda needs to be configured to use the conda-forge and bioconda channels. Here’s how to create a new environment that includes the tiledbvcf python package:

conda create -n tiledbvcf
conda activate tiledbvcf
conda install -c conda-forge -c bioconda -c tiledb tiledbvcf-py

Quick test:

python -c "import tiledbvcf; print(tiledbvcf.version)"

Let me know if that works for you.

5

We want to try tileDB using docker though but we are looking more interactive way, like using Jupyter notebooks.

I want to point out that we also have TileDB-VCF preinstalled in the hosted notebooks of TileDB Cloud. If you sign up for TileDB Cloud we give all users $10 in free credits and we have a handful of example notebooks preloaded for VCF use. The example notebooks are under the example/genomics folder when you start the notebook.

Please let us know if you are able to install TileDB-VCF from conda locally or if there is anything else we can help you with.

6

Thanks @Aaron, it worked, I’m able install tiledbvcf !.

7

Hi @Seth, Currently we are evaluating and doing pocs on the viable options, we’ll reach out to you if we need any thing.

Thanks.

8

That’s great, thanks for following-up!

And a friendly heads up: we’re very close to releasing the next major version of TileDB-VCF (v0.7), which uses a new schema for storing the variant data. This new schema improves query performance and paves the way for additional features in the pipeline, however, migrating a dataset from the current schema to the new one will require re-ingesting the data. So if you’re working with a huge dataset you may want to hold off on ingesting the entire thing.

A beta for v0.7 should be available on github in the next couple of days and we expect the final release to follow in the coming weeks.

Thanks for checking out TileDB-VCF and do let us know if you have any questions!