TileDb not working in Docker Windows container

1

Hi,
I’m running a simple java application which performs a dense array creation first and a write next inside a windows docker container and I get these exceptions:

  • using tiledb-java version 0.4.5:
	[TileDB::StorageManager] Error: Cannot open array; Array does not exist
  • using tiledb-java version 0.5.0 and 0.5.1 :
	Caused by: java.lang.UnsatisfiedLinkError: 'long io.tiledb.libtiledb.tiledbJNI.new_tiledb_config_tpp()'
        at io.tiledb.libtiledb.tiledbJNI.new_tiledb_config_tpp(Native Method)
        at io.tiledb.libtiledb.tiledb.new_tiledb_config_tpp(tiledb.java:541)
        at io.tiledb.java.api.Config.<init>(Config.java:68)
        at io.tiledb.java.api.Context.<init>(Context.java:69)
        at tiledb.Test.createArray(Test.java:8)
        at tiledb.Test.main(Test.java:49)
        ... 8 more

Since I don’t get this error running the application outside docker container I suppose it’s strictly related to docker.

The java code fragment is the following

    public static void createArray() throws TileDBError {
        try (Context ctx = new Context();
             Attribute a1 = new Attribute(ctx, "a1", Long.class);
             ArraySchema schema = new ArraySchema(ctx, ArrayType.TILEDB_DENSE)) {
            Dimension<Long> d1 = new Dimension<>(ctx, "d1", Long.class, new Pair<>(1L, 4L), 1L);
            Dimension<Long> d2 = new Dimension<>(ctx, "d2", Long.class, new Pair<>(1L, 4L), 1L);

            // Create getDomain
            Domain domain = new Domain(ctx);
            domain.addDimension(d1);
            domain.addDimension(d2);

            // Create array schema
            schema.setTileOrder(Layout.TILEDB_ROW_MAJOR);
            schema.setCellOrder(Layout.TILEDB_ROW_MAJOR);
            schema.setDomain(domain);
            schema.addAttribute(a1);

            // Check array schema
            schema.check();

            Array.create("my_dense_array", schema);
        }
    }

    public static void writeArray() throws TileDBError {
        try (Context ctx = new Context();
             NativeArray a1Data = new NativeArray(ctx, new long[]{1, 2, 3, 4}, Long.class);
             Array myDenseArray = new Array(ctx, "my_dense_array", QueryType.TILEDB_WRITE);
             Query query = new Query(myDenseArray)) {

            query.setLayout(Layout.TILEDB_GLOBAL_ORDER);
            query.setSubarray(new NativeArray(ctx, new long[]{1, 1, 1, 4}, Long.class));
            query.setBuffer("a1", a1Data);

            query.submit();
        }
    }


    public static void main(String[] args) throws Exception {
        createArray();
        writeArray();
    }

The Dockerfile used to build the image is the following:

FROM winamd64/openjdk:11-jdk-nanoserver-1809

ADD target/testTileDb-1.0-SNAPSHOT.jar C:/App.jar

CMD [ "java","-jar","C:/App.jar" ]

Could you please investigate about it?

Thanks for your help.
Kind regards,

Giordano

2

Hi @ggalanti ,
I was able to reproduce your error locally. Please try running
./gradlew clean assemble and try again.

Let me know if it solves the problem.

Best,
Dimitris

3

Hi Dimitris,
I tried what you suggested but unfortunately it did not work.
I tried also by using maven but i got the same error.

These attempts have been performed by using both version 0.4.5 and 0.5.1 so I still stuck on this issue.

Could you please give me some other hints?

Thanks and regards,
Giordano

4

Hi @ggalanti,

We will put together a runnable example next week.

Best,
Isaiah

5

Hi @ihnorton,
When the runnable examples will be available where could I find them?

Thanks for your time.

P.S. I forgot to mention that the java artifacts used are taken from maven-central

Regards,
Giordano

6

@Dimitris_Staratzis is working on this now. We believe the issue is that libtiledb.dll is not being included in your tiledb-SNAPSHOT.jar; currently the TileDB-Java releases on Windows do not include any libtiledb.dll, so libtiledb needs to be built from source. It looks like we are missing a packaging step to include that local-built DLL in the snapshot, so Dimitris is going to fix the packaging.

He will also try to activate using upstream release DLLs to avoid need for any local compilation.

7

Hi Isaiah,
Thanks for your help!

Regards,
Giordano

8

Hi @ggalanti,

This one was a trickier than we had expected. The issue is that the nanoserver container image is very minimal and, as such, it does not include a number of standard Windows DLLs which are linked by the TileDB DLL files (libtiledb and the JNI wrapper).

As a baseline, we’ve put together an example using the (larger) windowsservercore JDK image:

If you have a hard dependency on the nanoserver image, I would suggest to go through the dependencies for tiledb.dll and tiledbjni.dll (inside the JAR file), and copy those from C:\Windows\system32 one by one into a running image until the DLL loading is satisfied. You could see the necessary DLLs with https://github.com/lucasg/Dependencies, for example.

Hope this helps,
Isaiah

9

Hi @ihnorton,
everything clear, I’ll work on it!

Thanks for you support.

Regards,
Giordano