Build error in upgrading tiledb-presto with presto 340

Hi,

I want to build the tiledb-presto with presto 340 version. To upgrade this, I have changed presto version in pom.xml.

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
    <parent>
        <artifactId>presto-root</artifactId>
        <groupId>io.prestosql</groupId>
        <version>340</version>
    </parent>
    <modelVersion>4.0.0</modelVersion>

    <artifactId>presto-tiledb</artifactId>
    <description>Presto - TileDB Connector</description>

    <dependencies>
        <dependency>
            <groupId>io.tiledb</groupId>
            <artifactId>tiledb-java</artifactId>
            <version>0.3.2</version>
        </dependency>

        <dependency>
            <groupId>io.airlift</groupId>
            <artifactId>bootstrap</artifactId>
        </dependency>

        <dependency>
            <groupId>io.airlift</groupId>
            <artifactId>json</artifactId>
        </dependency>

        <dependency>
            <groupId>io.airlift</groupId>
            <artifactId>log</artifactId>
        </dependency>

        <dependency>
            <groupId>io.airlift</groupId>
            <artifactId>configuration</artifactId>
        </dependency>

        <dependency>
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
        </dependency>

        <dependency>
            <groupId>com.google.inject</groupId>
            <artifactId>guice</artifactId>
        </dependency>

        <dependency>
            <groupId>javax.inject</groupId>
            <artifactId>javax.inject</artifactId>
        </dependency>

        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-databind</artifactId>
        </dependency>

        <!-- Presto SPI -->
        <dependency>
            <groupId>io.prestosql</groupId>
            <artifactId>presto-spi</artifactId>
            <scope>provided</scope>
            <!-- https://mvnrepository.com/artifact/commons-beanutils/commons-beanutils -->
            <exclusions>
                <exclusion>
                    <groupId>commons-beanutils</groupId>
                    <artifactId>commons-beanutils-core</artifactId>
                </exclusion>
            </exclusions>
        </dependency>

        <dependency>
            <groupId>io.airlift</groupId>
            <artifactId>slice</artifactId>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>io.airlift</groupId>
            <artifactId>units</artifactId>
            <scope>provided</scope>
        </dependency>

        <dependency>
            <groupId>com.fasterxml.jackson.core</groupId>
            <artifactId>jackson-annotations</artifactId>
            <scope>provided</scope>
        </dependency>

        <!-- for testing -->
        <dependency>
            <groupId>io.prestosql</groupId>
            <artifactId>presto-main</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>org.testng</groupId>
            <artifactId>testng</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>io.airlift</groupId>
            <artifactId>testing</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>com.google.code.findbugs</groupId>
            <artifactId>jsr305</artifactId>
            <optional>true</optional>
        </dependency>

        <dependency>
            <groupId>io.prestosql</groupId>
            <artifactId>presto-tpch</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>io.airlift.tpch</groupId>
            <artifactId>tpch</artifactId>
            <version>0.10</version>
            <scope>test</scope>
            <!--  <version>0.10</version>  -->
        </dependency>

        <dependency>
            <groupId>joda-time</groupId>
            <artifactId>joda-time</artifactId>
        </dependency>

        <!-- https://mvnrepository.com/artifact/commons-beanutils/commons-beanutils -->
        <dependency>
            <groupId>commons-beanutils</groupId>
            <artifactId>commons-beanutils-core</artifactId>
            <version>1.8.3</version>
            <exclusions>
                <exclusion>
                    <groupId>commons-logging</groupId>
                    <artifactId>commons-logging</artifactId>
                </exclusion>
            </exclusions>
        </dependency>
        <dependency>
            <groupId>io.prestosql</groupId>
            <artifactId>presto-tests</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.jetbrains</groupId>
            <artifactId>annotations</artifactId>
            <scope>test</scope>
        </dependency>

        <dependency>
            <groupId>com.github.oshi</groupId>
            <artifactId>oshi-core</artifactId>
            <version>3.10.0</version>
        </dependency>
    </dependencies>

    <build>
        <plugins>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-surefire-plugin</artifactId>
                <configuration>
                    <useSystemClassLoader>false</useSystemClassLoader>
                </configuration>
            </plugin>
            <plugin>
                <groupId>org.apache.maven.plugins</groupId>
                <artifactId>maven-shade-plugin</artifactId>
                <version>3.2.1</version>
                <configuration>
                    <!-- put your configurations here -->
                    <packaging>presto-plugin</packaging>
                    <transformers>
                        <transformer implementation="org.apache.maven.plugins.shade.resource.ServicesResourceTransformer"/>
                        <transformer implementation="org.apache.maven.plugins.shade.resource.IncludeResourceTransformer">
                            <resource>META-INF/services/io.prestosql.spi.Plugin</resource>
                            <file>src/main/resources/services/io.prestosql.spi.Plugin</file>
                        </transformer>
                    </transformers>
                </configuration>
                <executions>
                    <execution>
                        <phase>package</phase>
                        <goals>
                            <goal>shade</goal>
                        </goals>
                    </execution>
                </executions>
            </plugin>
        </plugins>
    </build>

    <profiles>
        <profile>
            <id>ci</id>
            <build>
                <plugins>
                    <plugin>
                        <groupId>org.apache.maven.plugins</groupId>
                        <artifactId>maven-surefire-plugin</artifactId>
                        <configuration>
                            <excludes combine.self="override" />
                        </configuration>
                    </plugin>
                </plugins>
            </build>
        </profile>
    </profiles>

    <repositories>
        <repository>
            <id>oss.sonatype.org-snapshot</id>
            <url>http://oss.sonatype.org/content/repositories/snapshots</url>
            <releases>
                <enabled>false</enabled>
            </releases>
            <snapshots>
                <enabled>true</enabled>
            </snapshots>
        </repository>
    </repositories>
</project>

Below is the error that I am getting:-

[WARNING] Some problems were encountered while building the effective settings
[WARNING] expected START_TAG or END_TAG not TEXT (position: TEXT seen ...tps://maven.apache.org/xsd/settings-1.0.0.xsd">\n      ...\n      <p... @6:9)  @ /home
/shashikant/.m2/settings.xml, line 6, column 9
[WARNING]
[INFO] Scanning for projects...
[INFO]
[INFO] ---------------------< io.prestosql:presto-tiledb >---------------------
[INFO] Building presto-tiledb 340
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (default) @ presto-tiledb ---
[INFO] Ignoring requireUpperBoundDeps in org.codehaus.plexus:plexus-utils
[WARNING] Rule 0: com.github.ferstl.maven.pomenforcers.CompoundPedanticEnforcer failed with message:

###################################################
# COMPOUND: One does not simply write a POM file! #
###################################################

Please fix these problems:

POM_SECTION_ORDER: One does not simply write a POM file!
========================================================

Your POM has to be organized this way:

Actual Order   | Required Order
-------------------------------
- parent       |
  modelVersion |   modelVersion
               | + parent
  artifactId   |   artifactId
  description  |   description
               | + repositories
  dependencies |   dependencies
  build        |   build
  profiles     |   profiles
- repositories |


DEPENDENCY_ORDER: One does not simply declare dependencies!
===========================================================

Your dependencies have to be sorted this way:

Actual Order                                                         | Required Order
-------------------------------------------------------------------------------------------------------------------------------------------
- io.tiledb:tiledb-java:0.3.2:jar:compile                            |
  io.airlift:bootstrap:0.198:jar:compile                             |   io.airlift:bootstrap:0.198:jar:compile
                                                                     | + io.airlift:configuration:0.198:jar:compile
  io.airlift:json:0.198:jar:compile                                  |   io.airlift:json:0.198:jar:compile
  io.airlift:log:0.198:jar:compile                                   |   io.airlift:log:0.198:jar:compile
- io.airlift:configuration:0.198:jar:compile                         | + com.fasterxml.jackson.core:jackson-databind:2.10.4:jar:compile
                                                                     | + com.github.oshi:oshi-core:3.10.0:jar:compile
                                                                     | + com.google.code.findbugs:jsr305:3.0.2:jar:compile
  com.google.guava:guava:29.0-jre:jar:compile                        |   com.google.guava:guava:29.0-jre:jar:compile
  com.google.inject:guice:4.2.3:jar:compile                          |   com.google.inject:guice:4.2.3:jar:compile
                                                                     | + commons-beanutils:commons-beanutils-core:1.8.3:jar:compile
                                                                     | + io.tiledb:tiledb-java:0.3.2:jar:compile
  javax.inject:javax.inject:1:jar:compile                            |   javax.inject:javax.inject:1:jar:compile
- com.fasterxml.jackson.core:jackson-databind:2.10.4:jar:compile     | + joda-time:joda-time:2.10.6:jar:compile
  io.prestosql:presto-spi:340:jar:provided                           |   io.prestosql:presto-spi:340:jar:provided
  io.airlift:slice:0.39:jar:provided                                 |   io.airlift:slice:0.39:jar:provided
  io.airlift:units:1.6:jar:provided                                  |   io.airlift:units:1.6:jar:provided
  com.fasterxml.jackson.core:jackson-annotations:2.10.4:jar:provided |   com.fasterxml.jackson.core:jackson-annotations:2.10.4:jar:provided
  io.prestosql:presto-main:340:jar:test                              |   io.prestosql:presto-main:340:jar:test
- org.testng:testng:6.10:jar:test                                    | + io.prestosql:presto-tests:340:jar:test
- io.airlift:testing:0.198:jar:test                                  |
- com.google.code.findbugs:jsr305:3.0.2:jar:compile                  |
  io.prestosql:presto-tpch:340:jar:test                              |   io.prestosql:presto-tpch:340:jar:test
                                                                     | + io.airlift:testing:0.198:jar:test
  io.airlift.tpch:tpch:0.10:jar:test                                 |   io.airlift.tpch:tpch:0.10:jar:test
- joda-time:joda-time:2.10.6:jar:compile                             |
- commons-beanutils:commons-beanutils-core:1.8.3:jar:compile         |
- io.prestosql:presto-tests:340:jar:test                             |
  org.jetbrains:annotations:19.0.0:jar:test                          |   org.jetbrains:annotations:19.0.0:jar:test
- com.github.oshi:oshi-core:3.10.0:jar:compile                       | + org.testng:testng:6.10:jar:test


[WARNING] Rule 5: org.apache.maven.plugins.enforcer.RequireJavaVersion failed with message:
Detected JDK Version: 1.8.0-265 is not in the allowed range 11.
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  2.262 s
[INFO] Finished at: 2020-09-01T17:26:47+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-enforcer-plugin:3.0.0-M3:enforce (default) on project presto-tiledb: Some Enforcer rules have fai
led. Look above for specific messages explaining why the rule failed. -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionException

Can any one please help me in this?
@seth @ihnorton

After modifying the pom.xml many of error are removed. But got some new error:-

[WARNING]
[WARNING] Some problems were encountered while building the effective settings
[WARNING] expected START_TAG or END_TAG not TEXT (position: TEXT seen ...tps://maven.apache.org/xsd/settings-1.0.0.xsd">\n      ...\n      <p... @6:9)  @ /hom$
/shashikant/.m2/settings.xml, line 6, column 9
[WARNING]
[INFO] Scanning for projects...
[INFO]
[INFO] ---------------------< io.prestosql:presto-tiledb >---------------------
[INFO] Building presto-tiledb 340
[INFO] --------------------------------[ jar ]---------------------------------
[INFO]
[INFO] --- maven-enforcer-plugin:3.0.0-M3:enforce (default) @ presto-tiledb ---
[INFO] Ignoring requireUpperBoundDeps in org.codehaus.plexus:plexus-utils
[INFO]
[INFO] --- dependency-scope-maven-plugin:0.10:check (default) @ presto-tiledb ---
[INFO] No test dependency scope issues found
[INFO]
[INFO] --- license-maven-plugin:2.3:check (default) @ presto-tiledb ---
[INFO] Checking licenses...
[INFO]
[INFO] --- maven-checkstyle-plugin:3.1.1:check (checkstyle) @ presto-tiledb ---
[INFO] You have 0 Checkstyle violations.
[INFO]
[INFO] --- jacoco-maven-plugin:0.8.5:prepare-agent (default) @ presto-tiledb ---
[INFO] Skipping JaCoCo execution because property jacoco.skip is set.
[INFO] argLine set to empty
[INFO]
[INFO] --- git-commit-id-plugin:4.0.0:revision (default) @ presto-tiledb ---
[INFO]
[INFO] --- maven-resources-plugin:2.6:resources (default-resources) @ presto-tiledb ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 1 resource
[INFO]
[INFO] --- maven-compiler-plugin:3.8.0:compile (default-compile) @ presto-tiledb ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 27 source files to /home/shashikant/projects/tiledb-presto/target/classes
[INFO] /home/shashikant/projects/tiledb-presto/src/main/java/io/prestosql/plugin/tiledb/TileDBMetadata.java: Some input
files use or override a deprecated API.
[INFO] /home/shashikant/projects/tiledb-presto/src/main/java/io/prestosql/plugin/tiledb/TileDBMetadata.java: Recompile $
ith -Xlint:deprecation for details.
[INFO] /home/shashikant/projects/tiledb-presto/src/main/java/io/prestosql/plugin/tiledb/TileDBMetadata.java: /ho[0/1873]
3/projects/airflow-example/src/worker_shashi_20200830/tiledb-presto/src/main/java/io/prestosql/plugin/tiledb/TileDBMetadata.java uses unchecked or unsafe opera
tions.
[INFO] /home/shashikant/projects/tiledb-presto/src/main/java/io/prestosql/plugin/tiledb/TileDBMetadata.java: Recompile $
ith -Xlint:unchecked for details.
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /home/shashikant/projects/tiledb-presto/src/main/java/io/prestosql/plugin/tiledb/TileDBModule.java:[44,1] canno$
 find symbol
  symbol:   static parseTypeSignature
  location: class io.prestosql.spi.type.TypeSignature
[ERROR] /home/shashikant/projects/tiledb-presto/src/main/java/io/prestosql/plugin/tiledb/TileDBModule.java:[99,40] cann$
t find symbol
  symbol:   method parseTypeSignature(java.lang.String)
  location: class io.prestosql.plugin.tiledb.TileDBModule.TypeDeserializer
[INFO] 2 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  5.731 s
[INFO] Finished at: 2020-09-02T14:04:04+05:30
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project presto-tiledb: Compilation failure: C$
mpilation failure:
[ERROR] /home/shashikant/projects/tiledb-presto/src/main/java/io/prestosql/plugin/tiledb/TileDBModule.java:[44,1] canno$
 find symbol
[ERROR]   symbol:   static parseTypeSignature
[ERROR]   location: class io.prestosql.spi.type.TypeSignature
[ERROR] /home/shashikant/projects/tiledb-presto/src/main/java/io/prestosql/plugin/tiledb/TileDBModule.java:[99,40] cann$
t find symbol
[ERROR]   symbol:   method parseTypeSignature(java.lang.String)
[ERROR]   location: class io.prestosql.plugin.tiledb.TileDBModule.TypeDeserializer
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException

@victor_tiledb

Hi @Shashikant255. Thank you for reporting this. We are working towards a new release that includes support for all TileDB datetime types. The issue you report will be our very next step after that release.

Best,
Victor

Hi @victor_tiledb,
In master branch there is support for presto340 that has been pushed 3days ago. I have pulled that branch and able to compile it with presto 340 (some error due to version mismatch of maven, changed it to 3.6.3 and works well). With this I am able to query it to create table on local. But when I use same query to create array on s3 it fails.

CREATE TABLE region(
  regionkey bigint WITH (dimension=true),
  name varchar,
  comment varchar
  ) WITH (uri = 's3://bucket/region')

This fails and displays below error:-

Query 20200918_113745_00008_gm339 failed: [TileDB::VFS] Error: TileDB was built without S3 support

Can you please tell me what I have missed?

@Shashikant255 we are working on a tweak to the datetime support now, which should be soon. We’ll then release a new version with an updated TileDB-Java version used. We have been making some improvements to the prebuilt jar for TileDB-Java which should fix this problem. We’ll have a new release in the next few hours.

@Shashikant255 We just released a new version of TileDB Presto. This release includes the updated presto version to 340, as well as the latest TileDB-Java which is built with S3 support. Here is the link: https://github.com/TileDB-Inc/TileDB-Presto/releases/tag/1.6.2

Please let us know if you face any issues.

Hi @victor_tiledb ,
I have compiled the tiledb-presto 1.6.2 version. I am able to compile it with presto 340. But still I am having same issue. It still gives same error message.

Query 20200923_141954_00014_92u9w failed: [TileDB::VFS] Error: TileDB was built without S3 support

Do I need to set some flag before compiling it in order to build it with S3 support?

Hi @Shashikant255! Could you try the 1.6.3 version?

Let us know for any further issues.

Thanks for your quick response @victor_tiledb.
I have pulled 1.6.3 version and with this I am able to query or create array on S3.

Thanks for your help!!!