Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Build failing due to incorrect hadoop-common jar version


Copy link to this message
-
Re: Build failing due to incorrect hadoop-common jar version
I can confirm your observation as well for trunk (1.5.0)

The dependency com.cloudera.cdk:cdk-morphlines-solr-core:jar was upgraded
since the last release (1.4.0)

Here are the changes between 1.4.0 and trunk (1.5.0)

1.4.0 - com.cloudera.cdk:cdk-morphlines-solr-core:jar:0.4.0:compile via
com.cloudera.cdk:cdk-morphlines-solr-cell:jar:0.4.0:compile
1.5.0 - com.cloudera.cdk:cdk-morphlines-solr-core:jar:0.6.0:compile via
com.cloudera.cdk:cdk-morphlines-all:pom:0.6.0:compile

The new version of the morphline introduces the hadoop dependency that I
dont think is actually used within the module.

If you exclude the hadoop dependency from the module's pom file, the build
runs fine.

So, you can open a JIRA issue for the defect and submit a patch with the
exclusion of the dependency instead.

<exclusions>
    <exclusion>
        <groupId>org.apache.hadoop</groupId>
        <artifactId>hadoop-common</artifactId>
    </exclusion>
</exclusions>

This could also have been a typo in the chain of dependencies because
according to maven [1] only hadoop-core has a version 1.0.1 available.

modified:   flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml
diff --git a/flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml
b/flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml
index fd6b4ea..3a7fb3d 100644
--- a/flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml
+++ b/flume-ng-sinks/flume-ng-morphline-solr-sink/pom.xml
@@ -78,6 +78,12 @@ limitations under the License.
       <groupId>com.cloudera.cdk</groupId>
       <artifactId>cdk-morphlines-all</artifactId>
       <version>${cdk.version}</version>
+      <exclusions>
+        <exclusion>
+            <groupId>org.apache.hadoop</groupId>
+            <artifactId>hadoop-common</artifactId>
+            </exclusion>
+      </exclusions>
       <type>pom</type>
       <optional>true</optional>
     </dependency>

[1] http://mvnrepository.com/artifact/org.apache.hadoop/hadoop-core
*Author and Instructor for the Upcoming Book and Lecture Series*
*Massive Log Data Aggregation, Processing, Searching and Visualization with
Open Source Software*
*http://massivelogdata.com*
On Sat, Sep 7, 2013 at 9:46 PM, Jagat Singh <[EMAIL PROTECTED]> wrote:

> Hello everyone,
>
> Downloaded the code from flume git repo.
>
> Trying to build it
>
>
> mvn clean install -DskipTests
>
> It gives me this error.
>
> The project expects 1.0.1 version of jar which was never there at maven
> central.
>
>
> http://search.maven.org/#search|gav|1|g%3A%22org.apache.hadoop%22%20AND%20a%3A%22hadoop-common%22
>
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-remote-resources-plugin:1.1:process
> (default) on project flume-ng-morphline-solr-sink: Failed to resolve
> dependencies for one or more projects in the reactor. Reason: Missing:
> [ERROR] ----------
> [ERROR] 1) org.apache.hadoop:hadoop-common:jar:1.0.1
> [ERROR]
> [ERROR] Try downloading the file manually from the project website.
>
>
>
> [ERROR] Path to dependency:
> [ERROR] 1)
>
> org.apache.flume.flume-ng-sinks:flume-ng-morphline-solr-sink:jar:1.5.0-SNAPSHOT
> [ERROR] 2) com.cloudera.cdk:cdk-morphlines-all:pom:0.6.0
> [ERROR] 3) com.cloudera.cdk:cdk-morphlines-solr-core:jar:0.6.0
> [ERROR] 4) org.apache.solr:solr-core:jar:4.4.0
> [ERROR] 5) org.apache.hadoop:hadoop-common:jar:1.0.1
>
>
>
> ----------------
>
>
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-remote-resources-plugin:1.1:process
> (default) on project flume-ng-dist: Failed to resolve dependencies for one
> or more projects in the reactor. Reason: Missing:
> [ERROR] ----------
> [ERROR] 1) org.apache.hadoop:hadoop-common:jar:1.0.1
>
>
> [ERROR] Path to dependency:
> [ERROR] 1) org.apache.flume:flume-ng-dist:pom:1.5.0-SNAPSHOT
> [ERROR] 2)
>
> org.apache.flume.flume-ng-sinks:flume-ng-morphline-solr-sink:jar:1.5.0-SNAPSHOT
> [ERROR] 3) org.apache.hadoop:hadoop-common:jar:1.0.1
>
>
>
> To fix the errors I added upfront dependency to latest hadoop common in
>
> pom.xml for both these projects