Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> Question on hadoop dependencies.


+
Petar Tahchiev 2013-10-29, 19:22
+
Steve Loughran 2013-10-30, 12:54
Copy link to this message
-
Re: Question on hadoop dependencies.
Oh, hi Steve,

didn't know you were on this list :) ...

So spring-data-solr (1.1.SNAPSHOT) uses solr 4.5.1 (just came out a few
days ago), which uses Hadoop 2.0.5-alpha.
I would be glad if we can clean up the poms a bit and leave only the
dependencies
that hadoop really depend on.

I'll drop an email to the solr guys and see what exactly are they using
hadoop for.

Cheers, Petar.

2013/10/30 Steve Loughran <[EMAIL PROTECTED]>

> Why hello Peter,
>
> Which version are you using?
>
> The reason those dependencies are declared are because things like Jetty
> use them -and the classpath for the server side Hadoop is "things needed to
> run Hadoop".
>
> Client-side I think there's too much in the maven dependency tree
> (servlets, jetty, ...)
>
>
>
> On 29 October 2013 19:22, Petar Tahchiev <[EMAIL PROTECTED]> wrote:
>
> > Hi guys,
> >
> > I'm using Spring-data-solr in my project. SDS declares Solr-core as a
> > dependency. Solr declares hadoop-auth, hadoop-common, hadoop-hdfs
> > dependencies. Each and every one of those dependencies declares log4j and
> > slf4j-log4j12 as runtime dependencies, and also hadoop-common declares
> > servlet-api version 2.5 as runtime dependency. So in the end I also get
> > servlet-api 2.5, slf4 and log4j in my classpath.
> > This normally shouldn't be a problem, but in my case I'm using Servlet
> 3.0,
> > log4j2, and SL4j for log4j2. This completely messes up my classpath, so I
> > have to manually exclude those dependencies, like this:
> > -----------------------------------------------
> >         <dependency>
> >             <groupId>org.apache.solr</groupId>
> >             <artifactId>solr-core</artifactId>
> >             <version>${solr.version}</version>
> >             <exclusions>
> >                 <exclusion>
> >                     <groupId>org.slf4j</groupId>
> >                     <artifactId>slf4j-jdk14</artifactId>
> >                 </exclusion>
> >                 <exclusion>
> >                     <groupId>org.apache.solr</groupId>
> >                     <artifactId>solr-core</artifactId>
> >                 </exclusion>
> >             </exclusions>
> >         </dependency>
> >         <dependency>
> >             <groupId>org.apache.solr</groupId>
> >             <artifactId>solr-core</artifactId>
> >             <version>${solr.version}</version>
> >             <exclusions>
> >                 <exclusion>
> >                     <groupId>org.apache.hadoop</groupId>
> >                     <artifactId>hadoop-auth</artifactId>
> >                 </exclusion>
> >                 <exclusion>
> >                     <groupId>org.apache.hadoop</groupId>
> >                     <artifactId>hadoop-common</artifactId>
> >                 </exclusion>
> >                 <exclusion>
> >                     <groupId>org.apache.hadoop</groupId>
> >                     <artifactId>hadoop-hdfs</artifactId>
> >                 </exclusion>
> >             </exclusions>
> >         </dependency>
> >         <dependency>
> >             <groupId>org.apache.hadoop</groupId>
> >             <artifactId>hadoop-auth</artifactId>
> >             <version>${hadoop.version}</version>
> >             <exclusions>
> >                 <exclusion>
> >                     <groupId>log4j</groupId>
> >                     <artifactId>log4j</artifactId>
> >                 </exclusion>
> >                 <exclusion>
> >                     <groupId>org.slf4j</groupId>
> >                     <artifactId>slf4j-log4j12</artifactId>
> >                 </exclusion>
> >             </exclusions>
> >         </dependency>
> >         <dependency>
> >             <groupId>org.apache.hadoop</groupId>
> >             <artifactId>hadoop-common</artifactId>
> >             <version>${hadoop.version}</version>
> >             <exclusions>
> >                 <exclusion>
> >                     <groupId>log4j</groupId>
> >                     <artifactId>log4j</artifactId>
> >                 </exclusion>

Regards, Petar!
Karlovo, Bulgaria.
Public PGP Key at:
https://keyserver1.pgp.com/vkd/DownloadKey.event?keyid=0x19658550C3110611
Key Fingerprint: A369 A7EE 61BC 93A3 CDFF  55A5 1965 8550 C311 0611
+
Steve Loughran 2013-10-30, 20:07
+
Roman Shaposhnik 2013-10-30, 21:52
+
Petar Tahchiev 2013-10-30, 22:25
+
Steve Loughran 2013-11-13, 20:51
+
Petar Tahchiev 2013-11-13, 21:15
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB